You will be redirected back to your article in seconds
Back to IndieWire

The Downside of Measuring the Social Impact of Documentary Films

The Downside of Measuring the Social Impact of Documentary Films

Earlier this year, the firm that I founded, Aggregate, partnered with the organizers of the True/False Film Fest to conduct a survey of the filmmakers whose films screened at the Fest in 2014. True/False is a well-regarded documentary film festival, particularly among filmmakers who often talk about how well the festival organizers treat them and the obvious regard the organizers have for the art of storytelling.

The goal of our survey was to understand how these filmmakers felt about their films’ potential contribution to social change, any intentions they had to capitalize on that potential, as well as their views regarding measuring the social impact the films could have. While True/False is not specifically a social change documentary film festival, of those who responded to the survey, 72 percent believed that the film they screened at the 2014 Fest could contribute to social change.

READ MORE: How Do We Measure the Impact of Documentaries? Data from the Puma Impact Award Nominees

As we were getting ready to share the outcomes of the survey, The New York Times reported on Participant Media’s efforts to establish an index that would enable them – and others who invest in social change films – to determine which films “spur activism” and those, seemingly, which do not. The Participant Index, based on my understanding from the article, measures the ability of a film to inspire “emotional involvement,” as well as its ability to “provoke action.” So, while a film may generate an intense emotional response, if it does not also lead the audience to take action, it would generate a lower score and, perhaps, not receive the support of funders that want to invest in action that they believe will lead to social change.

It should not be a surprise to you that the filmmakers we survey expressed concern about measuring the social impact of their films: 66 percent were opposed to using metrics. And, while I believe very strongly in measuring impact, I share some of their concerns. If the “effectiveness”of storytelling is, for example, defined by actions taken by audience members in the short term, this could lead to a bias toward investing in didactic films (“This is how you should feel and this is what you should do.”) about issues with relatively clear paths toward resolution. And that’s not good storytelling.

My concern is that the path being taken by foundations and other funders of social change storytelling to demonstrate a return on investment – while paved with good intentions – has made a wrong turn.

Despite their belief in the potential of their films to contribute to social change, 56 percent of the filmmakers we surveyed indicated that they had no plans to do outreach specifically to increase the social impact of their films. Why? 42 percent said they didn’t have the time or budget to do so and 15 percent said they simply didn’t know how.

This was not a surprise. A few years ago – during a panel at True/False on social change films – I heard Steve James, the award-winning filmmaker of “Hoop Dreams,” “The Interrupters” and the newly released “Life Itself,” tell the crowd that, despite the social issues addressed in many of his films, he was a filmmaker and not an advocate. He felt strongly that he should leave it to those who know how to make change happen to determine how to use his films to do so.

What I believe is that (more) foundations should invest (more) in outreach strategies for documentary films to take (better) advantage of their emotional impact. Filmmakers could be involved in the development of these strategies, but advocates with knowledge of the issues, the influencers and the solutions – including foundations’ current grant recipients – should design them. And I believe we should measure the impact of THOSE investments, using metrics that assess REAL outcomes.

Yes, let’s measure the emotional impact of a film: the anger, the sadness, the sense of injustice, or the empathy or admiration for those who are suffering or who are creating change. But do not let us dismiss films that do not, on their own, provoke immediate action as being less than integral to social change. Think of the stories that have introduced you to new ideas, new worlds and new people. Think of the stories that have challenged your previously held beliefs. They are no less valuable to society because you didn’t sign a petition after viewing them.

There was an intriguing line in the New York Times article about the issues that were most likely to compel Participant audiences to take action. “Stories about animal rights and food production, it turned out, were the most likely to provoke individual action. But tales about economic inequality — not so much.”

In developing social change strategies to mobilize people to take action, one of the most important things to do is to understand where people are in relationship to the issue being addressed. Are they aware of it? Do they care about it? If they do care, do they understand the factors that allow it to persist or which led it to exist in the first place? If they do understand, do they believe that they – as individuals – have the capacity to impact those factors?

Sure, stories about animal rights and food production are more likely to provoke individual action – because people think they, as an individual, can stop going to SeaWorld or can start shopping locally. But ending economic disparities? Most audiences need a little help to believe they alone can do anything to have an impact on what most of us perceive as an intractable problem. Enabling people to understand how they could have a TRULY meaningful impact requires understanding the players and the policies that need to be changed. THIS is the job of the advocate: to take the baton from the filmmaker – who has generated the energy – and to share their knowledge and know how – to direct that energy wisely.

“Films are films,” wrote one filmmaker in response to our survey. “If they are a visually interesting experience, spark conversation and inspire people to engage new ideas, they’re successful. Films should not be reduced to advertisements, no matter how worthy the cause. They need to exist on their own terms. If they’re good, they’ll get people thinking.”

Byrne Fields is the Founder and President of Aggregate, a creative
strategy group in Seattle. Aggregate works with nonprofits, foundations,
authors and filmmakers to bring people and resources together to create
social change. She was also once a Google Trend. You can reach her on Twitter at @abfdc.

This Article is related to: Filmmaker Toolkit and tagged , ,


Pamela Yates

Allison, thanks for this thoughtful article. You say, “Filmmakers could be involved in the development of these strategies, but advocates with knowledge of the issues, the influencers and the solutions…should design them.” But it was actually we, the independent social issue filmmakers who have been doing this for decades – it’s part of our DNA as filmmakers understanding how to work with movement partners and developing theories of change – and we are the ones who went to foundations and built relationships by educating them on why they should fund documentary films that move people emotionally and to action. Those who make other kinds of documentaries must now do the same. Documentary film genres are exploding, we can be and should be inclusive to all.


On behalf of the artistic documentary community, I find this debate amusing. No matter how many awards our films may collect, we consistently concede funding to social issue films that are deemed more important. So-called social issue filmmakers are quick to fill grant applications and investor packets with promises of how they’re going to enact change and engage their audiences, yet the moment someone finds a way to actually measure those promises, they backtrack and call themselves storytellers. Some of them, of course, are indeed great storytellers, but Participant has a point – there are an absurd number of mediocre social issue films taking funding from more worthy projects. If we start holding these films accountable, perhaps the full spectrum of documentary will finally have a chance to grow.


As doc filmmaker, I have noticed this trend and agree whole heartedly with Alison. And with funding increasingly tied to such outcomes, it paints a bleak picture for the future of creative non-issue-based documentaries. I am producing two feature docs right now, one social/environmental issue and one not. The non-issue doc has a far more unique subject and will be a much more engaging film…. guess which one is more difficult to raise funds for. Who doesn’t want to see docs change the world? We all do, but that shouldn’t be the only metric for success, funding, or awards as it is increasingly becoming.

Amy Slotnick

As someone who handles outreach and community engagement for social issue films, I agree with Alison that metrics do not necessarily reflect a film’s impact or effective storytelling. It often reflects its budget. It is difficult enough for filmmakers to make their films and handling outreach may not be in their skill set. I think it is important to consider the target audience and their level of motivation to take action. It doesn’t necessarily apply to every film. However, for an independent doc or narrative, reaching your target audience, with or without a call to action, is still essential to build the audience.

Your email address will not be published. Required fields are marked *