Since I began my position as media impact analyst at The Center for Investigative Reporting last July, I’ve connected with many people working to measure, analyze and maximize the impact of media. Through phone calls, Skype meetings, Google Hangouts and emails, my contacts have generously shared their strategies, best practices and frustrations. While some were already connected through professional and social networks, when I began there wasn’t a coherent community of practice: journalists were in one silo, documentary filmmakers in another, foundations occupying a third, academics in the ivory tower and so on.
It was clear that my peers were facing many of the same challenges around how to define, track and measure impact. There was always a lot of “Do you know Jessica in New York?” or “Have you spoken with Sarah in D.C.?” when it came to suggesting people to talk to about measuring impact. Also, I found I was repeating myself a lot in explaining my approach to impact analysis.
To draw together these disparate conversations in one place and set a shared agenda with others committed to examining questions about media impact, CIR created and hosted its first Dissection: Impact event in Oakland, Calif., in October. Media makers, academics and foundation representatives attended the event to “dissect” both the concept of impact and tools for measuring it. The first Dissection was experimental, but the spontaneous combustion of ideas assured us we had touched on something valuable. CIR hosted Dissection B: Impact in January at Mercer University’s Center for Collaborative Journalism in Macon, Ga.
A few weeks ago, CIR hosted two more Dissection: Impact events: one in New York City and the other in Washington, D.C. Participants from the first two Dissections rejoined us, along with a new cohort of individuals and organizations – especially from public media – who infused the events with new ideas and concerns. During the course of these thought-provoking gatherings there were five core strands of conversation.
1. Impact feeds sustainability
CIR’s goal is to produce stories that protect the most vulnerable individuals and communities and hold powerful institutions accountable. Our funders invest in investigative reporting as a powerful and necessary force in our democracy, while at the same time expecting us to demonstrate measurable success in achieving our mission. Thus, impact measurement and analysis is increasingly essential to organizational sustainability.
“When I ran a newspaper, profit was driven by ad revenue,” says CIR Executive Director Robert J. Rosenthal. “For CIR, and other journalism organizations, impact helps forge the revenue that sustains us.” Impact translates into philanthropic revenue, membership, partnerships and an understanding by the public that investigative journalism is relevant to them and can touch their lives.
During CIR’s Dissection in Washington, we learned from NPR that it defines impact as behavior or attitudinal change, the process by which NPR “creates a more informed public – one challenged and invigorated by a deeper understanding and appreciation of events, ideas and cultures.” When NPR successfully achieves this type of impact, its listeners will likely feel an emotionally significant bond with stations and become members, share material online or talk about programming with others who will then be exposed to NPR and/or local stations. Thus, in the case of NPR, impact (attitudinal change) is linked to sustainability.
The specific process linking impact to sustainability will be different for each organization. There was a consensus among Dissection participants that by dissecting the process of anticipating and tracking impact, we can discover new ways to translate the good we do into outcomes that contribute to our sustainability.
2. Shared language
During all of CIR’s Dissections, words like taxonomy and typology came up time and again in discussions. Wonky words aside, everyone recognized the need for a shared language around impact that is both expansive and specific.
Some universally agreed-upon elements of the emerging framework were
- Impact is change.
- Impact is an ongoing, iterative process (not a phase in evaluation).
- There are many types of impact, from attitudinal change to law change.
- Impact of an organization flows directly from its mission.
- Organizational culture must value capturing and understanding impact.
During the New York meeting, a working group formed to further the conversation around this issue. If you work in a media organization, you can help with this project by filling out this survey.
3. Big data versus thick data
We have more data than we need. Participants at CIR’s Dissections recounted organizational histories in which attempts to capture all the data associated with Nielsen ratings, online comments, likes and shares, page-related analytics and other sources were so overwhelming and ultimately so useless that they quit data gathering altogether.
Rather than thinking solely about “big data,” we need to think about useful data. What can we glean from the data streams to which we have access? Which data are indicators of what phenomena? How can we get not only big, but qualitative, “thick” data?
4. We need humans
In order to parse the relevant data from the “noise,” impact analysis requires humans with the appropriate skill set to carry out this type of research. Analysts might need to be able to do textual, network and/or statistical analysis in order to make sense of not only the plethora of data available, but to identify other data sources not traditionally included in impact assessment. For example, if a media organization is interested in effecting attitudinal change, impact analysis might include surveys of listeners or textual analysis of online comments.
5. Community
“Keep me in the loop” has been the most common parting at all four CIR Dissection: Impact events. The value of face-to-face time to share experiences and strategies and to make the connections needed in order to form a true community of practice is immeasurable.
CIR is committed to continued research about the process of media impact, as well as to a strong community of practice. We welcome partners interested in working to further the field of impact measurement and analysis. If you’d like to join a Google Group for sharing resources, please contact me! Even better, if you’re interested in creating a media impact analyst position within your organization, CIR can share its experience with the process.
Republish our articles for free, online or in print, under a Creative Commons license.
