Measuring What Matters
Another end-of-the-year donation rush is behind us, so the onus to choose wisely may be less pressing now. But if you’re a donor who wants to get the most bang for your charitable buck, you’ll be getting another tool to do so. Charity Navigator, which aims to help donors decide which nonprofits to support, announced in September that it would change its rating system in 2016, adding a third dimension to its existing financial and governance benchmarks. The new system will include an evaluation of charities’ effectiveness, including how transparently results are reported.
Charity Navigator describes the purpose of its revised ratings as twofold: First, to achieve its mission to be a guide for intelligent giving, and second, to drive more money to high-performing organizations and to encourage other nonprofits to boost their results.
Charity Navigator rates nonprofits with a star system; its evaluative processes and results are published on an extensive website. Charities that don’t meet its minimum standards can be put on a “watch list” that suggests donors do extra homework before making new contributions. Charity Navigators’ four-star rating—its highest—is a sought-after label; just 59 Minnesota nonprofits were on that list when this column was written.
Charity Navigator’s announcement is another step along the path toward “results-oriented philanthropy,” a movement among donors to identify whether programs created to solve problems are succeeding in their usually noble purpose. Nonprofits are increasingly required to provide evidence that their work makes a difference, even as measurement systems and evaluation processes are in beginning stages across much of the sector. Meanwhile, donor groups like Venture Philanthropy Partners and Grantmakers for Effective Organizations are sharing information about the best ways to help build nonprofits’ organizational capacity, including how best to measure, observe, learn—and change when needed.
It seems likely that more funders will move more decisively toward a results orientation in the next decade. The reason is that carefully evaluating financial management and governance practices isn’t enough to determine a charity’s worth to a community. Nonprofits are proliferating (the Urban Institute has shown that the number of U.S. nonprofits grew by 25 percent from 2001 to 2011), and growing (aggregate income across the nonprofit sector increased 41 percent during the same period).
Too many nonprofits have counted the number of people served as the main measure of success. But donors want more. Overwhelmed and even fatigued by a growing pool of seemingly comparable applicants, they want to know whether nonprofits are achieving meaningful results, to what extent, and at what cost. Effectiveness becomes a key differentiator.
New York’s Robin Hood Foundation provides another example. With the motto “We’re introducing R&D to TLC,” the foundation identifies “the most effective nonprofits lifting and keeping people out of poverty in New York City,” and then raises money to support them. How? Program staff have developed a methodology to estimate benefit-cost ratios, assigning a dollar figure to the amount of philanthropic good per dollar of grant cost. These ratios are used to compare the impact of one grant against another, no matter what the purpose. Results of grant-funded activity are published on the Robin Hood website. For example, 92 percent of people who enter Robin Hood-funded housing programs do not return to shelters, and Robin Hood-funded programs increased the chances of passing the GED by 75 percent. The foundation raises money from individuals, corporations and others; in 2012, the total was nearly $250 million. Some nonprofit leaders chafe at the growing emphasis on ROI. Many nonprofits do not exist to solve “problems” per se, so measuring impact is complex. An art museum, science center, botanical garden or symphony orchestra does not fit neatly into a problem/solution framework. The same holds for nonprofit media like public broadcasting, literary presses or documentary film projects. Media Impact Funders, the donor organization for grantmakers in journalism and media, recently launched a “knowledge network” that includes research, analysis and articles of interest that discuss assessing the impact of media on society. Arts organizations, which for decades relied on economic impact data, are now exploring alternative impact evaluation based on intrinsic impact and how it can be measured.
St. Paul-based Wilder Research is among our state’s most prominent evaluation experts: Its 100-member research staff are an important resource for Minnesota’s nonprofit sector, working on some 150 to 200 evaluation projects at any given time. Director Paul Mattessich says that many nonprofits—both grantmakers and nonprofits—think of evaluating effectiveness “as a sort of ballgame, where you see the score and you’ve won or lost.” Instead, he says, we should be thinking of the nonprofit sector as a learning community, where we measure, learn and “move numbers progressively.” Mattessich points to medicine and business, as two sectors where the goal is incremental improvement, not “game, set, match.”
In a recent interview in Stanford Social Innovation Review, Mario Marino, who had a successful business career before co-founding Venture Philanthropy Partners, took care to include the funders themselves when it comes to strengthening nonprofit outcomes. “Nonprofit leaders know whether they sit across the table from a punitive funder or one who is truly invested in their success,” Marino said. “Funders need to focus on helping leaders build high-performing organizations . . . in an adaptive way, rather than with a linear and narrow mindset.”
Let’s hope Minnesota’s donor community and recipient nonprofits can sit around a table and talk candidly about what they each need to do to strengthen their important and mutual roles.
Sarah Lutman is a St. Paul-based independent consultant and writer for clients in the cultural, media, and philanthropic sectors.