AddThis.com robot tech.support@clearspring.com - Array ( )
Login

Up for Debate: How Do We Create a Culture of Measurement?

In an effort to capture the latest thinking and doing in metrics and evaluation, the Skoll World Forum partnered with the Aspen Network of Development Entrepreneurs Metrics Conference held just a few weeks ago, and asked some of their participants to reflect on the current metrics landscape, the challenges and opportunities facing different sectors, and how best to move the dialogue forward. Villgro Innovations Foundation, Root Capital, Grameen Foundation and ANDE all participated in the discussion, which is published in partnership with Forbes.

 
 

Can Financial Markets Be Good?

Steve Wright

Vice President, Poverty Tools, Grameen Foundation

Speaking of Social Impact, Words Matter

Jessica Seddon

Head of Knowledge Management, Villgro Innovations Foundation

 

Will Impact Investing Take Off?

Randall Kempner

Executive Director, Aspen Institute

When Measuring Social Impact, We Need to Move Beyond Counting

When Measuring Social Impact, We Need to Move Beyond Counting

Mike McCreless

Director of Strategy and Impact, Root Capital

July 15, 2013

 

Article Highlights:

  • We shouldn’t get stuck at output metrics.
  • Push forward to outcomes and impacts if you can, but a fuller understanding of contexts and practices can also serve as a useful guide to day-to-day decision-making.
  • Impact business model analysis can provide an integrated way to think about which combinations of contexts, inputs, and practices will support which financial and social objectives of an organization.

“Count something and make it count” is a catchphrase in the impact investing and social enterprise community. Just as commercial firms measure their success by profits, mission-driven organizations measure success by their ability to address social and environmental challenges, and they need yardsticks to gauge their progress.

For metrics gluttons (such as myself), the Annual Metrics Conference of the Aspen Network of Development Entrepreneurs (ANDE), held in Washington DC on June 12-13, offered up a smorgasbord. Organizations of all stripes were there, seeking ways to use social and environmental performance data to further their missions. And not only social enterprises and impact investors, but also multilateral agencies like the Inter-American Development Bank, government agencies like the U.S. Overseas Private Investment Corporation, and corporations on the leading edge of corporate social responsibility.

It was heartening to see so many organizations heeding the call to ‘count something.’ Yet a common theme of discussion was the need to do more to ‘make it count.’ Simply put, ‘counting something’ is necessary but not sufficient to measure our impact accurately and manage towards that impact in practical terms.

To give an example: nearly all organizations track a metric such as “number of people reached” or “number of lives touched.” For organizations seeking to reduce poverty, this is such a foundational metric that it would be unthinkable not to count it.

Yet the ‘number of people reached’ metric is ultimately unsatisfying. What was the impact on these people? For instance, Root Capital has reached over 600,000 small-scale farmers and over 3 million individual household members since our founding in 1999, but what were the types and depths of impacts on each family? Through our Impact Roadmap we are seeking to answer questions such as these, but we have much more to do.

To address these questions systematically, practitioners often think of impact as a chain:

chart2

Collectively, we as a sector aspire both to create and to measure impact, but in practice, most metrics collected by most organizations are at the level of outputs – how many dollars we lent, how many people we reached, etc. These are easy to count and typically measure the scale of our impact. Moreover, public standards such as the Impact Reporting and Investment Standards (IRIS) now provide clear and consistent definitions of commonly reported output metrics.

Likewise, the discussions at the ANDE conference focused on output metrics. Outputs are the point of entry for organizations adopting social and environmental metrics. Rightly or wrongly, many perceive that outcomes and impacts are simply too costly and time-consuming to measure systematically, because they require extensive data collection. That said organizations such as IDInsight are working to create more practical and affordable ways to apply rigorous evaluation methods to reach beyond output metrics and measure outcomes and impact.

A more recent trend is that, in parallel, several organizations are working not only forwards but also backwards in the chain, to the level of practices – the things we do that will create our desired impact. The rationale is, if we can’t cost-effectively measure all of the outcomes and impacts that we’d like, can we think more rigorously about what practices we can reasonably expect to drive the desired outcomes?

For example, this is particularly relevant in the environmental domain. Environmental outcomes – for instance, improvements in soil quality or biodiversity – can be more difficult, and costly, to assess than social ones because they require specialized expertise and equipment. The need to identify observable practices that can serve as reliable proxies for outcomes and ultimately impact is acute. EcoAgriculture Partners has helped Root Capital and other ANDE members to think about these issues.

Finally, an increasing number of organizations are looking even further back in the chain to the level of the context in which our clients operate, in order to channel scarce resources to where they are most needed. For instance, the Progress Out of Poverty Index (PPI) of the Grameen Foundation provides a relatively low-cost and efficient, albeit broad-brush, way to evaluate the poverty level of a given community.

To summarize so far: Count something, absolutely, but to make it count, we shouldn’t get stuck at outputs. Push forward to outcomes and impacts if you can, but a fuller understanding of contexts and practices can also serve as a useful guide to day-to-day decision-making.

A Necessary Tool: Impact Business Model Alignment Analysis

So far so good – if you happen to be the metrics manager of a social enterprise or impact investment fund. But isn’t this all a little arcane for managers seeking to build businesses that address social and environmental problems?

Not if you heard the discussion about “What is Impact” between Tom Adams of Acumen Fund, Allison Basile of Grassroots Business Fund, and Rhett Morris of Endeavor.

The debate started months earlier, when Endeavor posted on the Harvard Business Review blog its finding that Endeavor entrepreneurs “who prioritized financial goals over social goals were much more likely to experience high rates of growth and have greater social impact.” Tom Adams replied, calling this “an illusionist’s trick of sorts: finance equals scale, scale equals impact, and hey, presto, finance equals scaled impact.”

Underlying this debate is a critically under-appreciated point, which is that some business models are more aligned with some types of impact than other business models. Endeavor quotes Cathy Clark, of the Center for the Advancement of Social Entrepreneurship at Duke University: “Social entrepreneurs need to understand if their social mission is complementary to their financial goals or if there are going to be lots of friction points due to conflict between them.” Business models with fewer frictions between social and financial goals will face a smoother path to scale in both dimensions.

I tend to be congenitally skeptical of sweeping generalizations, particularly those regarding the tradeoff between profit and impact – or absence thereof. The more we can ground our discussions in the specifics of which aspects of which business models are aligned with which social and / or financial objectives, and why, the sooner we can get on with achieving both. For lack of a better term, I will call this type of analysis ‘impact business model alignment analysis.’

In the absence of a willing test patient, I will try out this experimental analysis on my own organization, Root Capital. By way of introduction, we are a social investment fund that that grows rural prosperity in poor, environmentally vulnerable places in Africa and Latin America by lending capital and delivering financial training to agricultural small and growing businesses (SGBs).

The Chain of Impact described above provides a useful way to organize this analysis. Through a recent strategic planning process, we found that most of the practices we can implement to become a better non-profit agricultural lender will improve both our bottom line and our impact. By becoming more efficient, we stretch every dollar lent further.

There is just one area in which there is a friction or tradeoff between our bottom line and our impact, and that is in choice of target population – that is, the context in which we choose to operate.

Specifically, Root Capital lends to agricultural businesses in Africa and Latin America. Our clients are farmer associations and private businesses that help build sustainable livelihoods by aggregating hundreds, or even thousands of small-scale producers and connecting them to markets.

These businesses range in revenue from $200,000 or less to $5 million or more. There is little variation in the cost to Root Capital of originating and monitoring these loans, but the revenue of these loans varies dramatically with loan size (along with interest rate, term outstanding, and other factors).

Loan size, in turn, correlates with client revenue. Larger agricultural businesses request larger loans. Of course, these businesses are large only within the spectrum of small and growing businesses that Root Capital serves, and the end-beneficiaries – small-scale farmers – are the same for large and small enterprises.

The rub is that larger agricultural businesses are likelier to be served by other non-profit or even commercial lenders. They still have unmet financial needs that require our support. For instance, a commercial bank might be willing to offer loans secured by fixed assets or personal guarantees, but not the type of trade financing that Root Capital offers. In addition, Root Capital seeks to innovate to serve additional unmet needs of our growing clients, for instance by financing renovation of small-scale producers’ farms.

Nevertheless, because they have other options, larger businesses arguably need our lending less on average than the small, early-stage businesses that receive the first loan of their existence from Root Capital every year – typically loans of $100,000 or less that are high-impact but not profitable to Root Capital.

Root Capital’s ‘impact business model’ hinges on the fact that these small businesses tend to grow and take out successively larger loans. Their growth not only increases impact but also underpins our business model as the revenue from growing clients cross-subsidizes loans to the next round of early-stage businesses.

At a portfolio level, we aspire to achieve operational self-sufficiency in our lending – that is to say, for the interest and fees from our loans to cover our lending expenses. We are more than three-quarters of the way there, but it is undoubtedly true that we could quickly enhance our economics by seeking out a more profitable target population, albeit one with less unmet need, or by jettisoning the less profitable clients. Our decision not to do so defines the blend of financial return and social impact that we offer to our investors.

In short, the context in which we choose to operate – the extent to which we prioritize under-served populations – directly impacts our economics. This insight is not original or unique to Root. Microfinance practitioners have struggled with this for decades.

Moreover, the point is not limited to contexts. Impact business model alignment analysis has the potential to unify the different levels of the chain of impact. Other organizations might conduct their own impact business model analysis and uncover synergies or tradeoffs between financial and social objectives at the levels of practices or even inputs. In turn, this analysis can drive choices of output metrics and inform efforts to measure outcomes and impacts.

More broadly, impact business model analysis can provide an integrated way to think about which combinations of contexts, inputs, and practices will support which financial and social objectives of an organization. If we could only crack the nut on measuring impact cost-effectively, we would have a grand unified theory of financial returns and impact. But until then, I would argue for a little bit more counting of output metrics and a lot more impact business model alignment analysis.

 
  • JeffMowatt

    Some years ago, we had a Skoll Social Edge conversation on Profit for a Purpose

    http://skollworldforum.org/2006/09/25/profit-for-a-purpose/

    We have argued:

    “Economics, and indeed human civilization, can only be
    measured and calibrated in terms of human beings. Everything in
    economics has to be adjusted for people, first, and abandoning the
    illusory numerical analyses that inevitably put numbers ahead of people, capitalism ahead of democracy, and degradation ahead of compassion.”

  • ralph taylor

    Mike,

    Excellent blog. Your skepticism of the discussion about trade-offs between profit and impact is obviously well-grounded. I also appreciate your weave of the discussions between Acumen, Endeavor, and the GBF as prologue to your description of impact business model alignment analysis. Root Capital is lucky to have you, and I look forward to finding out what EcoAgricultural Partners is doing to help you and others in ANDE use practices as proxies.

    I did not realize that “nearly all organizations” measure something like “number of lives touched.” This seems like such an obviously disingenuous output. Does Acumen still do that on its web site?

 
 

Let’s keep this going …

Why not join our growing community of social innovators? You’ll get exclusive content and opportunities delivered straight to your inbox and all the latest details from the Skoll World Forum.

We value your privacy and you can unsubscribe anytime.