The Struggle with 'Off the Shelf' Measurement Tools

Historically the purpose of investing has been to generate a financial return. But today, 72% of the U.S. population is interested in investing in environmental, social and governance funds. You may be an asset manager who has clients asking about the environmental and social impact of their portfolios. You may be a member of the C-Suite who has stakeholders asking questions about the environmental and social impact of the firm. You may work for a nonprofit with donors asking for reports on the environmental and social impact of their donation. In each case, there is an expectation from shareholders, clients, funders and our inner selves that investment reporting include an appraisal of environmental and social impact. As a result, investment management firms, businesses, and nonprofits have been seeking ways of assessing this. One way of doing this is with ‘off-the-shelf’ measurement tools. This article will define these tools, and discuss some of the struggles associated with their use. This subject is one of several considered in SVT Group’s Pulse of Impact Management report (2017).

‘Off-the-shelf’ impact measurement tools are a set of pre-made assessment approaches that, ideally, are ready to serve the needs of all clients without much customization. Two such tools that serve as the basis for this analysis at the time of the Pulse of Impact Management’s publication (2017) are the Impact Reporting and Investment Standards (IRIS), launched in 2009, and the Global Impact Investment Rating System (GIIRS), launched in 2009. IRIS was developed by the Global Impact Investing Network (GIIN), and GIIRS was developed by a nonprofit organization called B Lab, two separate entities that were in close communication especially early in their histories. Both tools are intended to evaluate the environmental and social quality of impact investments. The tools’ mutual goal is to provide a common way to assess impact and thereby increase the impact transparency and comparability of impact investments. The issue is that off-the-shelf impact measurement tools with limited customizability struggle to meet practitioners’ need for relevance and added value.

As part of the data collection process for the Pulse of Impact Management report, interviews were conducted with 17 impact investment firms generally known for taking impact measurement and management seriously, to uncover their experience with both off-the-shelf impact measurement tools and other tools. A pattern the Pulse study surfaced was a general distrust and unwillingness to use these off-the-shelf tools. According to the Pulse report, the common explanation for this pattern was that: “IRIS and GIIRS were not able to fully address the broad needs of the impact investor in part because they did not capture many outcomes generated by companies or portfolios of companies, or sufficiently clarify the relationship between outcomes and financial performance for particular companies or portfolios. In addition, social performance was not well-assessed and the tools were not user-friendly.”

In response to this kind of market feedback, these two tools took quite different directions. 

In 2019 IRIS+ was overhauled so that its new incarnation walks users through a guided process of thinking through what impact indicators and metrics might be relevant for their contexts, while still recommending metrics that have been found to be useful by the wider practitioner community within a specific industry or mission area. Although IRIS+ results in a recommended set of pre-determined metrics, it also allows users to create their own metrics. The IRIS+ ‘wizard’ also largely follows the “5 dimensions of impact” that reflect the shared consensus of thousands of practitioners, as defined through the Impact Management Project’s consensus-building process, and so helps users to consider the facets of outcomes and impact deemed crucial by the practitioner community; something that had been missing from the prior version of IRIS. These modifications address market feedback about user friendliness and the quality of outcomes assessment. However, IRIS+ still leaves it up to the user to collect, manage and analyze data collected via the suggested metrics, including understanding how financial performance relates to social/environmental outcomes.

In response to the market’s lukewarm uptake, GIIRS was augmented by something called B Analytics, launched in 2013, which enabled users to drill into the specific components of the score that made up the GIIRS summary score and rating. B Analytics also enabled investors deeper visibility into their holdings, so for example an investor in a fund could check whether the portfolio companies within that fund were consistent with their social and environmental intentions or not (e.g. whether a particular portfolio company within a given fund the investor had invested in had exposure to conflict minerals). While it plans to continue to support B Analytics, ultimately in 2019 B Lab announced that it would no longer maintain GIIRS after the end of the year.

Sara Olsen of SVT Group and Kate Ruff, then a PhD Student at Schulich School of Business, acknowledged the tension between comparability of impact measures and relevance, as quoted in the Pulse report’s conclusions: “the more we rely on common measures to solve the comparison problem, the more we end up compromising the meaningfulness of social impact measures themselves.” This is not to conclude that off-the-shelf impact management tools should be completely ruled out, but rather to offer a perspective on the necessity of enabling users to drill into those aspects of their impact performance that they consider most important, and to combine particular indicators with additional customized information, such as context variables, that enable a given indicator to become more relevant and meaningful to the user. 

Tracking meaningful and contextualized indicators facilitates positive impact while informing managers, investors and others about risks and what drives business value. Experienced impact advisors such as SVT can help companies to determine what information will be valuable for them.