Flashlights Not Hammers

As readers of this update know, we’re big believers in the power of data for learning and improvement. But we also recognize people can—and often do—use data in ways that create harm.

The latest example comes from this New York Times podcast, which aired on June 5. It’s the sobering story of how then-Baltimore Mayor Martin O’Malley‘s good intentions for data use led to severe unintended consequences, especially in poor black neighborhoods like the one where 25-year-old Freddie Gray died in police custody in 2015.

O’Malley, who ran for President in 2016 and appears to be considering a run again in 2020, won his mayoral race on a platform of improving safety in a city that had been hit hard by drug-related violence. Once elected, he implemented a version of New York City’s CompStat performance-management system, which had contributed to a major drop in crime in the Big Apple.

There was nothing inherently wrong with Baltimore’s system, which O’Malley branded “CitiStat.” O’Malley’s goals were admirable, and the technology was smart. But things got ugly when the city implemented the system in a way that used the numbers as a hammer rather than a flashlight.

The podcast included damning testimony from Tyrone Powers, a former FBI agent and Baltimore City Police Department advisor. Powers described a dysfunctional, numbers-obsessed police culture very similar to that depicted in the hit HBO series The Wire. Powers said that arrest statistics are “what got [officers] promoted, or it got them chastised if they were not making enough arrests…. Many of them were actually removed from their command positions right at [CitiStat] meetings…. No matter the consequences on police-community relations, they focused on the fact that this was the edict of the administration. And the administration said that if you want to keep your position, then you have to follow this. So that’s all they were focused on.”

To help us understand the full dimensions of this story, we reached out to four colleagues who know CitiStat and O’Malley well to get their take. From all four we heard the same message: While the administration’s narrow focus on arrest figures led to bad practices within the police department, in other parts of Baltimore government the use of CitiStat had positive effects on services, including in education.

We believe this complex, human story is a perfect illustration of a caution Mario expressed in Leap of Reason:

When public or private funders establish performance metrics and tie rewards or consequences to organizations’ capacity to meet them, organizations and people will migrate to the behaviors that will allow them to meet their defined targets. If the metrics are appropriate and closely tied to mission, the organization can benefit. But if the metrics are simplistic and unmoored from mission, organizations will go racing in the wrong direction.

The moral of the story is this: Data and performance-management systems are only as good as the people putting them to use. It’s not the data, but the values, experience, and discernment of those using the data that create the real value—or damage! Therefore, how you build your teams and your data culture really matters.

Keep the faith (and reason),

Mario and Lowell

---

 

Updates From Around the Leap Community

• Today, the Leap Ambassadors Community is releasing “Meaning, Purpose, and Joy: A Profile of Duncan Campbell,” the latest installment in a series highlighting funders helping their grantees pursue high performance. It’s the story of Campbell’s journey from severe neglect as a child to surprise wealth as an entrepreneur to life significance as a philanthropist. We hope you’ll download the profile and share it with funders in your networks.

• On June 6, the Leap Ambassadors Community teamed up with GuideStar to host the webinar “Small, But Mighty: Seven Ways Small Nonprofits Can Boost Their Performance.” If you missed it, you can now download the recording. Kudos to Leap Ambassador Debra Natenshon, as well as SHALVA’s Carol Ruderman and Sara Block, who did a great job illustrating the ways in which the core concepts from the Performance Imperative are relevant for community-based organizations with budgets between $100,000 and $3 million (almost 90 percent of the nonprofit sector). Kudos also to Leap Ambassador Adrian Bordone, the webinar’s host, who might want to consider becoming a broadcaster in his next career!

• The Laura and John Arnold Foundation’s latest report, “When Congressionally authorized federal programs are evaluated in randomized controlled trials, most fall short. Reform is needed,” is a must-read for anyone who cares about improving government-funded social services. The report makes a compelling case that “in a world where most attempts to make progress fail and a few succeed, spending as usual without a clear focus on evidence about what works is unlikely to solve the nation’s problems.” The report reinforces the key message of a 2013 Atlantic Monthly article by Republican John Bridgeland and Democrat Peter Orszag: Governments must learn how to play “Moneyball”—that is, use evidence and use it wisely—when they allocate taxpayer dollars for social programs.

• If you’re like us and live in a city in the midst of a homelessness crisis, please read David Bornstein‘s encouraging New York Times blog post “A Growing Drive to Get Homelessness to Zero.” Bornstein documented that humane and inspiring civic leaders armed with great data are reducing homelessness in 46 communities across the country. The 46 communities “are collecting and maintaining real-time data and lists of the names of people experiencing homelessness, and from those deepening their understanding of the dynamics of a complex and ever-changing problem,” Bornstein wrote. “And they are linking in a national network, capturing and sharing effective strategies, as they emerge, to improve their performance.”

• We’re excited to share a bundle of good news from the high-performance nonprofit Playworks, which has been working for 22 years to enrich kids’ recess experience with social and emotional learning (SEL). First, the RAND Corporation found that the Playworks Coach service meets the highest bar for evidence of impact under the federal Every Student Succeeds Act. Second, researchers at four universities have demonstrated the value of a new Playworks tool aimed at helping schools understand how they can improve the quality of the recess experience. The researchers will now work to distribute the tool, known as The Great Recess Framework, to as many schools as possible and then begin a study of how the quality of recess affects children’s performance in the classroom. Both successes were just highlighted in EdWeek, through an interview with Playworks Founder and CEO Jill Vialet and a post on SEL by Playworks President Elizabeth Cushing.

• We enjoyed reading “How Adaptive Strategy Is Adapting,” an SSIR post by Monitor Institute’s Dana O’Donovan, Gabriel Kasper, and Nicole Dubbs. The authors openly acknowledge how their thinking about effective strategy processes have evolved through their study of innovation, data science, and execution management. Their new insights mirror a book that has profoundly influenced Mario’s thinking and behavior for decades: The Renewal Factor (1987). The author, Bob Waterman, characterized the management process as similar to sailing. When captain and crew set sail, they 1) know their destination, 2) understand the capacity of their vessel, 3) know the skills of the crew, and 4) have an initial plan to guide their journey. But the minute that sailing vessel leaves the dock, the plan becomes less relevant, as the real challenge is reacting to the wind, current, and weather.

 

Events/Webinars for Raising Performance

June 26—San Francisco, CA
“Being the Change: Foundations Transforming for Greater Impact” panel discussion; Northern California Grantmakers

June 27—Online
“Getting Ready to Get Ready for Giving Season” webinar; GuideStar

July 12—London, UK
“Reviewing and Improving Your Charity’s Strategy” workshop; New Philanthropy Capital

September 12-14—Stanford, CA
Nonprofit Management Institute; SSIR

September 13-14—San Francisco, CA
“Imagining the Future of Family Philanthropy” symposium; National Center for Family Philanthropy

October 4-5—Washington, DC
Fourth Annual Feedback Summit; Feedback Labs