Data Gone Wild

July 24, 2014

Leap of Reason Update: July 2014

As regular readers of this newsletter know, we love great reporting about organizational cultures that embrace data as a way of ensuring material, measurable, and sustainable good for the people or causes they serve. In the past few editions, for example, we have pointed to the data-driven 10,000 Homes Campaign, the University of Texas at Austin’s efforts to increase graduation rates among disadvantaged students, and Roca Inc.’s approach to transforming young men at the “deep end” of the criminal justice system.

Now we’re going to do the inverse: highlight an account of an organizational culture that grossly misused data in ways that undermined the people it purported to serve.

In this week’s New Yorker, staff writer Rachel Aviv does a brilliant job of digging down to the root causes of the Atlanta Public Schools’ widely reported 2009 cheating scandal. Her article “Wrong Answer” is a powerful cautionary tale about how easy it is for human beings to pervert a drive for measurement into a dark, punitive exercise—one that’s far worse than no measurement at all.

Two elements of the story Aviv uncovered were particularly sobering. The first was the way the culture of cheating engulfed truly well-intentioned teachers. Aviv provides an empathetic portrait of one teacher in particular, Damany Lewis, who, before being fired for cheating, was a star teacher who would walk through walls for his students. Lewis’s principal, a Methodist pastor who preached that all “all decisions have to be made by data—you have to be baptized in it,” persuaded him to join a “team” of trusted teachers who were obtaining tests in advance and changing students’ answers. Lewis rationalized his cheating as helping keep the school, a safe haven in a tough neighborhood, from being shut down.

Another difficult element: the way foundations unintentionally provided lubricant to the well-oiled cheating machine. Aviv holds up reports like Annie E. Casey’s Beating the Odds at Atlanta’s Parks Middle School (no longer available on the foundation’s website), which celebrated the school’s “relentless focus on data.” Ouch. Let’s all be careful to avoid overhyping and inadvertently shining a spotlight on successes that aren’t.

And now we turn to updates from around the Leap of Reason community:

  • We recently learned that dynamic F.B. Heron Foundation CEO Clara Miller made the gutsy decision to stop asking grantees to submit grant reports. Instead of demanding traditional reports that do little more than gather dust, Clara is asking grantees to submit information to a public data warehouse—potentially saving grantees up to 50 percent of their reporting costs and doing far more to contribute to the fields in which Heron operates. If that’s not enough, Clara also spearheaded the foundation’s decision to dedicate 100 percent of its assets to its mission, not just the typical five percent. To learn more, check out Clara’s most recent annual letter.
  • The Coalition for Evidence-Based Policy just announced the winners of its low-cost randomized controlled trial (RCT) contest. Thanks to the contest and the funding that comes with it, skilled researchers will soon begin evaluating the impact of Bottom Line, Durham Connects, and workplace inspections by the Occupational Safety and Health Administration. By making use of readily available (generally public) datasets, low-cost RCTs can track randomly assigned members of test and control groups without the usual measurement expenses. They hold the promise of rapidly building the number of social programs that have been proven to work (or, equally important, not to work).
  • New York Times columnist David Leonhardt‘s new “analytical journalism” venture “The Upshot” just gave significant play to the Coalition for Evidence-Based Policy. In “The Quiet Movement to Make Government Fail Less Often,” Leonhardt reported that “the explosion of available data has made evaluating success … easier and less expensive than it used to be. At the same time, a generation of data-savvy policy makers and researchers has entered government and begun pushing it to do better.” We were pleased to see that the “moneyball for government” concept former White House officials Peter Orszag and John Bridgeland floated last year in The Atlantic Monthly appears to be turning into a meme.
  • The Urban Institute and UT Austin Professor Francie Ostrower recently published “Boards as an Accountability Mechanism,” a paper that contains the results of the first national representative survey of nonprofit governance in the United States. The survey, to which more than 5,100 nonprofits responded, reveals a finding that should be shocking but isn’t: Fewer than 20 percent of respondents said their board “very actively monitors its own performance,” and fully 45 percent said that their board “is not even somewhat active” in monitoring performance. Perhaps the title of the report should have been “Boards AWOL as Accountability Mechanism.”
  • On a parochial note, we were tickled to stumble upon the outline for University of Calgary Professor Robbie Babins-Wagner‘s graduate-level class “Becoming an Evidence-Based Leader,” which started last week. There are nearly 60 different books and articles on the reading list, but just two are listed as required texts: Leap of Reason and Working Hard—and Working Well. Given the sticker price ($0), at least they won’t put a dent in students’ budgets.

Events/Webinars for Raising Performance:

Mario and Lowell

Mario Morino is Chairman of the Morino Institute, Co-Founder and Founding Chair of Venture Philanthropy Partners, and author of the lead essay in Leap of Reason. Lowell Weiss is president of Cascade Philanthropy Advisors, co-editor of Leap of Reason, and advisor to the Leap of Reason initiative.

Download complimentary copies of Leap of Reason and Working Hard—and Working Well. Check out our suite of materials and video gallery for strategic planning sessions, performance-management projects, professional development, board meetings, or graduate/undergrad classes. And encourage colleagues and stakeholders to sign up for monthly updates to help power their leap.

Quotes From Leaders Like You

“You’re taking someone else’s money to get into somebody else’s life to try to make a difference. You better be showing you can make a difference!”
Molly BaldwinExecutive DirectorRoca, Inc.
See More