Thursday, July 21, 2011

How do you know a program is working

The Byerly Foundation is in the community building business and we do this by providing grants for programs that the Byerly Board hopes will make a significant difference in helping Hartsville become an even better place to live and one of the best places to live -- anywhere. That is a stretch goal, by almost any defihttp://www.blogger.com/img/blank.gifnition.

One of the major problems the Foundation has and nearly all Foundations and community development types of programs is how to know if programs work. Normally the groups implementing the programs will set their goals and list the measures they want to use to measure effectiveness. Then you assess the program based on the approved metrics. Sometimes this gets you the information you want and sometimes it doesn't.
More-Than-Good-Intentions
One thing that those working with Foundations and other funders do is continue to explore ways to measure effectiveness. This week the Stanford Innovation Review has a review of a book that discusses how programs might be measured. Just a few minutes ago I ordered this book from Burry Bookstore -- More Than Good Intentions: How a New Economics Is Helping to Solve Global Poverty by Dean Karlan & Jacob Appel

According to the reviewer, Kevin Starr, "Karlan and Appel believe that understanding what works for poverty alleviation programs boils down to one deceptively simple question: “How did people’s lives change with the program, compared with how they would have changed without it?” The primary—but not only—tool that Karlan et al. use to answer that question is the randomized controlled trial (RCT). In an RCT, a pool of subjects is randomly divided into intervention and control groups; the former gets the interventions and the latter does not. The two groups are fundamentally alike—both are measured before and after, and the impact is the difference between what happened to the intervention group and to the control group. RCTs are not new. The novel element here is the systematic and creative application of RCTs to test poverty solutions in the real world."

This is an interesting concept because when you are working on social interventions, it is difficult to keep something that is working away from control groups and often times you have try designing your research project to somehow give those randomized out a chance for being in. While I have not discussed this fully with Dr. Eve Puffer I believe this is a tactic she has used in a major innovation project fighting the spread of HIV/AIDS in Muhuru Bay, Kenya. The Byerly Foundation is currently funding a major project with the Hartsville Middle School called Foxy Leaders that aims to take students at real risk of not finishing school and giving them an even stronger foundation than they might get in regular classes. I am not sure how this could be an RCT. That is why it is going to be interesting to get that book and discover what they authors have found in their research.

Monday, July 11, 2011

Driving Social Change

Paul Light, a professor from New York University, has a recent book called DRIVING SOCIAL CHANGE, How to Solve the World's Toughest Problems.http://www.blogger.com/img/blank.gif It is an interesting book but more for the academic side of exploring social change than the implementing side. That said, he has some great discussion points.

In discussing the "breakthrough cycle" he mentions some tools developed by RAND that "participants in social breakthrough can use to increase their effectiveness and targeting..." page 142 This is the idea that I wanted to highlight: ..."the key to alertness starts with a landscape of possible futures against which to plan, and continues with an honest assessment of what has to go right for nothing to go wrong and what cannot go wrong for a plan to go right."

Not totally sure why I am taken with this phrase but I think it points to a way of thinking about solutions that may often be ignored in the planning process.
http://www.blogger.com/img/blank.gif
The truth in today's world is that many of the smartest of social innovators are getting crushed by limiting visions of possible futures that are only based on their assumptions and views of the world. For example, when it comes to our present, who among us would have believed that two years after the recession ended we would still be experiencing nearly unparalleled unemployment? RAND does have something to say about this type of situation. When participants in a breakthrough cycle notice assumptions breaking down they need to be agile, adaptable and aligned. (page 142/143)

As I reread this post I wonder if there is enough context for understanding but instead of letting that be a worry I think will publish to see this generates any thoughts or aligns with some other reading and researching that is being accomplished in current innovation programs.

We are in the midst of driving some of this social change at Coker College and as I think of the things being planned, I wonder "what cannot go wrong for the plan to go right?"