Note: I had to restore my site from an old backup to solve a link injection problem. I had to republish this post, and lost all the great comments. I’m sorry!
A subscriber to the International Development Careers List asked me a question that really wasn’t about jobs the other day. I figured I’d answer it here, on the blog, instead of on the list. He asked me
“Reading on to all the writing around how we “don’t know” how to solve / what works for global poverty/issues and that to some degree a lot of the agencies are just trying out methods?”
If I parse the question correctly, he wants to know – are development organizations just making all this up? If we don’t know what works, then why are we doing it?
This is where I stop to point out that I am not a development economist. I am not an economist at all. I took four college-level classes in econ as an undergrad, and I’ve spent the last decade and a bit working for development projects. So all I have is a gut feeling and a resume. That being said, I do think about this stuff. As does almost everyone I know who works in this field.
I don’t think that anyone is making their programs up. I think that sometimes we delude ourselves about the quality of our evidence. We are so sure we have the right approach that we start mistaking all our intermediate results for actual impact. So a lot of programs end up based on doubtful evidence. Especially big, broad-based programs intended to reduce poverty or achieve some other massive societal goal.
Even if you’re committed to making all decisions based on evidence, it’s hard to measure if that kind of program works. We can end up using proxy measurements that may or may not be accurate.
Next, I think we have evidence for a lot of smaller, targeted programs. We know how to improve child survival. We know how to improve school attendance. We know how to improve the agricultural productivity of small farms. We know quite clearly what it takes to do specific things that we hope will then reduce poverty. We just don’t know that these kinds of specific things actually do reduce poverty.
In the end, I am not sure it matters. While we shouldn’t fund poverty reduction programs if they don’t actually reduce poverty, that argument doesn’t hold true for, say, bringing down the maternal mortality rate. Fewer dead mothers is an inherent good. I don’t really care if it also helps with poverty reduction.
_______
Very well said. One could add that the business of “trying out methods” which concerns the subscriber to the IDC List belongs to the scientific method as one of its essential components. But development is not merely a social rather than a “hard” science—it’s a relatively new social science whose youth compounds the suspicion generated by any social science that it is in fact, despite protests to the contrary, an art. The exchange of highly public and sometimes ad hominem attacks among development stars and superstars only reinforces the impression that our fearless leaders know how to snatch and protect turf better than they know how to improve the lives of poor people.
Yet, as you correctly point out, a debate will typically involve how to reach some “massive societal goal” rather than, say, “how to improve child survival”—the sort of small thing that we know how to do and whose unchallengeable goodness requires no millenarian defense.
An astute reader just emailed me the comments I lost on this entry:
# Tom Says:
January 20th, 2011 at 11:19 pm
I’d just add we in the NGO world are increasingly investing in randomised impact evaluations (from the realms of political science) in order to demonstrate impact with a high level of scientific robustness. It’s expensive, time consuming and causes all kinds of implentation complications, but really is second to none in terms of isolating and measuring programme effects – even for broad based programmes. Innovations in programming can be tested very effectively this way by comparing a sample of communities where one approach was tried to others where a slightly different one was adopted.
# Ian Says:
January 21st, 2011 at 12:00 am
We are not making it all up, but we don’t have all the answers either.
The world is a messy place, and its not possible to have all the answers about what will work in advance, or even always to know when we have done something whether it has worked (I blogged on this some time agohttp://kmonadollaraday.wordpress.com/2010/11/22/the-truth-is-out-there-or-maybe-not/).
This means we are forced to start from what we know from evidence and experience and then plan from there, ideally monitoring closely what happens as we go and making corrections over time as we learn more about what works. Even when something has worked before, or usually works it’s not a sure thing it will work again so it’s always important to monitor and adjust.
We know more about some areas of development work than others, and some (perhaps many) development problems are not susceptible to being broken down into simple steps or recipes due to their complexity, so you need to take other approaches to dealing with them – in fact you need to experiment and iterate so in a sense you don’t know exactly what you are doing when you start but if you use a smart approach you can find out by doing. I also recently blogged on this (http://kmonadollaraday.wordpress.com/2011/01/19/whos-afraid-of-complexity-in-aid/)
So basically we only partly know what we are doing, but that’s OK it doesn’t mean we shouldn’t be doing it if we take a sensible approach to dealing with the uncertainties we face, and don’t pretend to know more than we do.
Sam Gardner Says:
January 21st, 2011 at 3:26 am
If people want to invest wisely, there are quite a few “guaranteed results” interventions, and actors specialised in them. The sites like good intentions are not enough can be used to identify them. Try e.g. de-worming, school-feeding, immigration or vaccination. The Copenhagen consensus, with Lomborg, follows the same track.
However you could also go for high risk, potentially high impact investments, like investments in governance, strengthening civil society, etc. In the high risk area, there is a lot of research to show what works in what context and what not.
The learning that is going on in the development world is ashtonishing, and knowledge is fast stale.
On the other hand, the easy way out is going for conventional wisdom, and just invest in what is fashionable or is good on photo.
It gets complicated when you look for funding. Visible interventions are normally not very good for development results. Institution building is boring and not visible and gets you no funding.
Sorry. Selection seems to prefer bad projects. It are not the beneficiaries but the donors that select.
[…] This post was mentioned on Twitter by George Darroch, Public Health Reader. Public Health Reader said: Blogs: Are we making it all up?: Note: I had to restore my site from an old backup to solve a link injection pr… http://bit.ly/hqDraf […]
I admit it seemed a bit odd at first when the author stated they didn’t care about reducing poverty. Solving poverty is one of the main reasons as to why many of the development agencies and organizations are around today. However, I feel the argument laid out is a valid one. Sure, it sounds admirable to say “we can end world hunger”, but this won’t happen overnight, nor will it happen without the proper education of the people who need it best. Starting small with some of the most basic needs such as prenatal care and childhood education will evolve into better living conditions for that village/town and beyond. I agree it would be best to start small, instead of trying to throw money and goods into a country, without the knowledge and education needed to help sustain the standard of living.
I would have to say I agree with all these comments so far. Discovering the minuscule issues facing impoverished or undeveloped states/communities is not an easy task. And once the issue is identified, the solution is not exactly obvious. If one is to say “if you don’t know what works, why are you doing it?” my response would be – great things are not done overnight. Just as with any other problem, working from the ground up accomplishes the task. I will say, developmental solutions are hit or (usually) miss, but the worse crime is not attempting.
I want to comment on the following line in your post:
“We know how to improve child survival. We know how to improve school attendance. We know how to improve the agricultural productivity of small farms.”
I personally work EWB Canada, partnered with the Ministry of Food and Agriculture in northern Ghana. And while I can guess that the answer to improving the agricultural productivity of small farms includes things like improved seeds, appropriate fertilizer application, timely weeding, etc., I wouldn’t say we KNOW how to do this.
I work in a district-level government office. This office has a year-plan to drive their extension services to farmers, which is their primary role. On top of that, the national government has instituted about 4-5 programs which are meant to help farmers (fertilizer subsidies, commercial farming input loans, farmer registration, etc.) but which really just dilute the quality of work of most extension officers since they’re running from place to place without enough fuel. On top of that, there are always at least 10 NGO or donor projects implementing work through my office, adding to the burden on field staff. The result is that NONE of these projects is “working” for farmers. (Seriously, none of them.)
The devil is always in the details – the IMPLEMENTATION details. Just because we know that this is WHAT smallholder farmers should do, doesn’t mean we know HOW to convince them to do it.
Instead of spending time figuring out the most impactful development intervention, we should be worrying about our implementation of these so-called “known solutions” (even if we can’t link them directly to poverty reduction, which is a whole other problem).
If we look back at history, we can see the trial and error of many global projects. Not only is it difficult to satisfy all the needs of communities economically in one’s own country, you can imagine how difficult it is to solve massively complex issues worldwide with barriers such as language, culture, and religion at every border. Until you understand how a specific society lives, you won’t be able to help them succeed.
Also, I feel these issues are always looked at from an economic and political stance, but rarely do we see written social theories which could apply to reasons why some anti-poverty initiatives have yet to stick. Perhaps, if we delve into learning all aspects of social perspectives of the needs of those who are impoverished, we can learn more effective ways to help them and not just throw money at them.
Since the issue of poverty includes a multi-disciplinary focus, we really need to find out where the fit works the best. Is the solution in politics, economics, or what? Depending on which different missions are attempted and fail, we are still gaining evidence and learning why they failed and where things went wrong. The truth is, these trials and errors seem insignificant now, but for future citizens living in impoverished communities…the evidence from these errors will eventually give us insight to what hopefully will work.
I agree completely. Aid is not just different methods that people are trying out. Yes, some aid has not always worked to the best intentions it has, like the 1milliontshirts, however a lot more aid has benefited people and helped lots. Child health and maternal health has imporved in the past several years. Less infants and mothers are dying in child birth and they are getting the needed vaccines to prevent diseases such as maleria. So yes sometimes aid may not work to the best benefits but i feel as though the vast majority of it helping and benefiting people greatly.
Evidence of success with projects in specific or small scale cases, does not guarantee the same succes when translated to broader, less targeted goals. This idea is so important when developing aid programs that are used globally. As another commentor suggested, the entire spectrum of social perspective must be considered, not solely political and economic aspects. The conditional and case specicific cultural frameworks must determine the path to development. I think this is where development aid is heading. These small improvements which focus on alleviating immediate problems like farming, medical treatment and maternal care assistance provide the kind of direct relief necessary. Many projects go in without evidence to guarantee success but that not prohibit success. The testing of methods is the only way to discover which ones fit in individual cases and which ones should be abandones. Piecemeal improvement is much more realistic than immediate gratification.
I agree with a lot of what has been said already. Indeed small changes are what create the big changes in the end. I think too often people get caught up in trying to solve huge problems such as poverty that they tend to look past the real solutions. They focus on the end goal and look for ways to achieve the goal by just straight jumping there. More should be looked at between the starting point and the final point. I believe that the presentation of goals should also be adjusted. When we look at things such as the MDGs they present huge focuses such as eradicating poverty and hunger. This is obviously a goal that can not be achieved quickly. There kind of are stepping stones in the MDGs; however, not so much. So when the goal is looked at, did we eradicate poverty and hunger?, the likely response is no. If smaller goals were presented such as “reduce hunger by 10%” or anything of the like it is easier to give a response that it was accomplished. When people often view goals of development and say whether they want to help contribute or not, they want to know if the program will work. If we can say “yes, we achieved this goal. and are now looking at the next goal” than I think there would be better support and also more confidence to continue.
I just want to reiterate what Erin has said above, the difficulty often lies more in implementing the interventions we know work. And that’s because the interventions are often developed in a controlled setting that’s far removed from the messy realities of the world. So, saying “we know they work” is not entirely accurate — we know they work “in the lab.” After 6 years of work at Jhpiego, I only recently had this epiphany: http://linearityofexpectation.blogspot.com/2011/01/what-i-didnt-know-yesterday-28.html.
It’s not that anything is being made up, it’s that the number as well as intensity of problems around the world are so vast that it makes it incredibly difficult to”fix” these problems. To go along with what G. Gaston said,it most assuredly will take smaller steps, not countries or banks writing massive checks to or formulating huge intricate plans to stop poverty in a country or to solve the hunger problem in the world or in a Specific country. These type of plans waste more money and time than they actually do to help the situations. Unless the people writing the checks go to the specified country and really understand what is happening not much can ever change. To reduce poverty it will take a formula and combination of certain “ingredients” that we obviously have yet to discover, and if we have we have not discovered the correct way to use them.
Toomer – I think understanding poverty takes a lot more than just going there…
The active involvement with the people of the specific country is paramount. Involvement needs to be truly involved rather than just throwing money from across oceans at problems that don’t get fixed which has happened for so many years. No organization no matter how vast their riches can understand the problems a country faces better than the people who live through these hardships every day. There must be an effort to work with the people themselves to better solve problems and better allocate aid. I think that this necessity’s scarcity is one of the major culprits in the current ineffectiveness of aid.
This is a very interesting issue that I think plagues a lot of people in the development world. I am a firm believer of any degree of help is good no matter which way you look at it. I do, however, also agree that more specific missions such as vaccinating 1,000 children for malaria is much more beneficial and attainable than say trying to rid a country of poverty. A lot factors contribute to poverty and a lot of those factors are completely uncontrollable by outsiders. I believe there needs to be a shift in the development world from the inconquerable to the completely achievable.
[…] one of those moments when you see that Alanna was dead on when she wrote (several times, actually) about how projects work, while grand theories and ivory […]