Why we need to measure inequality

How do we ensure that evaluations include gender and equity?

Let’s not forget the role that people play! It is people who use evaluation as a tool to illuminate and address inequities: guidelines, systems, and capacities are insufficient.

Let me give an example from my work.

We talk a lot about disaggregated data.  But just having that data does not mean it will be used to highlight inequities.  In Bihar, India, we are working with the government and other partners on an ambitious maternal and child health program.  The program captured baseline data in early 2012 from women who gave birth the previous year across the State. Over 13,000 women were surveyed. We had data on gender, birth parity, caste, income, religion, etc.  But that does not mean we had gender and equity covered.  Someone has to do and use the analysis. That takes resources and is a choice.

When I joined the foundation the baseline report had been written. It had some sub-group analysis but gender and equity had not been a core focus.

The program team wanted to reduce inequities but data was not being fully used to deepen this interest.

We ran additional analysis.  The following chart is one example of what we found.

Although we knew that most women (72%) on average were not getting adequate antenatal care (ANC), this analysis showed large variation among women.  Only 14% of illiterate women from scheduled caste, schedule tribe, or Muslim communities in the lowest economic bracket received three antenatal check-ups compared to 57% of literate women from less marginalized, and higher income brackets.

At this point the midline was upon us.  We were only weeks away from being able to examine whether inequities had been reduced two years on.

With more detailed analysis of inequities in hand, we asked the team whether they expected the program to reduce inequities, hold them constant, or widen them. This created a surge of interest in results for marginalized groups at midline.  Evaluation can lead as well as follow.

In hindsight the choice to deepen this analysis and use it to engage with programs sounds like just doing good evaluation.  And it is.  But the point is that it doesn’t just happen.  We make choices about what to look at all the time.

How did that story end?

Wonderfully. At midline, they had substantially improved outcomes for marginalized women in several areas including front line worker visits, breastfeeding, complementary feeding, and use of modern methods of family planning.  That’s not easy to do.

Obviously wide gaps still remain.  But the program teams have a renewed focus, enabled by measurement, on continuing to narrow those gaps.

Another thing we did in that midline was to questions on intimate partner violence.  The foundation was not working on intimate partner violence, so why do that?

Evidence cannot take us in new directions if we limit evidence collection to what we are already doing.

Anyone who has fielded a survey knows you can’t include everything.  My usual default is to include the most actionable questions.  By that measure, violence may not have been included.  But evaluation can’t challenge inequities if we don’t make space for them.

So I put in the questions.  Because I could.  And that’s the secret.

We found very high levels of violence (see table).  Almost 50% of the women surveyed reported physical abuse and 17% forced intercourse.  Among the youngest and most marginalized women, 76% reported physical abuse and 33% forced sexual intercourse. We looked at the relationship of this violence to health outcomes. Among several associations, we saw that women facing severe physical violence, though more likely to receive home visits from health workers were less likely to deliver at facilities, and more likely to give birth to low birth weight babies.[i]

I did not wait for someone to instruct me to ask those questions.  That’s not how evaluation works.  I included them because I believed if I had evidence on the factors and inequities underpinning health, my colleagues and partners working to improve women’s health would act.

I was right.

The data came out at a great time. The foundation was deepening focus on gender and equity and developing a Grand Challenge on Putting Women and Girls at the Center of Development. I could not have predicted this when I inserted these questions.  It doesn’t always work this way.  Sometimes people are not interested; that’s one reason inequities are so persistent.

Melinda Gates recently wrote in Science Magazine, that the Bill & Melinda Gates Foundation will be more intentional about addressing gender inequalities. I share my story to make the point that evaluation also has to become more intentional.

So don’t wait for a guideline to integrate gender and equity into your evaluations. Do it in stealth if you have to. That is how change happens.  My call to evaluators is to be accountable for equity. We are not observers; we are participants in the way development unfolds.   Take a position on equity and be active in measuring it with the tools of our trade.   If we do not bring a gender and equity focus to evaluations, that is on us.

Published in collaboration with Impatient Optimists

Author: Katherine Hay is the strategic lead on The Bill and Melinda Gates Foundation’s evaluations of programs in both Bihar and Uttar Pradesh, with decision making responsibilities for evaluation design, strengthening grantee organizations’ ME systems, and use of real-time monitoring data and evaluation findings.

Image: Girls stand inside their classroom at a primary school in Dobley town, 10 km (6 miles) from the Kenya-Somalia border, February 21, 2012. REUTERS/Thomas Mukoya