15 ^th November 2011

They say the most dangerous part of the climb up Everest is the crossing the icefields between Base Camp and Camp 1. The ice is continuously shifting and crevasses can rapidly form under your feet plunging you into the depth of the mountain. What might appear to be stable, is in fact a river of ice. Skilled sherpas work tirelessly to find a safe route, fixing ropes and ladders to span the divides.

This is increasingly how I view academic disciplines. We believe they are stable and established. Those of us who are used to working across and between them often speak frustratingly of silos. They have long histories and deep foundations. But despite these things, I have found that they are constantly shifting, new divides emerge, and if you think you have completely understood them, then you are probably mistaken.

I trained as an Anthropologist, but not in a traditional department. I began by learning some social anthropology, some biological anthropology and some archaeology. My interest in science took over and I stuck with the biological course, but even this was eclectic. None of my lecturers were trained anthropologists, but instead brought together population genetics, primatology, epidemiology and biostatistics, human ecology and human evolution. As described earlier, I’ve gone on to an even more eclectic existence working with a wide range of disciplines. This is what I’ve learnt.

There are always groups of people who establish foundations. They are purists and sometimes also traditionalists. They know their discipline well – it is their academic home. I have learnt enormous amounts from them even though I am neither a purist nor a traditionalist. Whilst those at the core of disciplines often highlight what makes them distinct from others, I have observed with some interest the similarities between them. I’ve also learnt that it’s not always wise to point out these similarities, as people’s careers are built on the distinctions.

I’ve learnt that before deciding what is similar or different between disciplines, I need to learn a new language. ‘Field experiments’ in international development relate closely to ‘randomised controlled trials’ in health care. ‘Observational’ studies in anthropology involve literally watching people, in other ‘sciences’ they are any study which isn’t specifically trialling a new intervention. Indeed the word ‘trial’ can mean anything from a pilot phase to a full blown experimental study. Acronyms are even more tricky. I remember well the small group discussion about “CSWs” which descended into laughter when we realised that some people were talking about commercial sex workers and others about community social workers – you can just imagine the crossed wires with regards who was being paid for which services.

I’ve come to appreciate the essential role of those who lay the ropes and ladders across the creaking crevasses. Working across disciplines requires skilled facilitation. It isn’t merely about one side of the divide moving a little closer to another (although I have seen this happen). Invariably there is a need for a third party to help the two sides to engage and relate to one another. And if the facilitator has a strong bias towards one side, the facilitation can dissolve from a boundary-spanning exercise to a boundary-establishing one. Observing poor facilitation has helped me appreciate all the more what good facilitation looks like and how much it can achieve: I believe (and yes, I am biased) that boundary spanning is an essential specialism in its own right and should be recognised as such.

I believe that it is at the edges where new learning occurs. Taking on new perspectives, new lenses, helps us to see the world in different ways. Interacting with those who are different from us helps us find more creative solutions. Despite what some appear to believe, no single discipline has a monopoly on truth. If academic endeavours are to advance learning and enhance our world, then they need to be constantly shifting. The slow progress of the glacial icefield needs to continue with its shifting boulders and creaking divides

Lastly, my experience of working across disciplinary boundaries seems to frequently leave me stuck. The foundations, principles, history and language within disciplines all lead to moments of impasse between disciplines which can seem impossible to overcome. Working across boundaries involves taking on challenges and finding ways through. I believe the kind of stuckness that occurs when you work across these ravines is to be sought out – it is where I learn the most. It is because of these challenges that I choose to work across disciplines.

[PS. I think the best advice on difficult stuff comes from Pirsig’s Zen and the Art of Motorcycle Maintenance:

"Stuckness shouldn't be avoided. It's the psychic predecessor of all real understanding. An egoless acceptance of stuckness is a key to an understanding of all Quality, in mechanical work as in other endeavors."]

17th October 2011

In June 2009 I began a 12 month placement at the UK’s National Audit Office as part of the ESRC’s placement fellowship scheme. I thought I was going to complete a specific project. I was relieved to be having a break from academic politics and pressures and nervous to be back at work after 5 months off. I didn’t know whether what I knew would be valued, and I understood little of audit or government.

I suspect many boundary spanners feel the same – unsure what they have to bring to the new environment, very aware of what they do not know and unsure about whether they will be welcomed. In the end, my ‘project’ took a back seat and instead I made myself useful – taking on tasks whenever I was asked, helping overstretched teams and eventually  filling the need for a qualitative methods lead. In doing so I learnt more than I imagined, took away very different things from what I expected and ended up with a new job.

I will write next time about what (I hope) I brought to the work of the National Audit Office. Here, I just want to summarise how my research benefitted from my time there.

I learnt where my research and academic research more broadly sits within a wider spectrum. I realised that academic research is only one aspect of the research industry (and at times the most out of touch) and saw how research does (and doesn’t) fit within government policy and practice. I realised how naive I had been about the role of research in decision-making and understood the compromises made to make research useful and attractive to politicians and other decision-makers. I experienced first-hand the tensions between quality, relevance and application of research and realised the inadequacies of much academic research for the requirements of auditors and decision-makers.

I learnt (a little) about how UK government works through working with teams across government departments. Having spent the previous ten years in a university department which aims to research and facilitate evidence-based decision-making in policy and practice, I thought I knew a little about policy-making. But I learnt a lot more. Perhaps most importantly I realised that academics have a tendency to lump ‘policy and practice’ together without unpacking what they mean. Researchers need to engage more thoughtfully and realistically with the work and the people they hope to influence.

I learnt (somewhat reluctantly, I admit) of the importance to decision-makers of financial data, as well as data on process and impact – none of my past research had reported the financial cost of interventions. Similarly I witnessed first-hand the emphasis on quantitative data within decision-making and saw how even poorly understood quantitative data is given considerable weight because of the allure of large data sets. I learnt how ideas of quality and rigorous analysis need to be presented sensitively to raise the standard of and attitudes to both quantitative and qualitative research within a community of non-researchers.

Lastly, I learnt how valuable it is to work in a different environment and learn about other cultures. This brought greater perspective to my career, provided me with a new network of colleagues, and gave me new energy and enthusiasm for my work.

So, are there lessons for others in this? Well, I learnt the value of stepping outside my comfort zone and spanning a boundary, and I gained the most when I didn’t have expectations, could appreciate new learning even when unexpected, and was willing to be flexible. Was my placement a success? Well, my research certainly seems to be having more impact now than it did in the past and as for me, well, in the end the bridge I built became a permanent one as I now have two jobs – one in academia and one in national audit.

27th September 2011

Here’s the first of a series of blog entries on bridging divides. They are designed to help me think through the range of boundaries I work across, the challenges they bring, why and how I try to span them, and what difference any of this makes (if any).

Ridiculous though it may seem, it’s taken me some time to realise that this is what I am – a boundary spanner – possibly not as useful as the one in your tool box, but hopefully still effective in some ways. I try to bridge divides, a huge range of them. To give you a feel for which ones, here’s a brief bio:

I’ve lived and worked in Africa, South East Asia and Europe. I grew up in rural Malawi, I currently work in London. Trained in anthropology, I work with sociologists, psychologists, medics, nurses, midwives, teachers, biochemists, statisticians, epidemiologists, accountants, geneticists… I work in academia with posts at the Universities of London and Johannesburg, as well as for the UK’s National Audit Office. I work with researchers, their funders and those they research. I work with policy-makers, practitioners, and service-users. My work has included conducting research (and supporting others to do research) in health care, health services development, public health, adult and child education, transport, defence, tax, governance, communications, the environment, international development, business regulation and more. I’m currently working in microfinance and financial inclusion. I use mixed methods including both qualitative and quantitative techniques. I have a PhD in multi-disciplinary, participatory, international education. If you play that game of who has visited the most countries, I tend to come out on top (I only know of one person who has travelled more). My maternal great-grandfather lived in Kenya, my paternal grandparents in India, my parents in Malawi, my siblings on various continents at various times. In any one month, at least one member of my family is guaranteed to be in another country. There is a constant stream of international visitors through my London flat. I’m a poor linguist: I smile in many languages.

I could go on, but by now you probably have an impression of a restless spacehopper bouncing about, and I’m not sure that would be accurate. I prefer this image.

There are varied academic literatures on crossing boundaries, from linguistics, education, psychology and international relations – you may know of more. And there’s a growing interest in ‘knowledge translation’ in the area of evidence-informed decision-making in which I currently work. Over the next few months, I am looking forward to exploring some of the boundaries I try to span and discussing the challenges inherent within them.

24th August 2011

I’ve started contributing to an internal blog for the National Audit Office on methods and thought I’d share some of my posts here too. I hope they are of some interest. This one seemed particularly relevant to a wider audience….

Anecdotes are interesting observations. They are selected because they are noteworthy, unusual, exciting or controversial and, as a result, they are influential.

To give you an example we’re doing research on microfinance at the moment. Most of the accumulated evidence on the impacts of microfinance is anecdotal – stories of poor men and women who had no means of generating an income and with the provision of a small loan from a micro-credit organisation were able to lift themselves out of poverty and live happily ever after. (Just one of many examples here: http://www.howwemadeitinafrica.com/banking-on-kenyas-women/2701/)

These carefully selected anecdotes have led to the growth of a multi-million dollar industry out of a small scale development initiative (and the award of the Nobel Peace Prize to its creator). An industry which makes money out of the world’s poorest people.

Anecdotes are not data, however many of them you have. ‘Data’ is not, nor has it ever been, the plural of ‘anecdote’.

As cracks have begun to emerge in the microfinance industry there have been calls for independent rigorous evaluation. Last year we published the first systematic review of high quality evidence on the impacts of microfinance on the poor. We found that when you set aside the anecdotal evidence, it is clear that microfinance causes harm, as well as good, making some people poorer, leading to failed businesses, and negatively impacting on the Millennium Development Goals. Anecdotes might be powerful, but in the case of microfinance they have been powerfully misleading.

What anecdotes do achieve is to communicate a story in a meaningful way allowing you to quickly generate understanding and empathy in your audience. The press know what they are doing when they select ‘human stories’ to explain what is happening in the world.

So this is our challenge: 1) to conduct rigorous, unbiased analysis of data, and 2) to harness the power of the anecdote to communicate our findings clearly. We need to make sure these two activities remain distinct, and occur in this order so that our conclusions are accurate and have an impact - a challenge for academics as well as government auditors.

30th June 2011

Over the last month, colleagues and I have submitted 5 papers for publication, finalised and published our new systematic review protocol, and one of us has even submitted her excellent PhD thesis for examination – Congratulations Carina! I don’t know what I did to deserve such an amazingly productive team, but I’m very grateful for such great colleagues. On top of that I have been privileged to be able to establish a network of talented qualitative researchers within the UK’s National Audit Office – and yes, that is as radical a step as it sounds: that an organisation of over 600 accountants is willing to engage with qualitative evidence is credit to some forward thinking leadership and a great director who is willing to give me the freedom to develop a programme of qualitative work. It’s been an extraordinary time.

To celebrate, I’m off to walk 240km of the Camino Portugues  through Portugal and Spain. I know it sounds exhausting, but I can assure you walking the Camino is the most restorative activity you can find. I look forward to returning with a clear head and some exciting new plans. See you on my return!

1st April 2011

I’ve just been to a two-day meeting of NONIE in Paris. I’d not been aware of the ‘Network of Networks on Impact Evaluation’ before, but I found a group of people I felt very at home with. I presented our systematic review of microfinance which led to some interesting discussions.

One issue which came up was the number of studies initially found in our searches and then excluded.  When an audience hears that you found over 6000 references but excluded all but 15, they get concerned about what you are leaving out. So… I’ve been thinking about it and have revisited the details of what we excluded from our review. In sifting the evidence, we employed six different filters:

1. Searching

We searched broadly in order to capture as much potentially relevant research as possible and identified over 6000 references. This is not an unusual volume of hits for a systematic review. Although unlikely, it is feasible that we missed some relevant research. Whilst we haven’t explicitly measured whether we have missed any literature, once they are published we will be able to check our included papers against the other related reviews on microfinance currently underway.

2. Screening on title and abstract

We screened initially on title and abstract and were deliberately over inclusive. As a result of this we reduced the number of potentially relevant studies to 383. I am not very concerned that we excluded any relevant studies at this stage as if we were in any doubt we included the research and sought the full texts.

3. Collection / availability of full texts

Of these 383, forty seven reports were unavailable within the timeframe of the review. An increased budget to pay for interlibrary loans would increase the chance of accessing these hard-to-reach reports, although even using interlibrary loans can take longer than the time available.

4. Screening on full texts

Of the 336 full texts we obtained, we excluded 245 for the following reasons:

-       9 were not based in sub-Saharan Africa

-       51 were not microfinance

-       111 were not outcome evaluations

-       49 did not consider outcomes relating to the poor

-       (25 were secondary papers describing already excluded studies)

Had our review been focussed more broadly by any of these criteria, these reports may have been useful. Given the focus of our question on the impacts of microfinance on the poor in sub-Saharan Africa, I have no concerns about excluded these 245 studies.

5. Screening for comparison groups

By now we had accounts of 69 studies (described in 91 reports). We then screened these to identify which had comparison groups as this was our baseline requirement in terms of methodology for assessing impact. We excluded 34 studies which had no comparison group. From the perspective of anyone who doesn’t agree with basic experimental design, excluding these studies may be problematic. We listed these in Appendix 3.1 in case any of our readers were interested, but I am not concerned that we excluded any relevant evidence at this stage.

We briefly described the remaining 35 studies in what we called our ‘map of the literature on microfinance in SSA’ (see Appendix 3.2).

6. Quality appraisal

We then assessed the remaining 35 studies for their quality. At this stage we excluded 20 studies for the following reasons:

-       12 for lack of information which made it impossible to assess the quality of the study or interpret its findings

-       6 for poor methods

-       2 for both lack of information and poor methods.

We were relatively generous in our judgement of quality (Duvendack et al apparently excluded some of the studies we included), so I am not concerned that we excluded any useful studies at this stage.

I am a little concerned about the 12 studies we excluded due to lack of information alone. With more time it may have been possible to track down more details about these studies. However, this relies on authors to respond to requests for information and even if they did, we would then assess the studies for the quality of their methods and may exclude them on this basis. We were therefore left with 15 studies in our review.

So, on reflection, should readers be worried about the thousands of studies we found but then excluded from our review? With the caveats above about the research reports we were unable to obtain, and those in which essential details weren’t reported, I am confident that the 15 studies we included in our review were the relevant and high quality studies within sub-Saharan Africa on microfinance and our findings therefore a true reflection of the evidence from the region.

Should you be uneasy about the excluded literature in any review, it is the inclusion criteria used for screening and the quality appraisal criteria which need critiquing. These are however set very early in the review process as they have implications for the initial searching. The need to consult on the scope of the review early in the process and to have the protocol peer reviewed at the start are therefore paramount. It is also worth noting that if the inclusion criteria are broadened, the amount of work required to complete the review can expand exponentially, and the usefulness of the review may be reduced.

5th March 2011

I thought I’d be writing about what it was like to stand on the roof of Africa, but things didn’t quite go to plan.

I’ve just been to Tanzania to climb Kilimanjaro, but after 4 days ascending to just under five thousand meters I had to come down urgently with severe altitude sickness. They call it HAPE (high altitude pulmonary edema)… basically my lungs filled with fluid making it very hard to breathe. You have around twelve hours to get back down to a reasonable alititude before the risk of death becomes dangerously high. I’ve had mild altitude sickness before in the Himalayas – this was something altogether different. I was very sick, I relied completely on a wonderful mountain guide to bring me down, and it took me most of a week before I could breathe normally again.

The experience was new for me. I’ve always been one of those people who can’t resist a challenge. I am strong. I like nothing more than training with men down the gym because they push me hard. I achieve what I set out to do. I am stubborn. This all made it a very humbling experience to have to turn back on the mountain and to watch others walking on.

I am also independent. I live on my own. I am used to looking after myself. To rely on someone-else to save my life was extraordinary and wonderful. Every time I had to rest on the slow walk down there was someone there, hand on my shoulder, waiting for me to gather my energy and lean on him again as we descended. I have never been taken care of better.

And then, a few days later I had to gather my physical and emotional strength when I headed to the trail-end to meet my new friends off the mountain after their long trek to the top. Facing that challenge, perhaps harder than any mountain I’ve ever climbed, was a rewarding learning curve in itself. The porters who had been with us on the mountain came down first and each one came over to ask me how I was and to offer their sympathy. The driver who waited with me brought me drinks and found me shade to rest in. People offered to join me on another climb whenever I wanted to return. When I spoke of how much I loved Tanzania, I got a proposal of marriage. And then my friends came down and showed me their photos from the top – on the peak of Uhuru they are pictured holding a sign with my name on – they had taken me with them in spirit. That evening we all went dancing – we had a lot to celebrate.

So here’s to mountains, high and low. To summits, past, present and future. To new friends. To  teamwork which crossed cultures and language and spread across the mountainside to echoes of WEH! and cries of the extraordinary motto of our Tanzanian hosts MAXIMUM RESPECT.

To Kilimanjaro, the great white mountain of Africa.

And to Whitey, asante sana. I owe wewe kila kitu.

31st Jan 2011

Very pleased to have completed the first systematic review of the impacts of microfinance to be published.

I look forward to discussing our findings and seeing the findings of related reviews in due course.

Happy Mondays!

Ruth

29th Jan 2011

Carina is over from Johannesburg at the moment and we’ve been busy proof reading the fancy version of our microfinance report. The designer has done a lovely job.

So, we are feeling quite excited about the publication of our glossy new report and its imminent publication. With microfinance in the media this week (on BBC’s Newsnight, and Radio 4’s From our own correspondent) we are anticipating some interesting discussions. We will be presenting our work and discussing our methods at a seminar on the 2nd Feb. For those of you who would like to see what we found, here are the summary and policy brief.

I look forward to hearing what you think.

19th Dec 2010

At the recent Cochrane and Campbell Colloquium I picked up a copy of the South African Medical Research Council’s Evidence in Action training tool – a CD-based game designed as part of an evidence-based reproductive health care package, but which ‘can also be used as a stand-alone learning tool’. The game is straightforward to play, has beautiful graphics and carefully chosen references to the challenges of working in rural health care.

It is unfortunate that it carries some of the excluding characteristics which it clearly hopes to counter – despite being a systematic reviewer with experience of working in rural reproductive health services in South Africa (albeit ten years ago), there were acronyms which I did not understand and terminology I was not familiar with.

The strong emphasis on systematic review evidence is to be expected, and something I support. But I found the narrow focus on aspects of Cochrane reviews a little frustrating – I had hoped for some more general references to the value of evidence and how the quality and relevance of such evidence might be appraised, interpreted, and applied to local decisions rather than the details of forest plots as a means to summarise statistical meta-analysis.

Wary that this ‘stand-alone’ training tool didn’t seem to be hitting the mark for me, I looked for more information on the target audience for this CD. There is little information provided about the game, so I turned to the links along the bottom of the screen: the Term of Use, the SA Cochrane Centre and the Acknowledgements. It turns out that to access this information about the game so carefully designed to be culturally and contextually appropriate to rural health services requires access to the Internet. As I sat in rural South Africa without access to the internet, I felt excluded and confused.

This game promises so much – and I am encouraged by the development of resources to train and support systematic reviewers – but I hope the next version is more inclusive, in content and design.

24th Oct 2010

The joint colloquium was a great success. Beautiful setting up in the mountains and probably the best plenaries I’ve ever heard. I always think that a conference is a success if I make one good connection and learn one thing. Some highlights from this one have been the amazing knowledge transfer programme in Wisconsin www.evidencebasedhealthpolicy.org and Ray Moynihan’s work exposing the pharmaceuticals’ involvement in defining disease. Listen to his excellent plenary here http://bit.ly/ctjc5P  And last, but not least, great to meet the lead author of an overlapping systematic review on microfinance and share experiences and papers.

Colleagues have all headed home now and I’m here in the mountains for a few days writing and walking… and it’s just started snowing. Heaven.

11th Oct 2010

Off to Keystone at the end of the week for an indulgence of discussions, presentations and posters with a whole group of others who know what systematic reviews are. Sometimes I think these colloquia are basically concentration of geeks doing their thing well. But that is also why we go… because it’s great to be amongst colleagues who strive for the highest quality and most rigorous research they can imagine, and do it well.

I’m presenting a poster on knowledge translation. It’s with the marketing guy at the moment being type-set. As soon as it’s finished (and assuming it’s not slated at the conference), I’ll have a link to the pdf here.

Maybe see you there?

5th Oct 2010

Okay, so I haven’t posted anything here for ages – take it as a good sign that there is LOTS of work to be done.

Anyway, since getting interested in banking and the unbanked I’ve been reviewing the evidence for and against microfinance in sub-Saharan Africa with colleagues from the University of Johannesburg. We’ve been looking at impacts of micro-credit and micro-savings on the poor. We’re still in the early stages of the analysis for our systematic review but so far we have found:

1. There is more good quality, rigorous reserch from the region than we expected

2. The evidence suggests that microfiance is doing harm as well as good

Next task is to write this all up in detail – watch this space!

16th June 2010

I’ve just started working on two different pieces of work about the ‘unbanked’ and am beginning to realise how lucky I am to have access to credit and even just a safe place to keep my money. I know that the financial sector has its own crises and don’t want to downplay the impact it is having on many people’s lives, but there are also millions of people who have no access to banking at all.

I am off to South Africa soon to start working with the University of Johannesburg on a project funded by the UK’s Department for International Development. We will be reviewing the evidence about the impact of micro-credit on poverty in sub-Saharan Africa. Any initiative which warrants a Nobel Peace Prize (http://nobelprize.org/nobel_prizes/peace/laureates/2006/press.html) must be worthy of thorough examination.

I look forward to sharing more about what I learn in due course.

27th March 2010

The evenings are lighter and Londoners are coming out of hibernation. Work seems to be busier than ever with lots of writing to get done with a new book chapter nearly ready.  And if you’ve ever done a systematic review, you will appreciate the long slog of screening thousands of articles for potential inclusion in a review – I’m busy reading papers about critical appraisal training for a Cochrane Review at the moment. It’s not the most exciting task, but getting there.

I’ve also just taken part in a pilot exercise using Multi-Criteria Decision Analysis at the National Audit Office. MCDA is a new approach to me and something I’m looking forward to developing for use with a range of stakeholders.

I’m off trekking in Nepal next week to clear some cobwebs away. And will be back to work before long.

Wishing you a very happy Easter.

22nd Feb 2010

Well, the winter seems to continue - a good time for getting my head down and catching up on work. I’m developing some new guidance for the National Audit Office on using qualitative methods in their value for money work, whilst helping them explore the potential of Multi-Criteria Decision Analysis. All interesting stuff.

Exploring opportunities for my next project too – lots of interesting possibilities. Watch this space.

11th Jan 2010

Well, like you, I’m back at work after the Christmas break. After two week’s windsurfing in the sunshine the cold in London has been a bit of a shock but the snow is beautiful too.

It’s a busy start to the year with some interesting training to prepare – for the National Audit Office on participatory approaches in multi-criteria decision-analysis (MCDA), for the Institute of Child Health on involving parents in the development of national leaflets for newborn screening and for the Institute of Education adapting our our MSc module into on a fully online course. All interesting work.

I hope your New Year is happy and productive too.