Experiences in Sri Lanka: pre-testing – a key component of any successful evaluation
By Alex Hamilton, GFP Oxford PhD scholar
Greetings Generations For Peace (GFP) Delegates and Pioneers, this is Alex again, the GFP Oxford PhD scholar. You may remember my previous blog post in November last year, when I was visiting Sri Lanka for the first time. I am exploring the links between public health and forms of peace-building evaluation to further both my own and GFP’s knowledge base in both fields. My project is examining the idea that the sport programmes for youth can have an impact on the physical and mental health of participants. It is often said that sport is good for your mind and your body – I want to put this claim to the test!
For this research we are collecting data on both physical and mental health in northern Sri Lanka. The variables we are measuring are body mass index, physical fitness and psychosocial wellbeing (for more information, see the links below). Measuring mental health is very difficult, because there is no perfect indicator. It is much like measuring ‘peace’, which I am sure all GFP volunteers will understand, as they have to measure the peaceful outcomes of their programmes all the time! Too often mental health is measured in post-conflict settings using Western ideas about behaviour and emotion. When preparing for my research, I found some great work in Sri Lanka that is using local ideas and behaviours to measure mental health – it is called the Sri Lankan Index of Psychosocial Stress (SLIPS). So, I decided to use this local mental health measure in my research to make it more applicable and useful for local, Sri Lankan context.
The stages of the proposed research follow a common framework used in public health evaluation, first outlined by Nutbeam and Bauman in 2006. I have adapted the framework and my plan has the following stages: Problem Definition, Solution Generation, Methods Testing, Intervention, Dissemination, and Maintenance.
Problem Definition is about identifying a key challenge. Based on what GFP programmes are aiming to do in Sri Lanka, I decided a key challenge for my research project was measuring the impact of programming on the mental health of the Target Group. Solution Generation concerns creating and pre-testing a solution to the problem. Through literature review, meetings with GFP volunteers, local stakeholders, and different Methods Testing I was able to select potential mental health impact indicators that were reliable and measureable in local conditions (the SLIPS).
In any research it is best to try and pre-test your ideas and methods. I visited Sri Lanka last year and carried out some independent work using the methods I had chosen for my GFP research project. Through this formative work I was able to make several informed changes to the data collection plans, in hope of improving my results. I found it useful to exchange ideas with local GFP volunteers on how ‘Methods Testing’ and ‘formative work’ in their programme context could be carried out. We discussed the most feasible and efficient data collection methods they could use when gathering baselines and endlines, and doing overall monitoring and evaluation (M&E) of their programmes. In spite of many barriers they face, they agreed that a preliminary testing (on a small sample!) of two or three data collection methods would still be the best way to ensure good data will be collected, and peaceful impacts their programmes are creating in local communities would be measured.
In March 2015, in what was to be my Intervention stage, I completed the first round of data collection in Maritime Pattu Division, northern Sri Lanka. To make my stay in Sri Lanka as successful as possible, GFP’s local partner, the Community Development Organization provided full logistical support for my research project. Local GFP Delegates, some of whom were selected to be on my small research team, worked tirelessly throughout the two-week data collection period – and we now have data on over 350 GFP programmes’ participants across seven schools. This is a great result, and it just proves what GFP volunteers can do when they put their hearts and minds together!
In the coming months, I will be comparing four schools where GFP peace-building programmes are taking place with three schools where GFP is not active. I hope to find some interesting connections, and will be returning to northern Sri Lanka in June to run the same tests again (last part of my Intervention phase). Hopefully, we will see improvements in the psychosocial well-being of GFP programme participants. So watch this space for some preliminary results – you will be the first to know!
For more information on what BMI is, check this website.
For an introduction on what physical fitness is, see this link.
For more information on psychosocial well-being, see this link.