The Ups and Downs of Mobile Data Collection
By Sairah Yusuf, Research, Monitoring and Evaluation Specialist, Generations For Peace Institute
Since May 2016, Generations For Peace (GFP) has used mobile-based data collection methods to survey close to 3500 people across the Middle East and North Africa (MENA) region – 1241 from 10 communities in Lebanon, 1274 across 24 school-based locales in Jordan, and 1073 across 3 governorates in Tunisia. For the organisation, this has been a massive expansion in terms of data collected via mobiles, and it has brought with it plenty of learning opportunities. Building on both our successes and our often-embarrassing hiccups, this blog post offers some practical tips for completing large-scale data collection using mobile phones.
Mobile Data Collection: A Solution to (Almost) All Our Problems
The advantages of mobile data collection have been well documented. Software providers like Magpi have published data collection guides that list out benefits in detail. The World Bank has a useful series on some of the advantages of using mobiles to collect data. Individual organisations like BRAC and The Hunger Project have written about the benefits of moving away from paper-based surveying. Building on the advantages others have documented, here are some examples of the benefits that GFP experienced:
. Saving money: Mobile data collection is cheaper than traditional, paper-based methods of data collection – not because mobile-based data collection software is cheaper, but because it cuts the expense of paying people for long days of surveying and tedious data entry. At GFP, using mobile data collection tools such as Magpi has allowed us to significantly scale up data collection at relatively little cost.
. Saving time: Collecting data using phones significantly reduces the amount of time that elapses between collecting data at the local level and storing it in an online database, in a manner that can be clearly analysed. In Tunisia, we gathered 555 surveys in 3 days of data collection (in one school, with 6 data collectors, it took us only 1 hour and 15 minutes to collect and upload 66 surveys to a central database).
. Real-time troubleshooting: Immediate updates make it easy for someone managing this data collection to spot potential problems as soon as they arise, and take immediate action to fix any issues faced. Our team has been able to quickly identify when a data collection site is experiencing trouble with uploads, or if data collectors deviated from a pre-decided sampling strategy – for example, by looking at the data coming in, we could see if people in one location had surveyed significantly more men than women.
. Improving accuracy: Mobile surveying can reduce the errors caused by transcribing people’s terrible handwriting. It helps make sure no surveys are lost in transport or collection. Most importantly, it significantly reduces the number of “unusable surveys.” In our case, we have been able to circumvent one of the major issues with paper-based surveys: we ensured that no one could “skip” key questions.
. Building on familiarity: People are used to using mobile phones – there is no need to spend time introducing the device to them, only the application that needs to be used. This means that the training required to help people use mobiles for data collection is often only about software, not about the hardware. At GFP, we collected data without arranging or paying for any extra equipment: all the surveying in Lebanon, Jordan and Tunisia utilised the phones data collectors already possessed and were familiar with.
Mobile Data Collection: Not as Easy as it Sounds
Hidden amongst all these positives is a set of challenges that people talk about less often. There are a series of overarching questions that need to be considered when shifting to data collection using mobile phones, to see if the undertaking is worthwhile for the organisation. Other challenges relate to the broader problems faced when carrying out a mobile data collection project: technological challenges, the need for specialised training, financial constraints, and concerns about data security.
The remaining set of challenges is overwhelmingly local and practical. The School of Data has a guide to mobile data collection with a section entitled “Common Mistakes in the Field.” These mistakes include enumerators entering incorrect answers in a hurry, misspelling information, and accidentally deleting half the collected files. At GFP, we faced all these challenges and more! For example, one enumerator insisted on taking personal phone calls during the surveying process, and was offended when asked to desist; another person accidentally signed out of the application and could not remember what email address was used to create the Magpi account in the first place. These things happen, and this post is aimed at helping others deal with these challenges.
Learning Opportunities: How to Confront Some Real-World Challenges
There is a great deal of advice out there on how data from mobile phones can be used to enhance development – there are detailed studies, training guides, and courses. This advice is incredibly useful in understanding how to set up and implement a large-scale mobile data collection project. But it does not always tell you what to do when you have collected 50 surveys on a single phone and even though everything worked fine during testing, suddenly you are unable to upload anything and your data collector needs to go home now and cannot come back.
Here are three important tips that helped us at GFP to deal with some of the practical challenges that arose during our expansion of the use of mobile data collection:
First, preparation is essential.
GFP employs a lean cost model when it comes to mobile data collection. Selected enumerators are typically not paid for their services. These enumerators are selected because they are active volunteers in GFP programmes and are in possession of a functional smartphone that can support mobile data collection software. In addition, local capacity building is a major priority. This means that GFP does not want to distribute pre-set phones or tablets, collect data, and then exit, leaving the community without devices they can use in the future. Allowing individuals to use their own phones, and giving them the ability to create and use their own login information, equips them with the both the knowledge and the hardware to utilise these skills elsewhere, in their own work. But this has its own challenges, as we do not have access to their phones and cannot set up anything for them in advance.
While this model is both cost-effective and empowering, it requires us to be extremely alert during the preparatory phase. In our experience, here are some of the things you can do:
. Schedule a solid period to train enumerators in mobile data collection. Just because they are using their own phones does not mean they do not need time to learn good surveying practices and the use of a new software.
. Alert the technical support services of your mobile data collection software provider, in case you need any quick help in the field.
. Do the legwork for your enumerators – make sure you have a full list of email addresses, passwords, and phone types noted down, so you can get them the support they need. They will forget these things once in the field – I would!
Second, technology matters.
. If the organisation is not bringing phones to the community, it is important to make sure the right technology is available in field. There’s no quick fix to be found when you have an Android-only application that you want to download to an iPhone.
. Some degree of technological literacy is crucial. Mobile data applications are designed to be simple for people who use smartphones on a regular basis – they become complicated for people who have never created email addresses or have never downloaded an app to their phone. This does not mean it is not possible; in fact, it can be incredibly rewarding in terms of equipping local community members with technical skills and capacity. However, it means the learning curve is higher and training time can take much longer.
. Technological testing is important under any circumstances, but it is even more relevant when you are working with phones that you have not encountered before. They could have a thousand glitches you are not familiar with.
Here’s what you can do:
. Ask selected people to send you a list of their phone models and operating system names before you bring them out for a training. If they do not have the right technology, you already know they are not the right people to train (unless they can get a hold of someone else’s phone). If they are not sure how to find out what their phone model and operating system is, maybe they are not the right people to train either, if you have limited time in the field.
. Test the technology thoroughly during training times – test it with and without internet, with and without battery chargers, indoors and outdoors.
Third, implementation needs time.
. Working with enumerators who are new to mobile data collection means that it will take a while to get everything off the ground, with a great deal of confusion, coordination problems, and technical trouble.
. People who are not paid will invariably have personal commitments and concerns that they will need to address, and you must be prepared for lack of attendance and sudden demands on their time.
Here are some steps to take:
. This sounds very basic, but: make sure you have more than the absolute minimum time required to survey everyone. This is because new data collectors, using their own phones, will always run into some strange technical problem that you have not thought of and could not reproduce even if you tried.
. Be very clear about what constitutes good and bad surveying practice – data collection is also a process of learning research methods, not just the use of mobile-based software.
. Make sure data uploads are done instantaneously, if you can find a way to provide internet through hotspot availability. This is important when you are working with people who are using their own phones, and who may need to leave quickly if there is an emergency.
. Identify “champions.” Natasha Beale from Equal Access International suggests that some individuals are more technologically savvy than others, and can be used to speed up mobile data collection training. The same is true for the actual data collection process; some enumerators will be able to help others deal with issues that arise in field, and data collection teams should be constituted keeping this in mind.
. Be ready for innovative troubleshooting. If surveys are stuck on an enumerator’s phone, can you retrieve them by looking up the answers and manually entering them into another phone? Does the software allow you to connect the device to a laptop and upload from there? You will need time to find a way around these problems.
For GFP, the key takeaway point is that using mobile data collection software like Magpi has allowed us to significantly scale up our ability to collect and analyse data in diverse contexts. With some of the tips and tricks mentioned here, it is absolutely possible to overcome some of the challenges inherent in using personal mobile phones for data collection, and – most importantly – to make sure that community members get to learn something along the way as well.
Sign up to our e-newsletter to learn more about the impact of our programmes in the Middle East, Africa, Asia and Europe.