How to Choose a Mobile Data Collection Platform

Type:
Guide

There are dozens of applications and platforms to collect data using mobile phones and other devices. Choosing the right mobile data collection (MDC) application or platform can save your program time and money, and it can help capture the right data at the right time to help make the best program decisions.


Description

Data is an integral part of many technology-enabled development programs. Various companies and organizations have developed mobile applications and platforms that can help collect and analyze data, and choosing the right tool can set your program up for success.

MDC speeds up the data collection and analysis process because data can be submitted in real time, becoming available immediately for viewing and sharing. Initial costs can be higher than paper-based data collection because you may need to procure tablets, smartphones or, in some cases, software licenses. But over the life of a program, as you continue to use the devices, you can often save money by removing recurring costs, such as printing and data transcription.

Features of MDC platforms vary significantly. These features include compatibility, connectivity requirements, data security protocols and processes for data export. MDC tools often have features like skip logic and minimum or maximum value parameters that can lead to better data quality and reduce the time required to clean data. The tools also provide the option to capture other kinds of information, such as GPS coordinates and photographs.

This guide outlines steps to help you select an MDC platform. These recommendations are summarized from existing references, which can be found in the Resources section.

Process

  1. Consider data needs. The first step is to figure out what kind of data is needed to support program implementation, monitoring and evaluation. In many cases, you must also identify what kind of data is not needed, so that unnecessary features are not included in the selection process. For example, some MDC tools are designed for initiatives whose primary goal is to visualize data, such as an education nongovernmental organization that is helping the government to create a more complete map of formal and informal learning centers. The defined data needs of this NGO would prompt it to look for an MDC platform that makes it easy to collect GPS coordinates and has integrated mapping features. Another consideration is if you will collect personal or sensitive data and must ensure that the platform has the necessary data privacy and security features.

Tip: If most of the necessary data are qualitative instead of quantitative, an MDC platform may not be the right choice because most options do not have good support for long-form submissions. Surveys that feature short open-response questions (such as name and occupation) are well-suited for MDC. But MDC may not be appropriate for an interview or focus group that will ask for long narrative responses, as open-response fields in MDC platforms often have limits on the number of characters that can be entered.

Tip: Note the types of questions (e.g., open answer, multiple choice, ranked) that you plan to include in your survey, and use them as criteria for selecting your platform. Also determine if you need to collect dates, times and GPS coordinates.

  1. Consider the ecosystem. Ecosystem factors, such as infrastructure, security and the technology market, should be considered when selecting an MDC platform. For example, is a stable internet connection available, or do you need a tool that works offline? If crime and violence are concerns, consider if or what mobile devices would be safe for a data collector to carry. If your data collection plan includes having individuals submit data using their own devices, ensure that you understand the devices that are commonly used and that your platform is compatible with those devices. You will also want to consider if or how individuals will be able to connect to a mobile or internet connection suitable for uploading collected data. If internet connections will be slow, you may need to consider collecting “light” data that does not include large files like images and audio.
  2. Identify and prioritize selection criteria. In addition to data needs and ecosystem considerations, several other factors could influence selection of an MDC platform (note that this list is not exhaustive and selection factors should be tailored to your specific context):
    • Short-term and long-term costs: What is your budget? Understand license, device, maintenance, support and training costs associated with the MDC platform.
    • Number of users, surveys and items: How many people will be using the MDC application? How many surveys will you upload? How many items will you be collecting data on? Some MDC platforms provide licenses for an unlimited number of users and unlimited number of surveys uploaded to the server, while others offer a per-user licensing model. If you have a lengthy survey, you may not want to use an SMS-based system, as data entry can be rather time-consuming.
    • Devices and data requirements for enumerators: What devices will data collectors be using? Some MDC tools work best on tablets, while others only work on certain operating systems, such as Android. If your data collectors will be using devices that they, or their organizations, already own, ensure that these devices are compatible with the platform you select. Also consider what the minimum standards would be for an airtime or data plan needed to operate the MDC platform. For example, if your collectors only have basic phones, you will want to use a platform that collects data via SMS or Unstructured Supplementary Service Data (USSD). If you are procuring new devices, consider devices that have sufficient battery life so that they do not need charging during a long day of data collection. Also consider how much training or technical support will be needed based on the device selected.

Tip: Be sure to test several devices in your context before making a final selection. Dimagi has published a matrix that shows devices they have tested for their MDC tool, CommCare.

    • Security and privacy compliance: What data security standards and procedures does your organization follow? Does your country of operation have national data protection regulations that you need to comply with? How does each tool protect user data, hide personally identifiable information and so on? Do you want location information to be tracked, either through GPS or geographic information systems (GIS) embedded in images? For example, you may need a tool that encrypts or anonymizes data as they are entered. This will be important if a device is stolen and if any information that could be recovered would be sensitive or would compromise the safety, security or privacy of respondents. This is also an important consideration if a data collection device is shared within an organization or if it is an individual’s personal device that family members may share.
    • Integration with other technology: Are you already using any platforms for data analysis, reporting or mapping? For example, does your organization use a companywide platform for reporting on common indicators? Consider if your data collection platform can easily work with the analysis and reporting platform you want to use or if a lot of time or steps would be required to transfer data.

Tip: SMS data collection can be useful when those submitting data do not have access to smartphones, but it also limits the types of data that can be collected.  Consider if the data you want to collect can be submitted sufficiently within the character limits of an SMS message. Successful SMS-based data collection efforts have asked for data on a small number of simple indicators. For example, Plan International Uganda piloted an SMS-based system for student councils to report when teachers were absent.

    • Offline collection: Is internet connectivity limited in your program area? Is it important that people can enter data when they don’t have an internet or mobile broadband connection?
    • SMS integration: Do users need to get SMS or push notifications? Will they be submitting data via SMS? Some platform providers will help you set up an SMS short code (a four or five digit number often easier to remember than a full phone number), making it easier for users to submit data, or reverse billing so that sending SMS messages is free for the user.
    • USSD integration: Beyond SMS, Unstructured Supplementary Service Data can provide a way for users to submit data when they do not have smartphones. USSD is the system used to add mobile credit to your account and is prompted by dialing *number#. RTI International used the Praekelt Foundation’s Vumi Go platform to operate Mobile Gateway Zambia. This USSD-based system allows schools to use any mobile phone to submit regular reports, including school statistics and student examination results, to education officials.
    • Authentication and user roles: Who is collecting data? Some tools allow anyone to download an app or access a link and start submitting data. This is useful for citizen data collection efforts. Other tools allow an administrator to assign privileges and survey instruments to specific people. This is useful if you have a cohort of trained enumerators who will enter data but do not need to edit the survey or view aggregated data.
    • Skip logic and data parameters: Do you want to customize survey questions based on previous answers or restrict types of data entered? Some tools allow skip logic, meaning certain answer choices will prompt enumerators to skip several questions or a whole section, while other tools allow you to program questions so that enumerators must enter a value within certain parameters, such as a range of numbers or dates.
    • Data analysis: Do you need results displayed in real-time charts and graphs? Platforms vary on whether and how they let users export and analyze data. Some platforms offer dashboards with charts and graphs that are immediately generated as data are collected. Others require data to be exported before they can be analyzed or viewed.
    • GIS and mapping: Do you want to map results? Some MDC platforms are built specifically to collect GPS coordinates for mapping.
    • Language: What languages will be used by data collectors and respondents? Some MDC platforms have been developed only for certain major languages like English or French, while others are not compatible with non-Latin scripts like Arabic. Determine if your platform is available in the language you need. The language of your survey could also affect what kind of data you can collect through SMS. For example, sometimes phones owned by average users support only Latin scripts but not characters in the local language. Text messages in some languages are also restricted to 70 characters, rather than the 160-character standard for Latin scripts.
    • Photos, audio and video: Do you need to capture media or play video or audio as part of the survey instrument? If so, this will increase the size of your data files and will require sufficient bandwidth to upload them. Also consider how you will be able to gather informed consent from participants before taking photographs, video or audio.
    • Ease of setup and use: How sophisticated are your users? How much technical support do you have? MDC tools vary in terms of the interface and processes for building survey instruments, entering data, exporting data and uploading data to a cloud server. Some are very intuitive, while others may require more extensive training and support.
  1. Research MDC platform options. Once you have identified your key selection criteria, begin to research tools that appear to meet your requirements. Talk to others working in your ecosystem to understand the MDC platforms that they have used and the lessons that they have learned.

Tip: The Resources section includes lists and descriptions of MDC platforms, such as NOMAD’s Online Selection Assistant (which has a set of 14 easy questions that will help you pick from 49 possible data collection tools) and Pact’s Mobile Technology Handbook.

  1. Rank options. Once you have determined the selection criteria, create a selection matrix to compare options (see Figure 1). Include “must-have” or “nice-to-have” features, and assign a rank or weight to each feature.

Tip: See U.S. Global Development Lab and FHI 360’s Paper-to-Mobile Data Collection: A Manual for an example of a weighted software selection matrix.

Tip: Another option for selection is to create an MDC tool checklist for tracking an extensive list of criteria. See Magpi’s Mobile Data Collection Guide for an example.

Tip: If possible, test several potential tools to see if they meet your needs in practice. Testing can save time in training and implementation, and it can limit frustrations later.

Figure 1: Sample software selection matrix

SoftwareMust-haveCompatible with Amharic scriptWorks offlineCaptures GPS coordinatesCaptures photosNice-to-haveDashboard with charts and graphsRequires limited or no IT support
Option 1XXXX
Option 2XXX
Option 3XXXXX
Option 4XXXX
  1. Consider whether to customize an MDC platform. In rare situations, you may find after researching and comparing options that no standard tool fits your needs. In that case, consider working with the open source community or a developer to customize an existing platform to get all the features that your initiative needs. If you choose to customize, be sure to build in sufficient time for planning, designing and developing custom features, and ensure that the features can be reincorporated into the open source platform being used.
  2. Select and test your platform. Using the information that you have compiled on your data needs, ecosystem, additional selection criteria and available platform options, select the MDC platform that is the best fit. Test the selected tool with intended users to see if they have challenges or concerns before making a final decision.

Outcomes

These outcomes are illustrative and have been collected from digital development organizations that have followed the steps outlined in this guide.

  • Improved understanding of the types of data you will collect and the most important features needed in an MDC platform.
  • Improved ability to select the MDC platform that best fits your needs and will work in your ecosystem.
  • Improved data quality.
  • Increased efficiency of data collection and analysis.

Common Missteps

  • Choosing a tool solely based on price. A tool with low or no licensing fees, such as an open source tool, may seem like the most attractive option. But choosing a tool that lacks features essential to your initiative could affect data quality. Be sure to consider the total costs over the life of the tool, not just the initial costs.
  • Not testing a tool prior to implementation. While an MDC platform may have worked well in another context or for another program, challenges could arise when it is applied in a new context with different levels of connectivity or when using different devices. If possible, test multiple platforms in the communities where you will collect data before making a final selection. Alternatively, running a small pilot prior to a large data collection activity can allow you to identify any quirks or challenges with the software and account for them in training and during full implementation.
  • Not planning for scale and sustainability. A tool that works well for a small initiative may prove challenging to scale or sustain due to factors like cost and required devices or technology infrastructure. When selecting a platform, consider the future data collections needs of your organization or program and whether the tool and devices you are selecting for one data collection activity could be reused for future surveys. Also, dedicate sufficient time to train local staff so that they have the necessary skills to manage the platform, capacity-building activities, and future replication and reuse.

 


Resources

Alidade: A Guide to Choosing Technology Wisely for Your Social Change Project, The Engine Room. https://alidade.tech/

CommCare Help: Recommended Phones and Choosing a Phone, Dimagi. https://confluence.dimagi.com/display/commcarepublic/Recommended+Phones+and+Choosing+a+Phone

Mobile Data Collection, BetterEvaluation. http://www.betterevaluation.org/en/evaluation-options/mobile_data_collection

Mobile Data Collection Guide, Magpi. http://home.magpi.com/cta/mobile-data-collection-guide/

Mobile Data Collection Presentation and Training Materials, American Red Cross. https://americanredcross.github.io/mobiledatacollection/

Mobile Technology Handbook, Pact Inc. http://www.pactworld.org/sites/default/files/Mobile%20Technology%20Handbook%202014.pdf

Online Selection Assistant, NOMAD. https://humanitarian-nomad.org/online-selection-tool

Paper-to-Mobile Data Collection: A Manual, U.S. Global Development Lab and FHI 360. https://www.fhi360.org/sites/default/files/media/documents/Paper_to_Mobile_Data_Collection_Manual_1.0.pdf

Training Guides, Open Data Kit (ODK). https://opendatakit.org/help/training-guides/

Using Mobile Data for Development, Cartesian and Bill & Melinda Gates Foundation. https://docs.gatesfoundation.org/Documents/Using%20Mobile%20Data%20for%20Development.pdf