Design for Scale
Achieving scale is a goal that has been elusive for many digital development practitioners. The mHealth field, for example, has identified the problem of pilotitis, or the inability to move initiatives beyond pilot stage. Achieving scale can mean different things in different contexts, but it requires adoption beyond an initiatives pilot population and often necessitates securing funding or partners that take the initiative to new communities or regions. Different implementers may define scale as reaching a certain percentage of a population or a certain number of users. Designing for scale means thinking beyond the pilot and making choices that will enable widespread adoption later, as well as determining what will be affordable and usable by a whole country or region, rather than by a few pilot communities. You may need to evaluate the trade-offs among processes that would lead to rapid start-up and implementation of a short-term pilot versus those pilots that require more time and planning but lay the foundation for scaling by reducing future work and investment. By designing for scale from the beginning, your initiative can be expanded more easily to new users, markets, regions or countries if the initiative meets user needs and has local impact.
- Plan and design for scale from the start.
- Develop a definition of scale for your initiative.
- Keep your design simple, flexible and modular to make it easy to change your content and adapt to other contexts.
- As you make technology choices, think about whether they will make it easier or harder to scale.
- Identify partners early who can help to scale your tool or approach.
- Consider your funding model, including revenue-generation options, social business models, the cost per user and financial paths to sustaining the initiative.
- Gather evidence and demonstrate impact before attempting to scale.
- Don't attempt to scale without fully validating that your initiative is appropriate in a new context and addresses a priority need.
Analyze & Plan
- Collaboratively define what scale means. There are many definitions of scale, so be sure stakeholders are working together toward the same end goal. Are you interested in expanding to new geographies or new markets? Do you want to increase coverage within your current market and in the same geographical area? Do you want to expand to different kinds of users? Keep in mind that not all initiatives will start with a clear path to scale but it may still be valuable to pilot and test the approach.
- Develop a theory of change that includes scale as a goal. A theory-of-change process allows you to assess the preconditions and activities needed to reach your intended outcomes and goals. It can also be very helpful for identifying assumptions, such as pre-existing infrastructure. If scale is a goal, your theory of change should outline a path to it.
- Invest the time needed to plan for a tool that could scale. Based on funder pressure or other desires to roll out a pilot quickly, it could be tempting to put off designing with the user, collecting user feedback and making improvements, conducting ecosystem analysis, or developing partnerships with local actors such as policymakers and nongovernmental organizations until after the pilot has started or concluded. Investing the time now rather than later affects your budget and project timeline but saves time and money later, in addition to helping you build a better tool. Discuss these trade-offs with your funder, and try to secure support for more work before rolling out a pilot.
- Evaluate whether necessary enabling factors are in place to scale your tool and, if not, plan for how you can address them throughout the project lifecycle. The following sample factors are detailed in The Journey to Scale, a white paper published by PATH:
- A strong case for action, such as an urgent need.
- The right leaders: people with the necessary vision and the resources.
- The right solution: an effective product and a viable economic model.
- The right approach: supportive policy, regulation and standards and effective program management.
- The right capacity: human capacity.
- Seek guidance from others who are already working at scale. Consult communities of practice, companies, governments and organizations working in your region or sector, and learn from their experiences.
- Build on existing technologies, systems or platforms, unless there is a very clear reason why something new is needed. For example, mHero was launched in Liberia as a pilot, in November 2014 during the Ebola crisis. It connected national systems already in place and used common standards of the Open Health Information Exchange (OpenHIE) architecture to support rapid communication. Following this success, Liberia and Guinea institutionalized mHero by including it in their countries national strategic plans for health information systems. Note that some communities have greater willingness to share and reuse than others, so some contexts may require more time to research what can be reused.
Tips & Resources
- Tip: Build plans for scale into proposals to signal that long-term impact is the ultimate goal of the initiative. To maximize impact and use resources efficiently, funders and countries are looking for initiatives with a path to long-term sustainability and scaled initiatives, even though grant cycles and funding mechanisms typically do not align with this objective.
- Tip: Include your vision for long-term impact and scale even if it was not part of the request for proposals. This can help start the conversation with the funder about where resources will come from for initiatives beyond the grant cycle.
- Resource: An Introduction to Theory of Change, Harvard Family Research Project.
- Resource: The Journey to Scale: Moving Together Past Digital Health Pilots, PATH.
- Resource: The Maturity Model, Dimagi.
- Resource: Publications (e.g., Theory of Change: The Basics and Facilitators Source Book on the Theory of Change Process), Center for Theory of Change
Design & Develop
- As you make technology choices, think about how they will affect your ability to scale later. Using your initiatives definition of scale, seek out technology choices that will make it easier to scale. For example, if you are planning to scale to other audiences who speak different languages, select or design a tool that supports multiple languages. If you are planning to scale to regions with low bandwidth, consider a tool that has the option to work offline. Cultural user preferences, which will be included if you are designing with the user, are rarely compatible when designing for scale. Using tools or systems that are easy to customize with many configuration options can make it much easier to adapt to new regions or contexts. Plan that you will need to change your content. For example, you can design your information structures, images and icons so they can be changed easily if you scale to a different context.
- Keep your design simple, flexible and modular to make it easier to adapt to other contexts. What you design can be tailored to different contexts by using modules that can work either independently of one another or together and that can be adapted later. For example, the open electronic medical records software OpenMRS is made up of a series of modules that can be used individually. This modular design allows for countries to adopt only those components of the software relevant to their health care systems.
- Determine the steps and processes that will lead to the way you plan to scale. For example, if you defined scale in the planning phase as expanding to a new region, your steps during implementation may include identifying partners in that region or conducting a context analysis to learn how that region differs from your pilot region. If scaling means moving a tool beyond early adopters, such as urban populations or men, consider how you will market the tool or make it accessible to harder-to-reach populations.
- Develop plans for how your initiative will address the capacity building needed to achieve scaling and institutionalization of a tool into local systems. If your scaling plan includes local government uptake and support, you need to consider the training required so that these local stakeholders can support rollout and implementation. Be sure to create capacity-building and training materials and approaches that can be adapted easily to contexts and regions beyond your pilot region.
- Engage people who can provide input on what is needed to take the initiative to scale. How you are defining scale will determine whom you meet with, which could include government leaders and other organizations or companies that have scaled or attempted to scale similar tools. Begin meeting with these groups as soon as possible to make sure you are working in the right direction and not making decisions that will limit scaling later.
- Identify potential partners that can help scale. Consider civil society, multilateral, or public or private sector organizations, as well as donor organizations that are similarly interested in seeing your tool scale in other countries or contexts. Identify resources, capacities or information gaps that partners can help you fill, including local knowledge about markets and stakeholders. Also be sure to demonstrate how the relationship is beneficial for the partner. The MAPS Toolkit includes a section on partnerships with a self-assessment and other resources.
- Design your approach for measuring impact. To justify scaling, you will need to demonstrate that your tool had a positive impact. Plan and budget for which baseline, midterm and endline data you will collect and which evaluation methods you will use.
Tips & Resources
- Tip: Being able to link the initiative to the United Nations Sustainable Development Goals and targets can help to show how the initiative aligns with national and global development agendas and may appeal to potential partners.
- Resource: Scaling Mobile for Development, GSMA Intelligence.
- Resource: Designing Systems at Scale, Rotman Magazine.
- Resource: When and How to Develop an Impact-Oriented Monitoring and Evaluation System, Methods Lab.
Deploy & Implement
- Conduct a self-assessment of your initiative or tool before deciding to scale. The MAPS Toolkit includes detailed self-assessment questionnaires and a scoring mechanism that can be adapted for use beyond the health sector.
- Meet with users in the ecosystem where you want to replicate the initiative. Discuss their priorities and whether they think the initiative makes sense for their context. Don't attempt to scale without understanding the new ecosystem and working with new users.
- Review your tool to identify which components can be scaled and which will need to be adapted. Some aspects of your deployment may only be feasible for a small-scale initiative. For example, if your initiative included in-person outreach to encourage uptake or in-person training, you may need to consider how those activities could be done at greater scale and more efficiently by using tools such as radio or e-learning software. You also may need to plan for how a user community could replace roles currently done by staff members, such as making software updates or providing support by responding to user questions.
- Identify risks for scaling and develop mitigation plans to address those risks. As you scale, new risks could emerge. Risks could include new users adopting the tool faster than anticipated, overwhelming the tools capacity, or a rapid change in the technology environment that suddenly makes your tool obsolete. Complete a risk assessment matrix to identify internal and external risks. For each risk, make a judgment about the probability that it will happen and the severity of the impact (e.g., high, medium or low), and brainstorm actions to minimize the risk.
- Identify the resources needed to maintain the tool at scale. First consider whether you can adapt your tool to work without those resources, including people, money, technology and institutional capacity. If you determine those resources are still needed, outline a plan to build them over the life of the initiative. Business processes, budgets and workflows will change, hardware and communications infrastructure may need to expand, and you may need to identify, hire and possibly train new staff.
Tips & Resources
- Resource: The MAPS Toolkit: mHealth Assessment and Planning for Scale, World Health Organization.
- Resource: Sustainable Financing for Mobile Health (mHealth): Options and Opportunities for mHealth Financial Models in Low and Middle-Income Countries, mHealth Alliance and Vital Wave Consulting.
- Resource: The Pathway to Supply Chain Sustainability: A Planning Tool for Scaling & Institutionalizing Innovations Within Public Sector Supply Chains, John Snow Inc./SC4CCM.
- Resource: Creating Scalable, Engaging Mobile Solutions for Agriculture: A Study of Six Content Services in the mNutrition Initiative Portfolio, GSMA.
- Identify indicators across the steps described in the theory of change to determine if your program design was sound. Ask if the theory of change held up in the given context and if the intended outcomes and impact were achieved. If the answer to these questions is no, consider what corrections are needed in the program design to be able to achieve your intended outcomes.
- Identify indicators for measuring short- and long-term outcomes. Data on these indicators and the ultimate effect you hope to achieve will help you decide if it makes sense to scale. Also consider if these indicators would still apply if the tool scales in another country or context.
- Design your monitoring and evaluation plan to work at scale. Just as you need to be mindful of how your tool or system can scale, consider whether and how your M&E approach and processes can scale.
- Gather qualitative and quantitative data on usage. Quantitative data can allow you to understand how many people used your tool and how frequently they used it. By also gathering qualitative data, you can better understand why users did not use the tool or why they used it more or less than originally envisioned. Use this information to make improvements to your tool or to decide if your tool should not be scaled. If usage is higher than expected, you may need to expand the capacity of your tool or identify more resources to support the additional users.
- Evaluate how the ecosystem may affect or has affected the ability to scale. Because of ecosystem factors such as infrastructure, government policies and regulations, and existing tools and systems, a tool that scales in one place may not scale in another. An example of this is mobile money. In Kenya, M-Pesa scaled rapidly and has had a significant positive impact on financial inclusion. In South Africa and Latin American countries, mobile money was not adopted in the same way, given contextual factors like local regulations and established consumer behaviors.
- Calculate the cost per user and determine the cost effectiveness. The cost per user can be a useful measure to determine if scaling a tool is feasible. Your final costs may show that your tool would be less expensive than other approaches currently being used. When calculating the cost per user, keep in mind that costs for scaled solutions and pilots will be different. If you are working with government partners to scale the solution , factor in their budget constraints that would limit the total cost per user. For example, education funding allocations may specify how much can be spent per student.
- Consider using an impact evaluation methodology. When investing the resources to scale a tool, you want to be certain that you understand if it really has had an impact. The evaluation design could be experimental or quasi-experimental, depending on budget, expertise, timing and priorities. A rigorous way to measure the effectiveness of a tool or approach is by setting up randomized treatment and control groups and measuring the differences between the two over time. If outcomes for the treatment group – the group using your tool – are better than for the control group not using the tool, these data can justify to funders or other stakeholders the value in scaling the tool. Setting up a randomized control trial requires planning from the beginning of the pilot and significant funding support because it typically requires more time and resources than other impact evaluation methods.
Tips & Resources
- Resource: Table for Scaling Electronic Payments Within Your Organization, USAID and NetHope.
- Resource: Livelihoods in the Philippines: Impact Evaluation of the Project “caling up Sustainable Livelihoods in Mindanao, Oxfam International.
- Resource: Introduction to Evaluations, J-PAL.
- Resource: Monitoring and Evaluating Digital Health Interventions: A Practical Guide to Conducting Research and Assessment, World Health Organization.