SERVIR-Amazonia, the latest of five SERVIR Hubs developed a plan for monitoring and evaluating its intended outputs and expected results. The plan will serve as a management tool for systematically reviewing Program progress, troubleshooting implementation constraints, and determining whether specific elements require adjustment or refocusing to respond to evolving conditions.
About the SERVIR initiative
A joint development initiative of National Aeronautics and Space Administration (NASA) and United States Agency for International Development (USAID), SERVIR works in partnership with leading regional organizations world-wide to help developing countries use information provided by Earth observing satellites and geospatial technologies for managing climate risks and land use. SERVIR is improving awareness, increasing access to information, and supporting analysis to help people in West Africa, Eastern and Southern Africa, Hindu Kush-Himalaya, the Lower Mekong, South America and Mesoamerica manage challenges in the areas of food security, water resources, land use change, and natural disasters.
From a Theory of Change to a MEL plan
The plan was developed during a 4-day facilitated Theory of Change exercise at the program outset, to deepen our understanding of the strategic approach, planned activities and the results chains. From the 19 indicators that were identified in this exercise, USAID recommended using eight- three of which are USAID standard performance indicators. Illustration 2 below shows the relationship between strategic approach, intermediate results and corresponding indicators.
With help from the SERVIR Support team, SERVIR-Amazonia will be populating the MEL data management and reporting platform (DevResults system) to facilitate the tracking of indicators and reporting.
The SERVIR approach emphasizes user-focused services. It is important therefore to notice that each service that the program will develop will have its own theory of change, which should be related to the global SERVIR-Amazonia results framework. More information about SERVIR’s Service Planning approach can be found at: https://www.servirglobal.net/about-Servir.
SERVIR-Amazonia MEL plan includes a learning agenda, and many questions that we have identified during the development of the Theory of Change will be addressed in annual “Pause and Reflect” sessions. Here are some examples:
- Program Management: What are the perceptions of Hub and Consortium partners on Consortium governance and effectiveness (learning events)?
- Service development: Does intention to be more transparent and share data lead to data-sharing by the participating organizations? What are the barriers? How can we best collect and incorporate user feedback into subsequent service design?
- Stakeholders: How effective are we in exposing decision-makers to services? Are beneficiaries – and the intermediaries who can reach them – well understood and adequately represented?
- Sustainability: What business model or strategies should SERVIR adopt to nurture sustainability of the services?
The challenge of MEL: Capturing Impacts
One of our extensive, interesting and ongoing discussions revolves around capturing and measuring programmatic impacts. During our conversations with Zoltan Milic, our MEL Specialist from the SERVIR Support team, three potential types of impacts were identified:
- Services being used in practice. Decision makers at local, regional or national level use our services, yet we do not look into the type of decision they make based on the use of the service. SERVIR-Amazonia envisions a service design based on co-development and delivery to increase chances for services to be used and focuses on open data sharing among partners and their networks. We also plan to develop case studies that explore the notion of “use” more in detail.
- Services are used as vehicle to promote effective decisions. Decision makers at local, regional or national level propose, adopt, or implement laws, policies, regulations or standards based on the information of services co-developed by the program. This is one of the MEL plan indicators. The challenges for achieving this result are well known: antagonistic political environment, changes in decision-makers, or “unfriendly” service design for non-technical users.
- Benefits for the people. End users such as farmers, indigenous people, and others who live and work in the areas that are to be protected, experience benefits from the application of these laws, policies, regulations or standards. To measure this type of impact, SERVIR would need to engage in an economic analysis or mixed-method research that would likely expand beyond the program duration.
As NASA and USAID are set to consider the future of SERVIR, development of the SERVIR Global MEL Strategy would help all Hubs to benefit from a common MEL framework, MEL tools and indicators, in addition to deeper discussion and clarification of the MEL approaches to track results and impacts of geospatial services.
Authors of this blog post
Simone Staiger and Zoltan Milic
Simone is a senior knowledge manager and the coordinator of communications and Monitoring, Evaluation and Learning (MEL) for SERVIR-Amazonia.
Zoltan is a Monitoring & Impact Evaluation Specialist, and Grants Manager for the SERVIR Support Activity. He has worked on monitoring and evaluation for USAID projects for over 12 years.