Lessons Learned Series: Promoting relevance for technology-enabled interventions

The CEI Team
 
The Lessons Learned series highlights practical takeaways from CEI’s Journeys to Scale report, produced in partnership with UNICEF, that tells the story of innovative education interventions as they attempt to scale. To see past volumes of Lessons Learned, click here. To find additional details about any of these innovators’ journeys to scale and more, be sure to check out the Full Report.

 

Subpar education outcomes, exacerbated by poor teacher and student attendance, are prevalent in Peru, especially in remote communities. In order to respond to this challenge, UNICEF Peru and local NGO Kunamia introduced the EduTrac model, an SMS-based tool for improving data collection and monitoring of schools in hard-to-reach communities.

During the pilot phase, both student and teacher attendance in EduTrac Peru schools showed steady improvement, increasing from 76 to 90 percent among teachers and from 73 to 84 percent among students. Preliminary findings also pointed to the innovation’s potential to effect change among  a diverse set of stakeholders.

Using context clues to avoid past ICT interventions’ pitfalls

The examples of many failed ICT interventions have shown that, in order for a technology to reach its intended aims, it must be designed for the context in which it is intended to operate. A carefully considered analysis of the extent to which an enabling environment exists, followed by actions to create such an environment, are critical steps for enhancing a technology intervention’s traction, and ultimately its effectiveness. In Peru’s decentralized education system, this pre-implementation diagnosis revealed that community accountability structures were not always in place to monitor the quality and frequency of data reporting and ultimately the status of local decision-making. This assessment recognized that, in highly decentralized contexts, information must be disseminated among the intended targets of the policy reform to promote “a critical citizenry” and “effective participation by poor people and by a well-organized civil society.” In short, it was determined that introducing technology absent community mobilization in Peru would almost certainly lead to data collection without genuine accountability. In order to foster such participation, UNICEF Peru and Kunamia took two steps. First, recognizing that target communities often lacked internet, they produced monthly paper reports that summarized collected data, which were then used to inform discussions held during local community meetings. Second, they helped form local Community Education Committees who, by monitoring data reporting and agreed-upon decisions, filled the perceived accountability gap and facilitated the successful use of the new technology.

Intensive local collaboration

In order for an innovation to continually remain relevant in the eyes of its beneficiaries, it needs to incorporate the voice of communities. Recognizing that adjustments to the EduTrac model would need to be made repeatedly throughout the pilot phase, UNICEF Peru recruited local community leaders to serve as “quality promoters.” Responsible for overseeing various sites within their region, the promoters proved to be essential actors in building strong relationships with project stakeholders and identifying areas for potential improvement. By interacting directly with the communities in which they lived, they collected insights that otherwise would have otherwise not been communicated. For example, the EduTrac program was initially viewed as a mechanism for punitive action for poor teacher attendance. To combat this misperception and highlight the benefits of the intervention, the promoters developed workshops with local stakeholders to identify characteristics of an ideal school experience and determine how EduTrac can help schools advance toward such an experience. Community feedback collected by the promoters also led to a redesign of data reports to be more user-friendly (through the incorporation of simple, colorful graphics) and selection of the day in which reporting occurs (according to when attendance tended to be lowest). While resource-intensive, efforts to continually engage with community stakeholders contributed to a model that was viewed as highly relevant to end users.

The importance of institutional commitment

Strong institutional commitment from education officials is needed to sustain innovations and validate local decision-making. This is particularly the case for bottom-up initiatives like EduTrac. Unfortunately, initial interest in the innovation from government officials was slow to be accompanied by utilization of EduTrac data in decision-making and resource allocation processes outside of local communities. As Fox notes, “Localized, information-led ‘demand-side’ interventions” on their own are not enough to generate real change without institutional commitment. Such commitment at higher levels of the system would acknowledge the significance of the community-led efforts and reinforce local participation.

Looking Ahead

Short-term funding from UNICEF and GPE (through the UNICEF-implemented Data Must Speak project) in 2015 provided a limited window of opportunity for UNICEF Peru and Kunamia to adapt, test, and demonstrate the effectiveness of the EduTrac model.  The protracted process of contextualizing the innovation and securing local buy-in, while key to generating local ownership over the program, proved both time and resource-intensive and left EduTrac Peru with only a few months of funding to conduct the initial testing period. While positive ongoing conversations with regional governments and interest from international private actors show the potential for future implementation, this experience presents a cautionary tale for those providing short-term funding to pilot innovations that need time to test and adapt their models, particularly in very diverse contexts. 

Photo Credits (top to bottom): Teaching Excellence and Achievement Program (TEA) ; EduTrac ; Shawn, USF Peru 

See more blogs