Ilm on Wheel's Learning Boost is a holistic program, implemented by TeleTaleem, that addresses multiple dimensions for exceptional improvement in teacher competencies and children’s learning outcomes in early to primary grade literacy, with special focus on reading skills and mathematics. The program is based on effective and sustainable use of technology to deliver learning in public schools. The program uses a satellite (VSAT) equipped, self-contained van, called ‘School Garee’, containing computer servers, Android tablets, a power-generator, UPS, and routers to bring technology capability directly to a school.
School Garee is used to deliver just-in-time teacher training which is synched with what is being taught in the classroom. A master teacher trainer delivers this training using a virtual classroom environment from a remote location. The van allows a master trainer’s expertise to be shared readily with a large number of remote schools (8-10 schools per cluster). The teachers are provided with Android tablets with pre-loaded materials and assessments so they have ready access to material being taught. Students and teachers are assessed on each visit of the van (once every two weeks, and a total of twelve sessions in an academic year) and the results are discussed between each teacher and the master teacher trainer who also acts as a mentor. In addition, Stallings-type class observation videos of each teacher are recorded and uploaded to a main system through the satellite link. Teachers’ classroom behavior is rated on a rubric and they are provided with structured feedback on how to improve their classroom teaching and management practices. Finally, the parents, community and educational administrators are sent SMS messages on the current performance of each child and classroom. This will help improve teacher accountability, transparency and promotes better governance.
Moreover, the program trains and involves Community Learning Workers (CLWs) who deliver reading camps and parenting sessions in communities. The reading camps follow a curriculum that compliments learning of early grade literacy and numeracy. The reading camps attract significant interest from students. These camps are also open to and attended by out-of-school children.
Click here to see full program profile
CEI approaches in action
900+ reading/math and parenting camps conducted across 30 communities
The project was first conceived and implemented in 2012, addressing improvement in Maths competency of Grade 4 and 5 students and their teachers in 10 public primary schools in Balakot. Based on the positive results achieved the intervention was scaled to 46 schools in 2013 in the same target area, covering early grade literacy and numeracy component in addition to components deployed previously. Community engagement component was added as well.
Presently, the program has been replicated to other districts and provinces with an expanded footprint of 300 public primary schools with a beneficiary count of 600 teachers and 50,000 students. Proposals are in process for extending the program to 13,000 public primary schools in Sindh province and another 1000 schools in KP province over a 3 year period.
Monitoring & Evaluation
The evaluation methodology is based on the Non-Equivalent Groups Design (NEGD), which is a widely used method of quasi-experimental evaluations and social research. This design takes into account before and after comparisons of project (treatment) and similar (but not identical) control populations. In addition, since no standardized tests were available to determine learning achievements, Item Response Theory (IRT) and classical analysis was used to develop standardized assessment tools for each grade. Impact assessment has been done at multiple levels. Students' progress in learning achievements in measured through standardized tests conducted at the start and end of the acedmic year for schools in both control and treatment groups. The results for the treatment group schools are analyzed and statistically compared with control schools to determine real progress and impact. In addition to this, regular formative and summative assessments have helped measure on-going progress at the child and class level, enabling the local teacher to take in-time remedial actions. TeleTaleem's advanced learning environment provides seamless recording and archiving of complete daily session activity in a central LMS tagged with school/class information and date. This can later be assessed by internal/external M&E teams to report on any aspect of the project. Regular satisfaction surveys are conducted to assess children and teachers' level of satisfaction with the intervention.
Student assessment is conducted by teachers every two weeks over a 6 month cycle, using tablets. The assessment is automatically graded; results are available to teachers, disaggregated by student, learning outcome, etc, for appropriate remedial actions. The results are uploaded to a central results repository for online access and/or dissemination via SMS to education managers, parents and relevant stakeholders for accountability and advocacy purposes.
Teacher performance is monitored during training sessions and in class using in-class videography and classical evaluation tools, on a monthly basis. Master Trainers review these inputs to maintain a Teacher scorecard to provide actionable feedback and focused and personalized mentoring.
Statistically significant learning gains were observed. The mean numeracy score increase in the control group was 7.7% while those for the treatment group was 21%, which is a three times increase.
Learning gains for treatment and control groups were normally distributed (Anderson-Darlington; p>0.15). A two sample single-sided t-test shows that learning gains for the treatment group (M=0.211, SD=0.22) were significantly higher than those for the control group (M = 0.077, SD=0.24), t(200)=-3.73, p= 0.000. The effect as measured by Cohen’s D was found to be 0.62 which can be interpreted as a medium effect. Cohen’s D also had a fairly tight 95% confidence interval of [0.56, 0.67] indicating that there is a 95% probability that the Cohen’s D is between 0.56 and 0.67.
Since age, (r(200)= 0.165, p<0.05) and pre-score (r(200) = -0.461, p<0.05)) were correlated with the overall learning gain, an Analysis of Covariance (ANCOVA) using age and pre-score as covariates was carried out. After controlling for the effects of age and pre-score, the treatment was found to be significant F(185,1) = 23.9, p<0.000. A Cohen’s f value of 0.29 for the ANCOVA also indicates a moderate effect after adjusting for the covariates where η² was 0.08.
Similarly, in case of grade V Maths, the mean score increase in the control group was 5% while those for the treatment group was 22%, which is more than 4 times increase. In case of literacy, the weakest students’ mean score increase for comprehension was 26% for control group and 50% for treatment group while increase in reading accuracy was 17% for control group and 34% for treatment group.
The program has been able to transform the perception of quality education and has clearly resulted in measurable improvement in student learning achievement and attendance in schools where the intervention has been carried out. However, there have also been additional softer impacts. The communities, teachers and the children look forward to Traveling Learning Van coming to their schools. A recent survey showed that over 19% more children now enjoy studying Math as compared to the pre-intervention suvey. 97% children enjoyed use of technology in the classroom, and 100% of children shared the Traveling Learning Van experience with their parents and 100% of the parents expressed a lot of happiness with the intervention. These high numbers indicate childrens' level of excitement about the experience and their parents' acknowledgement of this unique way of learning.