How Brazil is improving education

Last month saw the release of the latest annual Survey of ICT use in Brazilian Schools. Now in its fourth year, this initiative from Brazil’s Center of Studies on Information and Communication Technologies (or CETIC, to use its acronym in Portuguese) is emerging as a model for how many other countries are considering conducting — and funding —regular data collection activities related to the increasing availability and use of various educational technologies within their education systems. The survey results, as well as a number of accompanying essays, are presented in one volume [pdf] in both Portuguese and English.

(Hint: If you’re just looking for the data, start from the back of the report. And: Here’s an earlier EduTech blog post about the first such survey effort in Brazil.)

In addition to offering a current ‘snapshot’ of what’s happening in schools, now that four years of data have been collected related to a number of common themes, the survey finds that some trends are becoming apparent. One trend which will come as no surprise to those who know Brazil is that there are some significant variations in many data by region. (Whereas municipal Rio de Janiero is in many regards a leader in educational technology use in South America, for example, the practical reality of ICT use in schools in northern and northeastern Brazil is much different.)

Some high level findings from this most recent survey:

• Schools in urban areas have an average of 19 computers, serving an average of just over 650 students. Most of these are in administrative offices and dedicated computer labs. While classroom and mobile access are growing quickly, with 30% of teachers reporting that classrooms are now the main venue for computer use in their school, labs remain the main point of access to computing facilities overall. (For what it’s worth, almost half of Brazilian households report having a computer.)
• 95% percent of schools with computers are connected to the Internet (no word if any computer-less schools are connected!), although the speed of these connections leaves more than a little to be desired: Only 39% of schools meet the minimum target of 2 Mbps for schools in Brazil.
• Almost half of public school teachers with their own laptops brought them to school, and most professional development related to technology use for teachers is a result of their own efforts (and thus not the result of government training programs).

For the first time, teachers were surveyed on the reasons behind their use of digital teaching and learning resources, and it appears that most of this use is self-motivated (i.e. a result of personal choice by teachers, and not something mandated, or necessarily even encouraged, by official education authorities). As the report states, “The ICT Education survey presents a scenario of relative autonomy for teachers in terms of educational content, given that the proportion of teachers that combine isolated contents such as images and texts is higher, surpassing access to video lectures and readymade presentations. The data indicate the importance of teacher initiative in the use of digital content in their teaching practices, as well as a concern for the demands of and benefits to students and colleagues. The reduced mention of institutionalized incentive – whether from the school administration or government authorities – indicates an important field for the development of public policies in the area.”

As public policies in this area continue to evolve across Brazil, the actions of a number of private foundations in supporting innovative uses of educational technologies are helping to suggest possible ways forward. A notable group in this regard is the Sao Paulo-based Fundação Lemann. Denis Mizne, who heads the Lemann Foundation (to use its English language name, which is how I’ll refer to it here), stopped by the World Bank back in September and shared emerging lessons from initiatives supported by his foundation andpartners to translate and implement the Khan Academy for use in Brazilian schools.

Support for the Khan Academy is one of a number of projects from the Lemann Foundation that are exploring innovative answers to the question, “How can we make sure we are making the best use of the short time available for instruction within schools in Brazil?”

I have transcribed my notes from the Mizne talk below, together with some short explanatory background as might be relevant, in case they might be of interest to a wider audience than just those who attended the related presentation in person at the World Bank.

Those who are unfamiliar with the Khan Academy may wish to visit its web site or check out two related recent EduTech blog posts. My boss, Claudia Costin — who used to be the secretary of education in Rio and who spearheaded the use of the popular Educopédia platform in schools there — has also written about the use of Khan Academy in Brazil, noting among other facts that, as a result of the Lemann Foundation efforts, Khan Academy content is now accessible to 70,000 students, 2,500 teachers, and several hundred schools in Brazil. (An additional 850,000 students across the country have used the materials since they were opened up in February to students and teachers outside of the schools involved in the official roll out.) On a somewhat related note, Jim Yong Kim, the head of the World Bank, has discussed what he has learned from Sal Khan.

 
What’s been happening with the Khan Academy in Brazil

Mizne noted that he has observed a widespread misunderstanding in Brazil of what Khan Academy offers, and what the Lemann Foundation has been supporting. Lemann wanted to introduce a tool to help teachers in Brazil make more informed (and thus hopefully better) decisions about how to help their students learn, and not just make available lots of additional content in Portuguese for students and teachers to access and use. There is, after all, no shortage of quality educational content available in Portuguese already. (When most people think of the Khan Academy, they think first, and often only, of the well-known instructional YouTube videos.)

At a basic level, it is true that the Lemann Foundation has been supporting the translation of thousands of Khan Academy educational videos into Portuguese. While this is perhaps what is most visible activity, however, it is really only the tip of the iceberg of what Lemann has been doing, and enabling. More fundamentally, it has made the full Khan Academy platform – including not just the content (the videos, the problem sets, etc.), but also the enabling content management system and analytical tool – available in Portuguese for utilization in Brazilian schools.

It was the potential ‘intelligence’ that the Khan Academy platform might offer about what individual students are learning, and not learning, that was in many ways most attractive to the Lemann Foundation. Translating the Khan Academy videos, while certainly a valuable thing to do in and of itself, was in many ways a ‘price of admission’ for the Lemann Foundation here, enabling access to the underlying ‘intelligent’ platform that lies at the heart of what the Khan Academy helps to deliver to students and teachers. If you just copy the way something looks (i.e. translate something that already exists), Mizne noted, it is difficult to keep up with the changes that occur as individual content changes over time. If you adapt the entire platform, you can keep up with the inevitable changes that will occur (to individual content, to the sequencing of content, as a result of the development of new content) much more easily. This hypothesis has been born out in practice in Brazil, Mizne observed, as during the course of the translation process, and during the early stages of implementation, there were many more changes to source content by the folks at the Khan Academy than they had been expecting.

Translating a vast, rich educational resource like the Khan Academy for adoption and use across an education system where English is not the language of instruction was difficult, but Lemann Foundation funding has also meant that there is now a more robust process, and platform, to enable similar efforts in other countries and languages going forward. Groups translating and localizing the Khan Academy platform for use in places like Mexico  (Spanish), Turkey (Turkish) and West and Central Africa (French) are now benefitting from the related funding that the Lemann Foundation provided to the Khan Academy to help facilitate these processes in Brazil. So, while the original process of translation and localization of the Khan Academy in Brazil was difficult, it is now much easier — in Brazil and more generally.

As difficult and expensive as the process of translation and adaptation of the Khan Academy tool has been in Brazil, Lemann recognizes that simply making a new tool and set of resources available for use by students and teachers will, most likely, not in and of itself bring about any significant changes to teaching and learning practices in classrooms. Support for ongoing teacher professional development and support for teachers has thus been fundamental to the Lemann Foundation’s efforts. Training programs for teachers to help them understand how to use Khan Academy resources as part of their teaching (and, just as importantly, where other resources or approaches may be more useful) has been a big focus, with particular attention to how teachers may wish to change some of their pedagogical practices as a result of the data that the Khan Academy platform makes available. Perhaps more important than the initial training efforts has been the ongoing monitoring and support for teachers that Lemann has supported, which includes the animation of various related online communities of practice for teachers to enable related peer support and guidance. There is a very active realted community of practice for teachers on Facebook, for example. (For those who don’t know: Facebook is huge in Brazil, as is social media in general. My boss, who joined the World Bank in July to lead our efforts in the education sector, has over 68,000 (!) followers on Twitter, a legacy of her time as the secretary of education in Rio.)

What’s been learned

A lot of things are being learned as a result of the work of the Lemann Foundation with the Khan Academy in Brazil. Here are a few of them that Mizne highlighted:

*Many* different usage models are emerging related to the Khan Academy in schools in Brazils. (This is consistent with what is being observed in the United States as well.) This means that there are, as a practical matter, no simple ‘recipes’ for teachers to follow. The Foundation sees the Khan Academy as a tool for teachers to decide how best to use in their classrooms themselves; the Foundation’s role is then to provide training and support for teachers to help them make more informed decisions about how and where best to make use of the tool, and not to prescribe simple ‘solutions’ to problems that teachers themselves are best placed to diagnose and respond to.

The “flipped classroom”, the much discussed approach often associated with the Khan Academy where students view lectures at home and do their ‘homework’ in class, with support of their teacher and peers, appears for now to be more useful as a concept and slogan to help advocate for the consideration of new pedagogical processes and tools than something observed in actual practice in Brazil. In reality, and in practical terms, what is being observed in Brazilian classrooms is that student utilization of the Khan Academy means that students are able to move at their own pace through certain materials, while teachers are enabled to be more personal in their approach to how to help individual or groups of students with specific concepts and activities. IN other words: Classrooms aren’t being ‘flipped’, but students are now more free to move at their own speeds.

Lemann’s experience with the Khan Academy has exposed what it sees as a prevailing myth that has currency with many people, namely that “it is easy to identify who is doing well, and who isn’t”. According to this belief, a student who is ‘good at math’ is simply good atanything math-related, for example, whether ‘math’ entails (e.g.) understanding place value, making estimations, deciphering mathematical concepts and information presented graphically, or calculating the area or volume of shapes and objects. Based on data being generated by the Khan Academy platform in Brazil, however, it turns out that student and understanding performance varies across different mathematical concepts processes much more than many people commonly believe — at least in Brazil. Just because a student quickly learns how to do something like multiplying fractions, for example, doesn’t mean that she will quickly learn how to do represent the same process using pictures, let alone understand something quite different within the same mathematics curriculum (like solving a word problem involving making change after a purchase). Armed with data that helps unpack all of this, teachers can then (hopefully) be better equipped to provide more differentiated support for students where it might be possible and appropriate.

Mizne related an interesting anecdote about a student in one of the pilot classrooms where the Khan Academy tool was being introduced, whom I’ll call ‘Mateus’. Mateus was the first student in Brazil to achieve a perfect score on all 23 concepts covered and assessed in a certain curricular area. One of the people helping to coordinate the Khan Academy implementation who happened to be visiting the city where Mateus lived stopped by his school to compliment his teacher on Mateus’s performance. The teacher responded, proud but a bit confused: “Now … which student are you referring to?” The point here was not to criticize the individual teacher – large class sizes, student rotation, the tradition lecture-centric manner in which many classes are typically conducted, and the absence of daily formative assessment tools that would provide detailed insight into how individual students are performing can all contribute to making it difficult to know how a particular student is doing with her or her learning at a particular point in time. The point that Mizne was making was that, given all of these challenges, tools like those in the Khan Academy platform can potentially provide teachers with unique data personalized to an individual student to help them better track how well (or how poorly) particular students are doing long before a student sits for a formal test. In addition: Using the Khan Academy platform, teachers can actually drill down pretty deep, if they so choose. If a student doesn’t understand a concept – something that a dashboard on the Khan Academy platform can help to highlight — a teacher can actually go back and see (e.g.) how many times a student viewed a video that attempted to explain and demonstrate the concept, if the student watched all of the video, if she watched all of it, if she stopped or rewound it, etc. Such knowledge can make it easier for teacher to do things like group students within a classroom of based on who knows what, who is struggling with what concept, etc.

Based on what is being observed in Brazil as part of the Khan Academy implementation, Mizne noted that, in practice, calling teachers for help is a last resort for many students, for a variety of reasons. Many would first prefer to review materials again themselves and/or ask their peers for help. Access to Khan Academy data can help teachers focus their time and efforts where they are needed the most, while students progress to a larger extent at their own pace, as a result of their demonstration of mastery of individual concepts, and follow the common practice today – ‘temporal promotion’ – whereby student all learn Concept A on Monday, then all move on to Concept B on Wednesday, etc., whether or not COncept A has been fully learned.

Criticisms and pushback

Mizne commented that, to date, there has been less resistance to their efforts to help introduce the Khan Academy in schools than they had been expecting. This is not to say that the process has been easy, of course, nor that there has been no resistance or criticism at all. One common criticism relates to the fact that technology is an integral part of this effort. This line of criticisms is not unique to the use of the Khan Academy, of course; there is no denying that many educational technology efforts have been expensive while yielding little impact, so skepticism in this regard isn’t only informed by simple anti-technology sentiments. Another criticism is related to a discomfort with the use of a ‘non-indigenous’tool, i.e. something developing in the United States and then imported into Brazilian schools. Some critics have lampooned the effort by saying things to the effect of, “oh, you think you’re going to solve everything with a bunch of YouTube videos”. Mizne noted that, while the videos themselves can be quite useful, he sees them only being one piece of a larger set of things (the problem sets, personalized data dashboards to help monitor activity and progress, teacher professional development and ongoing support) that the Khan Academy effort in Brazil is trying to introduce and support. A more fundamental, and difficult, criticism relates to questions the pedagogical value of the way individual concepts are presented and discussed. (Is this actually good math?”) In this regard, Mizne notes that the Khan Academy content is regularly improving, and that the data which the platform generates help developers to better understand how to improve things – and that the developers are quite interested in and open to improving things.

Some challenges and recommendations going forward

Costs

Overall, Mizne observed that what the Lemann Foundation has supported is not cheap. That said,  if you divide your investment costs across your entire user or beneficiary base, doing something like this can become quite inexpensive as you scale – provided, of course, that you do actually reach scale. There are a number of barriers to ‘scaling up’ successfully, however, and some challenges that only become acute as you reach a large scale.

Connectivity

In order to take full advantage of the Khan Academy platform, schools, teachers and students need to have robust, reliable, affordable connectivity. (An online platform with video: As far as bandwidth requirements go, there is nothing worse than that! When the Internet is down on a day when using the Khan Academy is planned, life for a teacher can be rather difficult, to say the least.) In 2013, 400 schools applied to be part of the official cohort of schools across Brazil participating in the Khan Academy roll out. Lemann did some independent testing of the actual connectivity situation in these schools and found that *none* of them met the minimum related requirements. (This may mean that the connectivity data reported in some places may be a bit, well, misleading …. In some cases, for example, connectivity may be largely reserved for use by school administrative purposes.) Even where connectivity is available in sufficient amounts, firewalls can really muck things up. The recent trend in Brazil is for tablets to be distributed, not laptops or PCs, and the user experience on tablets is not (yet) as good as it is on those more full-featured devices, with their keyboards and mice.

Side note: The free, offline KAlite tool can be useful in environments where connectivity (and electricity) are spotty. The World Bank will be exploring the use of KAlite in education systems in at least two countries in 2015.

Mizne predicted that, inevitably, students and teachers in Brazil will get their own devices. They may not get them quickly, they may not get the same devices, they may not get them according to the same schedule, and there will be important equity issues that arise as a result. That said, devices will be there and available for use in schools. Access to reliable, affordable Internet connectivity, however, is not happening as quickly as access to devices is occurring, and prices are not going down as quickly either. To make full use of tools like the Khan Academy you need broadband, you need fiber – better bandwidth is an area of critical need within the education system in Brazil.

Project management (the first month is especially critical!)

Mizne observed that the first month of the roll out of the Khan Academy initiative within a school requires a *huge* effort. While teachers have been enthusiastic, there has been resistance to actually start using once teachers return to their classrooms from initial training sessions, and so a LOT of related support is required, even very basic support, both technical and pedagogical. If this is not provided at the start, even well-intentioned efforts that meet evident, tangible needs in practical ways can falter, for a whole host of reasons. The Lemann Foundation closely monitors general usage data day-by-data so that it can quickly identify where something might be wrong, investigating and providing related support immediately. It does not want to wait until problems are reported weeks or months later, by which time it might be quite difficult to get things re-started.

As adoption occurs within classrooms and more and more students actively use the platform, a lot of effort and support is required to get teachers to access and utilize the individual data available about how each student is using the tool. The availability of such data is a new thing for teachers in Brazil, and it takes a while for them to internalize that it is available, and to figure out how and when to best utilize it to support their teaching.

Support for teachers

A common theme throughout Mizne’s talk was the importance of providing relevant, practical, just-in-time support for teachers. Adoption and success to date has largely been a function of teacher engagement, something which can perhaps be catalyzed as a result of initial training courses, but which is sustained and which grows over time as a result of access to related communities of practice and informal knowledge sharing about what works, and what doesn’t, between teachers.

Data privacy

Mizne stated that issues related to the confidentiality and privacy of student data will be a huge issue going forward in Brazil. The roll out of the Khan Academy platform in Brazil has helped to highlight what some of the nuanced complexities of some of the related issues are in practice, beyond some of the high level, sometimes theoretical issues and concerns that are commonly considered and debated. The absence of related frameworks for data protection, widely-disseminated ‘good practices’  and accompanying auditing and compliance mechanisms represent a real threat to the sustainability of online education efforts such as this going forward.

Evaluation frameworks

Mizne noted that there is a shortage of relevant, practical frameworks to understand and make use of individualized student data and efforts to evaluate efforts meant to promote ‘personalized learning’ of the sort that the Khan Academy is meant (in part) to help enable.  In the absence of such frameworks, and related evaluation tools and approaches, some people may just simply say ‘the technology doesn’t work’ and advocate that everything return to the ‘ways things were done before’. If that decision is to be made, it should be made because it was demonstrated that things did not work, not because we are unable to tell if things worked or not.

Going forward, efforts like those of the Lemann Foundation in Brazil to support the roll out of the Khan Academy in schools in Brazil should yield many useful insights about what it takes to succeed, and how efforts might fail, related to the increasing use of new technologies to support teaching and learning around the world.  An agreement is in place with the national ministry of education to support for the expansion of the program, and, as the fit within the formal education system is explored in greater detail, one expects that there will be regular debate about, and examination of, whether efforts should be directed primarily to impact students within the formal education system, or whether it might make more sense to target students outside of school instead. While insights and lessons from project supporters and implementers will continue to be valuable, so too will be independent assessments of what’s been happening, and what impact this might be having. Stay tuned.

This article was first published by The World Bank’s EduTech blog. Publication does not imply endorsement of views by the World Economic Forum.

To keep up with Forum:Agenda subscribe to our weekly newsletter.

Author: Michael Trucano is the World Bank’s Senior ICT and Education Policy Specialist.

Image: Professor Christian Agunwamba writes on the board while teaching his “Fundamentals of Algebra” class. REUTERS.