Philanthropists and technologists are now testing whether a mix of live teaching and self-paced computerized instruction can improve educational outcomes. The early results are exciting.
Following is an edited transcript of a talk by Sal Khan, one of the pioneers of computer-assisted teaching, delivered at the 2011 Annual Meeting of The Philanthropy Roundtable on October 28. After his prepared remarks, Hewlett-Packard CEO Meg Whitman had a conversation with Khan, which is also transcribed below.
Remarks by Sal Khan:
Khan Academy started as a collection of online instructional videos for high school and grade school students—covering math, science, and other topics. It’s now up to 2,800 videos. I did a couple new ones this morning. The videos, however, are just the tip of the iceberg. And we have a lot more to do as we launch a new form of teaching, learning, and assessment that relies on a mix of live instruction and computerized lessons and testing.
A model is emerging—one that is very different from much of the conventional wisdom on schooling.
A year ago we were a one-person operation—me. We’re now 20 people, so we’ve been ramping up quickly. We’ve passed over 80 million lessons delivered. Our usage has been skyrocketing, organically and voluntarily, at a rate of three- to five-fold per year. We’ve had 3.5 million unique users this past month. That’s more than six times the number of students who have attended Harvard in its entire history, just over the last month.
It all started with a bit of happenstance. It was 2004, and a newly married cousin was visiting me in Boston. She was a really bright student, fun to engage, but she was having difficulty with math. I told her, Nadia, you’re obviously smart, you’re obviously hardworking, I have to think you’re just stuck on some mistaken concepts. How about when you go back to New Orleans, we do some type of long-distance tutoring? She agreed, and in the fall of 2004 we would get on the speakerphone every day. I would call her up after work, or she would call me after school. We would use Yahoo Doodle as kind of a shared whiteboard.
It worked out. A couple of months later she was able to ramp up in mathematics and placed into the right class. Then I started tutoring her brothers, a couple of other cousins here and there. And I started writing some primitive software to aid my teaching. By 2006, I had about 15 cousins and family friends around the country that I was doing this with. And I still had my day job.
By November of 2006 the firm I was working for had moved to Palo Alto, and I was having dinner there with a buddy. I showed him this little piece of software I was working on for my cousins, and told him what I was doing. I said my one frustration was that I was finding it hard to expand. He said, well, why don’t you make some videos and put them up on YouTube? My answer was that YouTube is for cats playing piano; it’s not for serious mathematics.
By spending extra time on stumbling-block concepts, we see students moving from below average to above average performance.
But after I went home that weekend I got over the idea that this wasn’t my idea, and I decided to give it a shot. The first videos I made were very rough; they are still up there. (You can sort on YouTube by date uploaded.) I left them for the historical record.
I told my cousins: take a look at these and see what you think. Their initial feedback was slightly backhanded but profound. They said they preferred me on YouTube to in person. It is counterintuitive that someone would prefer an automated version of their cousin to their real cousin, but it actually makes a lot of sense when you put yourself in their shoes. Now they could pause and repeat. They could watch something as many times as they needed without feeling judged. When I had sessions with Nadia sometimes I’d say “do you get this concept?” and I’d get the feeling that maybe she didn’t, but she was embarrassed to say so. Now, she can just go back and watch the video repeatedly until the concept sinks in.
So I just kept making videos and posting them on YouTube. And other people started watching them. At first there were just very simple comments. Then I started getting responses from students saying “this helped me passed my algebra exam,” or “this gives me the confidence to become a physics or engineering major in college.” So I was excited. Here I was, I was taking a shower or shaving or hanging out with friends, and my videos were able to teach people around the world and affect their lives in at least small ways.
Then a few months into it, I started to get letters that were above and beyond just small ways. This letter, which I received when I had posted about 100 videos, mainly in algebra and pre-algebra, particularly helped change my thinking about what this work could lead to: “Hello Sal. My 12-year-old son Ethan has autism and has had a terrible time with math. We’ve tried everything, viewed everything, BOUGHT everything.” The word bought was in capital letters. “Well, we stumbled across your video on decimals and it got through. Then we went on to dreaded fractions. Again, he got it. We couldn’t believe it. He was so excited. It was your soothing voice and calm manner coupled with an easygoing explanation and examples. I can’t thank you enough.” I’m shortening, but then it says, “You are their magic math teacher. I’ve shown this to everyone I know who are in similar situations to my own. There are so many of us. Thank you so much and God bless you. Alison.”
I was an analyst at a hedge fund. I wasn’t used to people saying God bless you. It changed my value system a little bit.
And the site kept growing and growing and growing. By 2009, I frankly had trouble focusing on my day job. I’d set the venture up as a not-for-profit a few years before earlier, and when I’d have a bad day at work, I’d say “I’m going to quit. I’m going to do Khan Academy full-time.” Finally, my wife and I sat down. We decided we had enough savings that I could take a year off. And hopefully someone would realize by the end of that year that the project was a good social investment.
So I quit my job in 2009. After about nine months no one had noticed. I started updating my resumé. Some people had suggested I put a little PayPal link on the site so they could donate. I was getting $10, $20, $30 donations from around the world. That was powerful: poor college students donating $15 just out of goodwill.
Then all of a sudden, I got a $10,000 donation one day. I looked at who it was, it was someone named Ann Doerr, who I’d never met before. I sent her an email to say thank you, this is the largest donation I’ve ever gotten. If we were a physical school, you would now have a building named after you.
She was based in Palo Alto like I was, so we ended up meeting. Over lunch she said, “what’s your vision?” I said a school for the world, not just videos but also exercises and assessment software. If I’m your largest donation, how are you supporting yourself, she asked? I said I wasn’t.
Then I drove home. And shortly after, I got a text message from Ann saying “I just wired $100,000 to you. You’re supporting yourself now.” So that was a good day. This was just slightly over a year ago.
Technology enables interaction and frees up teachers to spend more time with individual students.
Then a whole series of events started happening. I was running a little one-week summer camp in the Bay Area for kids, just to experiment with some of my ideas. And I got another text message from Ann. She said “I’m at the Aspen Ideas Festival and Bill Gates is on stage, in front of an audience of hundreds of people and he’s talking about you right now.” I didn’t know Ann was such a prankster. I thought maybe she sent it to the wrong speed-dial button on her phone. So I went on line and saw that people were Tweeting about Khan Academy like crazy. It was surreal.
I thought, what do I do now? Do I phone him? About a week or two later I got a call from Larry Cohen, who is Bill Gates’ chief of staff. He said, “you may have heard that Bill is a fan. He would like to meet you if you have time.” I was staring at my calendar as we were talking. It was completely blank. Yeah, I said, I think we can meet on Wednesday at 2:45. I just snuck him in there; I did have a dentist appointment and some laundry to do.
So we met and he decided to support us.
And simultaneously—it’s amazing how these things happen—some folks at Google invited me over. They asked “what would you do with $2 million?” I said I would build a virtual school, hire a software team, translate the content, et cetera.
So one year ago, both the Gates Foundation and Google became major sponsors, allowing us to acquire office space and build the initial team. I keep making videos. And part of the Google funding allow us to translate them into more than ten languages. Our major new focus, though, is on software that complements the videos.
We’ve developed a knowledge map that’s built on the idea of mastering one concept before moving on to the next. It’s a very simple idea. It’s actually how you learn anything well. It’s how things go in a video game. Only when you master level one do you get to go to level 2. Yet this, unfortunately, is completely the opposite of what’s happening right now in our traditional school system.
Right now what we do in schools is fix the amount of time someone has to learn something. The highly inconsistent variable is how much of the concept each student actually absorbs during that allotted time. Classrooms of kids spend two weeks learning systems of equations. Then they have an exam. Some of them get As; some gets Bs; some do much worse. But at the end all of them move on. Even those who may have failed that exam must move on to the next concept. And the concept that follows builds on top of the previous one.
It’s as if you were building a house, and you got the inspector in. He looks at the foundation and says this foundation is only 80 percent sound. Well, that’s passing, so let’s build the first floor now. That’s exactly what’s happening in many classrooms.
So we said, we’re going to do the opposite. Instead of making the amount of time you work on something the fixed input, and let how well you master the concept become the variable, let’s set how well the student knows something as is the centerpiece. Everyone should master the topic. Everyone should essentially have 100 percent understanding. The variable will be how long it takes to get there. Lessons can slow down, or speed up, to match the student. That’s what we’re doing here.
We’re building modules extending all the way from basic addition through calculus, and then through physics, accounting, computer science, so on. The videos already cover a lot of these. But we want more than just the instructional videos. We want someone to be able to practice, and get feedback. If a student gets stuck, he or she needs to retrace steps and find out exactly where his or her knowledge breaks down. It’s a process of trying to ensure mastery. Seventy percent, 80 percent, even 95 percent isn’t good enough. What was that 5 percent that the student didn’t understand? We want to identify that. Maybe it was something important that’s going to show up later in their careers.
Some months ago a local school district in Los Altos contacted us to say they had heard we are doing interesting stuff, and that they’d like meet us. When we got together, they asked, “what would you do if you could set up a classroom however you wanted?” By this time, I’d hired the smartest guy I knew at college, and he and I said that we’d have all of the students work at their own pace, using videos and exercises, whatever they need, on their own time. Then we would give teachers lots of data about what these exercises showed each students strengths and weaknesses to be.
Instead of the teacher plowing through one-pace, one-size-fits-all lectures, we’d have them do direct interventions with small groups of students or individuals, or even better getting the students to mentor each other. We would use technologies to make the classroom more interactive. Make it more engaging, not less engaging.
The school administrators nodded. Afterward, my colleague and I went to the parking lot and agreed these were very nice people, but no way would they ever go for our vision, which was a somewhat radical break from traditional teaching. However, this is in Silicon Valley, where innovation is in the air. And two days later they asked us whether we could start fleshing out our vision in two weeks, right after Thanksgiving? I said to my friend, “we better hire more people.”
We started our pilot programs in Los Altos a couple of months ago. The teachers walk around the room, behind rows of students when they are involved in independent study. The teachers can see how each student is faring on the knowledge map he or she is working through.
Green means the student has already mastered that concept. Blue means they’re working on it, so far so good. Red means they are working on it, but they seem to be stuck. They’ve watched the video; they’ve gotten the extra hint; they’ve done a bunch of sample problems—but they’re still not able to get to mastery. That’s when the teacher kicks in extra hard, going straight to that student and doing a one-on-one intervention.
By spending extra time on stumbling-block concepts, we see students moving from below average to above average performance.
So a model is emerging. A model that is very different from much of the conventional wisdom on schooling. There is an obsession today with student-to-teacher ratios. That’s an important measure, but what’s even more important is the student-to-valuable-time-with-the-teacher ratio. In the traditional model, the teacher spend most of the day in group lecturing, grading homework, disciplining. Maybe 5 or 10 percent of her time is actually sitting next to students, forming connections with them. In our system, most of the teacher’s time is spent in direct, close, personal instruction. We think this is actually increasing the humanity of the classroom, not letting machines take over.
We give lots of data to the teachers, and students also get data on themselves. It’s been powerful to see how the students have taken control of their learning. They are seeing their weaknesses clearly. They are setting goals.
In our reports, one sees immediately what students have been working on week by week, and how well they are achieving their goals. You can check which instructional videos they’ve watched, which exercises they’ve done, where they have drilled down deeper. Blue shows a problem the student did right, red is wrong, the height of the bar shows how long it took the student. The teacher can see what hints they used, when they took each step. We’re improving the software to the point where the teacher can actually see how the student interacted with each problem, what choices they made as they solved it.
The dashboards we give teachers allow them to track every student in their class as each works at his or her own pace. This includes a kind of powerful group narrative. The horizontal axis shows days spent working. The vertical axis lists the number of modules completed. What we see in class after class has been eye opening for us: almost always there is a group of kids who race ahead. There is a group in the middle, and there is a group who fall behind. This is the source of tracking in traditional schools, where kids are separated into remedial and gifted groups.
When you let every student work at his or her own pace, however, those initial surges eventually become much less significant. Students who start out below average are generally eventually able to fill in their gaps. In the traditional model where everyone moves in lockstep, those gaps would remain, lock in, and eventually become crippling. But by spending extra time on stumbling-block concepts, we actually see some students moving from below average to above average performance.
Over and over again this can be seen in our data. We ask, who is this student? Where did he come from? How did he become the second or third best student in the class after starting out as one of the worst? It’s clear to us that the traditional group instruction and mass snapshot exam is not the best way to filter and judge and teach our kids.
Whenever you talk about computer-based learning, or “blended learning” as the current buzzword goes, people imagine, oh, he’s going to turn school into a factory; the kids will just be doing rote problems all the time with little human interaction. Actually, it’s the opposite. The technology is enabling interaction. It frees up the teacher to spend time with individuals, it allows more peer-to-peer interaction, it accomplishes the basics of instruction and assessment efficiently so there are opportunities for projects, engineering, whatever else the teacher might want to offer.
When we started off, honestly the kids didn’t want to go recess. It became their favorite class, even kids who before didn’t like math. Obviously that’s not a rigorous test. We want to see data on hard results too.
Some of the achievement-test results from our Los Altos classes are encouraging. The seventh-grade students we worked with were essentially remedial math students. A lot had learning disabilities or came from families that don’t speak English at home. Going into the year, 23 percent were at grade level. Six months later, that was up to 41 percent. Fully 6 percent of this remedial math class was now testing at an advanced level. They had actually leapfrogged who were not placed in the remedial math class. Normally when you go to the remedial math class, it’s the end of your math career. You just fall further and further behind. But these kids actually leapfrogged.
The other thing is that the whole curve shifted in a good direction. We didn’t have any students below basic anymore. We saw a broad-based shift. So it wasn’t just helping a small number. It was helping students at all levels.
Meg Whitman, Sal Khan
Conversation between Meg Whitman, CEO of Hewlett-Packard, and Sal Khan, founder of Khan Academy:
Meg Whitman: I thought I might start with a little story. I’m on the board of Summit Prep, a chartered public high school with four campuses in Silicon Valley. Two of the schools teach in the traditional manner, and they’ve had terrific results. Two that opened this fall use a blended-learning method—combining instruction from the regular teacher with Khan Academy computerized instruction.
In the quarter of the year so far, the traditional schools have zero percent of their kids scoring at 80 percent of grade level or above. In the schools using the Khan Academy blended-learning model, it’s already 30 percent.
Sal Khan: Oh, I didn’t even know that. That’s good to hear.
Meg Whitman: It’s quite a remarkable story.
Khan: Yeah, that’s exciting.
Whitman: Let me ask you a couple of personal questions. Are you still the only one who creates the YouTube videos?
Khan: I was until Wednesday. Things are happening fast. We found two art historians who are doing some tremendous art history videos. I called them up, asked if they wanted to be our art history instructors, and they signed on. One was the head of art history at the Pratt Institute, and one was head of education at MoMA in New York. So now we have faculty other than me, and there are art history videos at the Khan Academy.
Next, there is another woman who is coming on to add math instruction.
Whitman: Is it true that you did these videos in your closet?
Khan: The first 1,500 or so were done in my closet. That was the only feasible place. My son was born two-and-a-half years ago, so when I quit my job and set up my office at home, he was there. The closet was the point in the house that was farthest from his crying. That was the only way I could record something.
Whitman: It’s amazing to me that you could do videos on all these different subjects. How do you know about physics, and chemistry, and algebra, and so many other topics.
Khan: My wife will tell you I have the ability to pretend I have authority on things that maybe I should not.
Once you have everyone learning at his or her own pace, it breaks all barriers. Why separate by age group? Why separate by subject?
Whitman: Sort of like politicians, you just assert.
Khan: I will say, it startles me when people express surprise that I remember my school math. What’s so shocking about that? We take 12 years of a subject, and then go to college. It’s a sad state when we’re surprised to find someone who actually learned and retained something from that process.
Math and science were my strong suits, so that’s where I started. There is also a lot of finance, which was the industry I was working in when I began the videos.
One of the inflection points of Khan Academy was during the financial crisis. I started making videos on credit default swaps and other arcane details. Reporters started watching the videos before they wrote their stories. I actually had one banker—I won’t name the bank—who emailed me to say “I now know what I’m doing at my work. Better late than never.”
Whitman: I know it’s all so recent, and the venture is young, but what do you see happening over the next couple of years.
Khan: I mentioned the K-12 pilot exercises we’re doing in a few California schools. Summit Prep, which you mentioned, is one of them. We’re a small organization and not able to deploy to every school in the country, so what we want to show now is that our method works. Not just in Los Altos; not just for 5th graders or 7th graders; but also at places like Summit.
We have another pilot at the Marlborough Academy, a high-end girl’s school in Los Angeles. We told them, with this demographic we want to really push the limits. Let every student work at her own pace and see how high she can rise. It’s amazing. Seventh- through twelfth-grade girls are in the same classroom teaching and learning from each other.
Once you have everyone learning at his or her own pace it breaks all barriers. Instead of one teacher and 20 students in each room, siloed, why not two teachers and 40 students, or three teachers and 60 students? Why separate by age group? Why separate by subject? The pilots are really to see what we can do with this, how it might work in different form factors.
At present, though, most of the three-and-a-half million students using us right now are home schoolers, or people using us as a complement to their school classes, or whatever. We’re just trying to make that experience as deep as possible, adding more content beyond calculus, physics, biology, chemistry. We are going to have exercises history, art history, computer science.
One long-term goal is to decouple learning from the expensive and often phony process of credentialing. Right now you can go to a community college and learn a subject as well as a Harvard grad, but no one is going to know what you learned. If we had more effective universal credentials, and a more open playing field for learning, serious learners will be recognized. And it won’t necessarily take tens of thousands of dollars a year and heavy loads of personal debt.
If Khan Academy can compete with community colleges, high schools, universities, with learning on the job, with the apprentice model, that will just blow open everything, in a very constructive way.
Everyone should master the topic. Everyone should essentially have 100 percent understanding.
Whitman: Have you thought about how you could teach writing?
Khan: Anything that’s currently being done in a lecture mode, we think can be done better with on-demand video. Writing is actually not that different from computer science. Both are hands-on; you need experience. That’s not our perfect form factor. I think our current format could do grammar and vocabulary, but that’s not writing.
I can envision that in the next three to five years we could start giving students writing projects that they would do on our site. They’d build a portfolio on our site, and then the powerful part would be peer-to-peer critical analysis. All of a sudden you’re not just catering to one English teacher who may or may not agree with your style, you get a mass audience. When other people are going to read your work, it changes how you approach it.
Whitman: I think you could also potentially get teachers from around the country who just out of the goodness of their heart think that this is interesting to help kids. Almost a Wikipedia of writing. Maybe people who are in the next stage of their career, are 55 and interested in doing something new.
Khan: Actually, there is an experiment in, I think, England called Cloud Grandmother where all these grandmothers signed up to befriend and mentor young people. Fabulous.
Whitman: Well, I think you’re fabulous.
To listen to this talk or other sessions from the 2011 Annual Meeting, visit our resources page.