Response to post about game mechanics on KA

The game aspects of the site are relatively new. Almost all of them have appeared since @kamens and I joined Khan Academy in November of last year.

As you note there's a reward system in place that's using a variable ratio schedule to encourage different kinds of activities: going for breadth by completing as many modules as possible (and getting n-number of module badges), going for automaticity by completely lots of problems in an individual module quickly (speed badges), going for depth by completing all of the modules in a particular topic (Expert/Artisan). It seems like this i, at least so far, interesting choice for lots of our users.

As @IanSchreiber says, the badge itself is not a mechanic, but the system that delivers it is. We've worked hard to balance giving users choices beyond simply filling in all of the squares on the map and ensuring that they are actually learning the skills they need to know. We've been measuring our success based on teacher feedback and school assessments in our intensive 5-month-and-counting daily use trial. More info:

http://www.thegatesnotes.com/TED/Speakers-Topics/Sal-Khan and

http://lasdandkhanacademy.edublogs.org/

Another key piece of game play, the streak required to achieve proficiency, has a really interesting impact behavioral momentum it's great unless it breaks at 8 or 9 (10 being required for proficiency). We've got some additional incentives built-in for people who break their streaks and then answer a few questions correctly, or answer a lot of problems in a given module without achieving a streak of 10 in order to help smooth out the loss of motivation that can occur. We're still analyzing data to figure out how big of a problem this is, but we're not totally happy with these mechanics and this particular aspect of game play at the moment.

So far, the news from the pilot is good. Students are still motivated. Teachers are reporting improvements in skills, math vocabular, and confidence.

What I was referring to in my tweet was the moral hazard that is present here because we want the outcome of the game to be directly tied to real deep understanding of math (and eventually science etc). The students in the pilot, and by extension many of our users, are extraordinarily sophisticated customers. They are quite familiar with games and game mechanics. The mere existence of any extrinsic reward encourages many to look for ways to game the game. It certainly keeps us on our toes, and it forces us to be driven by continuously collected data and feedback.

Needless to say, there are other things that we've tried, or teachers have tried, that have failed and been removed, but I've gone on long enough on your blog.

p.s. Periodically, modules in which you've earned proficiency change to "review" modules and you have to go back and answer a few more questions to maintain proficiency. We're still working on getting the timing of this where we want it.