Neal Stephenson’s 2014 Magic Leap Blog Post

16 dec 2014
an unexpected adventure
by neal stephenson

A few months ago, two Irishmen, a Scot, and an American appeared on my doorstep with Orcrist, aka "Goblin-cleaver," the ancient sword forged during the First Age of Middle Earth by the High Elves of Gondolin, later retrieved from a troll hoard by Thorin Oakenshield. It’s not every day that someone turns up at your house bearing a mythic sword, and so I did what anyone who has read a lot of fantasy novels would: I let them in and gave them beer. True to form, they invited me on a quest and asked me to sign a contract (well, an NDA actually).
The sword-bearers were from a company that had been jumping up and down in my peripheral vision for a year or two. A company with a funny name (Magic Leap), an out-of-the-way location (Fort Lauderdale), and a tendency to play its cards close to its vest. Sir Richard Taylor, founder and head of WETA Workshop, is a member of Magic Leap’s board of directors, which explains why they are able to show up with ice-breakers like Orcrist. Alas, I didn’t get to add it to my own personal troll hoard. But there is talk of replacing it with a high-tech katana like the one wielded by Hiro Protagonist: a fictional character in Snow Crash, a book I wrote twenty-five years ago.
If you’re one of the seven billion people who haven’t read Snow Crash, I’ll explain that it has a lot to do with both virtual reality (VR) and augmented reality (AR). When I wrote it, it seemed as though those technologies were just around the corner. Accordingly, Snow Crash is set in a near-future world.
In practice, it has taken longer than just about anyone expected to get that kind of tech consumer-ready. The devil has turned out to be in the details of satisfying the amazingly finicky human visual system.
The retina lies outside of what we normally think of as the brain, but it performs brain-like processing operations on the light that strikes it. What it feeds down the optic nerve to the brain proper isn’t so much an image as it is the beginnings of an idea--something that has already been thought about by that chunk of peripheral brain in the back of the eyeball. Banging photons off of this thing in an effort to fool it turns out to be hard. For hundreds of millions of years of animal and human evolution, seeing stuff that’s actually there = not dying = getting to have children.
So it was in an appropriately skeptical frame of mind that, a few weeks later, I traveled to what for me is the opposite corner of the United States. In a teeming, overcrowded hardware lab in a South Florida strip mall, I got the demo from Rony, the founder and CEO. Shortly thereafter, I agreed to become Magic Leap’s Chief Futurist.
Here’s where you’re probably expecting the sales pitch about how mind-blowingly awesome the demo was. But it’s a little more interesting than that. Yes, I saw something on that optical table I had never seen before--something that only Magic Leap, as far as I know, is capable of doing. And it was pretty cool. But what fascinated me wasn’t what Magic Leap had done but rather what it was about to start doing.
Magic Leap is mustering an arsenal of techniques--some tried and true, others unbelievably advanced--to produce a synthesized light field that falls upon the retina in the same way as light reflected from real objects in your environment. Depth perception, in this system, isn’t just a trick played on the brain by showing it two slightly different images.
Most of the work to be done is in applied physics, with a sizable dollop of biology--for there’s no way to make this happen without an intimate understanding of how the eye sees, and the brain assembles a three-dimensional model of reality. I’m fascinated by the science, but not qualified to work on it. Where I hope I can be of use is in thinking about what to do with this tech once it is available to the general public. "Chief Futurist" runs the risk of being a disembodied brain on a stick. I took the job on the understanding that I would have the opportunity to get a few things done.
============================================================
Magic Leap is not exclusively about games. It’s also going to be a great tool for readers, learners, scientists, and artists. Games, however, are a good place to start talking about why this tech is different.
I’m a huge fan of games that consist of two-dimensional figments moving around on flat screens. I have been since I was in high school. Of note is that I haven’t been in high school for four decades.
My friend Seamus Blackley has one of the finest collections of restored arcade games in the world. A lot of these--Space Invaders, Robotron, the 1983 Atari Star Wars arcade game--played a big part in my life, and so I am with him up to a point. But he has other things he could be doing with his talents than hunching over thirty-year-old circuit boards with a soldering iron. I have come to understand that Seamus, somewhat like an art historian checking out Rembrandt’s brushwork, appreciates all of the cleverness and artfulness that the makers of these games--the nerds behind the curtain, the hacker artists--had to show in order to achieve fun results on what now looks like primitive hardware.
I sometimes feel that the creative minds who make games have done about as much as is possible in two dimensions. It’s hard to imagine how the current crop of games, for example, could be more finely tuned to deliver that particular kind of entertainment. It feels like the right time to give those people a new medium: one in which three-dimensionality is a reality and not just an illusion laboriously cooked up by your brain, and in which it’s possible to get up off the couch and move--not only around your living room, but wherever on the face of the earth the story might take you. Making such games is not going to be a matter of porting existing ones to the new system. It’s going to mean redefining the medium from the ground up.
What applies to games applies as well to other things of interest, such as making the world safe for books, doing new things with science and math visualization, and simply creating art for art’s sake. Regarding all of those things, I look forward to making future posts on this blog.