The last few years have been marked by some major leaps forward in augmented-, virtual-, and now mixed- reality. So much innovation is happening that it can be easy to forget that almost right from the start an enigmatic company has been waiting in the wings to sweep everyone else from the board.

That company is known as Magic Leap and it is probably one of the most mysterious names in the AR/MR business; so much so that for a long time it wasn’t really clear what it is they planned to sell. Today we know a bit more about the company, but they have been surprisingly good at avoiding leaks. Yet what has been teased by Magic Leap seems genuinely exciting, and if it’s more than just marketing fluff it could make all other MR innovations look childish in comparison. So let’s take a moment to recap what Magic Leap is and what we know about it.

magic leap whale

Silent Partners

Stop for a moment and consider the fact that Magic Leap was founded in 2010 and to this day has not released so much as a product shot. Yet the company managed to raise more than half a billion dollars all those years ago to develop their ideas. Tech giants Google and Qualcomm (who make your smartphone hardware) are both major investors in the company.

Before becoming “Magic Leap”, the company was known as “Magic Leap Studios”. The “Studios” version of the company was seated in the entertainment industry. It made movies and other traditional media. Then something changed; for now, the exact details are hidden behind pure stonewalling. The company did release an AR app all the way back in 2011 after becoming a corporation, but not much else was known.

Pulling in the Talent

In 2014, Magic Leap started making waves thanks to a few big names that started joining up. This was important, because these people were telling us by proxy that Magic Leap is doing something special. We might not know exactly what, but these people DO know and are convinced enough to drop everything and get on board.

Game designer Graeme Devine was one of the first notable joiners. Devine has worked for the absolute cream of the gaming crop, most notably ID Software – the guys responsible for so much graphical innovation. You might recall that ID Software also gave us programming genius John Carmack, who also believes in VR so much that he left ID to join Oculus permanently. Now even the CEO of Google is on their board of directors. Clearly a lot of very smart people believe in whatever it is that Magic Leap is selling, and so we should be paying attention!

So, What Are They Selling?

Curious? Well so are all of us! Various technology news outlets have done some digging over the years, so there are a couple of things we do know. The main fact everyone seems to be sure of is that Magic Leap is making an MR headset. In fact, they were making a mixed-reality product long before anyone knew to call it “mixed reality”. We haven’t actually seen even a real hardware prototype, but there are some patent mockups. For example, look at this picture from an article in The Verge.

Pretty futuristic! Other pictures show a totally different device that looks more like traditional eyeglasses. There’s also a unit clipped to the belt, suggesting that some hardware will be outside of the actual headset itself. We also don’t know if it uses offboard computing power or if everything is built in.

We have no idea how close the company is to consumer hardware, but in 2017 they indicated more money was needed.

The company was valued at an insane six billion dollars as I write this, so I have little doubt additional funding will be coming.

What Does the MR Look Like?

In the end the proof is in the pudding – the actual hardware the experiences will be delivered on. In that department they’ve had plenty to show. Look at this demo from 2015:

This is the embodiment of what they are trying to achieve. In particular, the realistic rendering of light so that virtual objects appear totally solid and seamless with the real world is apparent. Look at how virtual light sources interact with real objects. Even better, there’s occlusion so that virtual objects can move BEHIND real ones. The little robot floating between the table legs in particular is jaw-dropping.

One of the key technologies that Magic Leap seems to be using is the simulation of light fields. In other words, they use a computational process that calculates the path of all the photons on a given volume. It treats light as a sort of magnetic field. Light field technology is set to be a major part of both VR and AR, but very few hardware systems that use it have been seen. Also, such systems require an insane amount of processing power.

For example, cameras that capture light fields in order to create volumetric video need their own offboard server unit to process the data. I suspect that the main challenge Magic Leap is facing has to do with how to squeeze the tech down into something small enough to use.

Will Magic Leap Change Everything

Magic Leap is pursuing several Holy Grails of mixed reality all at once. It is an incredibly ambitious project, but with some of the finest minds and almost unlimited money behind it. Should the company succeed, we are going to see a generational leap in mixed-reality technology that will push the envelope hard. Will they make it? No one knows. If they do, things will never be the same.