When I saw the first promo videos of the Microsoft Hololens I honestly thought this was another one of those “visions of the future” videos they do every now and then. If you want a bit of a laugh, check out this old one:

It’s not that they got it wrong, just that it’s all so hilariously clunky. In any case, the Hololens demo came across as that sort of thing. Here comes a guy wearing a headset and the whole world around him turns into this holodeck-like world of floating graphics and virtual objects.

“This is just another concept” I thought to myself, but I could not have been more wrong. It turns out that not only was the Hololens not just a concept, but the working prototype used in live demos would soon be replaced by an actual commercial product small enough to fit everything into a headset.

What is the Microsoft Hololens?

The Hololens is a “mixed-reality” headset. You’ll hear about this concept a lot on this site, but mixed reality is a more advanced implementation of augmented reality. Mixed reality systems generally have technology in them that can map and measure the real world. This allows them to adjust the rendered imagery in such a way so that it seems to blend into the world seamlessly.

For example, if the system has a rangefinder to detect flat surfaces such as walls and tables, it can then render virtual objects that persist in their relationship with those surfaces. A virtual TV screen will appear to hang on the wall just like a real one; doing essentially the same job.

The Microsoft Hololens is one of the first truly practical mixed-reality headsets made, but it turns out it probably isn’t for the likes of you or me. At least not yet. It’s not exactly a made-up vision of the future, but it’s a reality only for those who can afford the $3000 price tag.

the Microsoft Hololens

All-In-Wonder

The Microsoft Hololens is a completely self-contained device. It doesn’t have a tether to a bigger, more powerful computer. There’s no backpack or wireless display transmission. What you see is what you get.

The headset isn’t exactly svelte though. It’s a chunky beast, but at least it isn’t too heavy. It comes in just a smidge under 600 grams. Compare that to something like the DJI Goggles drone cameras headset which weighs twice that much. That porker is plenty comfortable, according to users, so Microsoft hasn’t weighed early adopters down.

The design is pretty simple, though. A single headband wraps around the user’s head, with a semi-transparent visor covering the eyes. It’s actually quite elegant and contemporary. The headband is not monolithic. Instead there’s an inner ring that is what’s actually resting on your head. This inner ring can tilt back and forth, meaning you can adjust the position of the Hololens without having to shift its position on your head. That’s a big departure from the current generation of VR and AR headsets that use either the clunky three-strap design or the monolithic headband with tilt-up visor.

It goes beyond even that. What appears to be a solid band can actually EXPAND to accommodate bigger noggins. Good news for me, since I’ve never found an off-the-shelf hat that actually fits. I have a seriously large head, guys.

A Closer Look

If we really look at the device beyond that initial impression, it quickly becomes clear that there is a lot going on under the hood. Over the user’s eyes are what appear to be a pair of glasses. Above the “browline” of the visor we see several camera lenses. The optical systems are quite involved.

Looking at an exploded view of the Hololens, it turns out those little camera blisters we see are part of a complex sensor array nestled in a compartment above the visor. It’s like looking into the eyes of some strange techno-spider. These sensors are what let the Hololens physically detect the world around it. The two lenses that sit right in front of the eyes are referred to as “holographic” by Microsoft. They seem to be a type of optical waveguide that has high-quality imagery projected onto it.

The reason the imagery projected onto those lenses seem to fit so perfectly into the real world around us is thanks to a dedicated processor. This little electronic brain weaves together all that sensor data with the renderer and projection system to create mixed-reality images. Microsoft calls it the “HPU” or Holographics Processing Unit. There’s really nothing to compare it to, since it sits next to the CPU and GPU as its own distinct thing. It’s not quite light-field technology, as Magic Leap is attempting to do, but the HPU is certainly an incredibly impressive dedicated hardware component. It shows that specialized hardware has a future in VR, much in the same way we moved from software rendering to dedicated GPUs.

The one big drawback of the current hardware is its incredibly narrow field of view. While Microsoft has not given an official number, independent people who have used the Microsoft Hololens estimate that its FOV is around 35 degrees. Contrast that with premium VR headsets that start at 90 degrees and typically have a horizontal FOV of 110-degrees. It also means that many of the promotional videos that show us what users are seeing are way too optimistic about the field of view. A key improvement that has to be worked on is the widening of the FOV.

Hololens Apps

Of course, without actual software applications the Hololens is just a goofy-looking head ornament. However, with a 3K price tag it’s hard to imagine a flood of applications for the machine. That’s not to say there isn’t anything for the Hololens. The product is of keen interest to people in corporate environments or scientific fields. Even in education and training spheres you’ll find a market for the Hololens. It makes much more sense if you think of it as a shared organizational resource rather than something each individual person is expected to own. So companies, labs, and universities can purchase a small number of units to be shared.

So what apps are there worth knowing about? Let’s have a look at the most interesting Hololens apps.

Skype for Hololens

skype

Skype is a pretty mundane part of our lives these days, although 15 years ago I would have been flabbergasted by mainstream video-calling technology. Now I have Skype on my phone, tablet, and PC. Using Skype on a Hololens turns the magic back on again. While it doesn’t create a Star Wars-like holographic illusion, the Skype video box and UI elements are projected into the space surrounding you.

Of course, since you are wearing the Hololens the people you are speaking with can’t actually see your face, but Skype for Hololens has another trick up its sleeve that might make up for that particular shortcoming. You see, the people on the other end of the call who are just using a normal Skype device can see through the Hololen’s “eyes”. In other words, they can see what you are seeing.

Even better, they can use their tablet to draw directly onto your video feed. So it’s possible for someone to teach or instruct you remotely. The idea, it seems, is that people can get a better type of technical support. There are also business use cases where people have to collaborate on projects remotely. On top of this it features spatial sound, so you only hear the audio coming from the actual chat window. The app also allows for hand-tracking so that you too can draw and visually communicate with the other person.

Insight Heart

insight heart

I’m always fascinated by AR and VR applications that teach us about the physical world. I especially like anatomy applications. There are few things as beautiful as the complex mechanics of the human body.

Insight Heart is also available as an Apple ARKit program, but it’s the Hololens version that really caught my attention. The program focuses specifically on the human heart and does so in great detail. The heart is an organ that has played a central part in human culture. Before we knew what it actually did, many thought that the soul or mind was seated there. Even today, it has a special place in the mind, since although you can live with one lung or kidney a busted heart is a one-way ticket to Deadsville.

The actual model of the heart is beautifully detailed with great animation and sound. It shows that this application only focuses on one organ. I also think that Insight Heart is a great example of how to create an MR interface. The menus look intuitive and are crystal-clear.

Holobeam Tech

HoloBeam Tech

Remember how I said that Skype for Hololens didn’t do the whole Star Wars hologram thing? Well there’s an app for that and it is called Holobeam Tech. In fact, it’s not just me – the actual app description directly references Star Wars.

Holobeam Tech replicates that experience by using a special 3D capture method. The person you are speaking to is captured, digitized, and sent across the internet. To you it looks like their ghostly apparition is there in the room with you. It’s a little creepy, but also really cool. We’ve seen this sort of holographic telepresence thing in sci-fi movies for ages, but it’s awesome to see in real life. Of course, if this were to work both ways then both people would appear wearing their
Hololens units. A little lame, but future versions of the technology are likely to be much more compact and less intrusive.

HoloStudy (Demo)

HoloStudy

One of the most exciting applications of VR and MR technology is in education. Putting students in a virtual environment where you can control their first-person experience in detail allows for unique and powerful teaching potential. There’s a tech-demo out of exactly this sort of approach for the Hololens. It’s called HoloStudy and showcases a number of MR lessons in things like biology, chemistry, physics, and geology.

The imagination that went into these lessons is amazing. For example, the chemistry portion of the app starts you off with a virtual periodic table projected onto the wall. Then you can call up individual chemicals and view them as atomic or molecular models in 3D, 2D, or as “real” matter.

The geology lessons have an amazing 3D model that breaks the Earth up into its individual layers. The animation and solidity of the graphics are amazing, especially considering that this is mobile rendering hardware we’re talking about here. You can also see amazing models indicating how the Earth’s gravity puts a dent in space-time, and a great visualization of the planet’s magnetic field.

HoloAnatomy

HoloAnatomy

I have at least two anatomy apps on my iPad and one of them has a pretty sweet ARkit-powered mode where you can lay the virtual body down on a table and then examine all its bits and pieces. It’s an awesome experience and I have literally spent hours learning about all the parts of the body through pure discovery. I’d see something in the model and then wonder “What does that do?” with only a tap of the finger separating the question from the answer.

HoloAnatomy blows that app out of the water and then off the planet. It’s the product of Cleveland Clinic and was designed specifically for use in medical training. Groups of medical students who are wearing Hololens units can all see the same shared graphics. They can discuss and interact with the renderings of working, detailed human anatomy. The level of detail is simply astonishing and I really believe tools like these will become standard in medical training practice.

Shared mixed-reality experiences are a cornerstone of the future “AR world” I talk about in a different article on this site. Extend this concept to other fields and even just to our recreational activities, and you’ll have a good idea of what things will be like in the future.

MineCraft for Hololens

This is without a doubt the Hololens application that got the most attention. Minecraft is already such a phenomenon, but it’s inclusion in a mixed reality platform makes so much sense the minute you see it. Don’t take my word for it. Check out the official demo:

In the demo, the player walks up to a real physical table that the Hololens recognizes. It then proceeds to build a Minecraft world on that table. It looks like a little diorama or some tiny world lorded over by a mighty, glasses-wearing god. The persistence and detail are just out of this world. I’m not even that big a fan of Minecraft, but seeing this demo actually made me want to play this version.

This was also the game demo that made certain genres such as real-time strategy really click for me. Most of us tended to think only of first-person games. Seeing Minecraft like this is beyond cool, and many gamers who saw that original demo audibly gasped when it all came together.

Windows Mixed Reality to the Rescue

The Hololens is an astounding piece of technology that simply isn’t ready for the mainstream. The technology just isn’t there yet. It’s too expensive, for one thing, and the FOV limitations really hurt how attractive it is for consumer purposes.

Halolens Demonstration

Luckily, Microsoft did not chuck all of their eggs in one basket. They’ve expanded their mixed-reality ecosystem to include more traditional mixed-reality headsets. I discuss the platform more fully in another article in this section, but basically it’s like a normal VR headset with two cameras and a few other sensors stuck on the outside. In fact, I believe that right now those MR headsets have far more impact than the Hololens itself. They are the most practical and elegant VR/MR headsets I have seen so far. On top of that, they cost nearly ten times less than a hololens.

Of course, these are still tethered devices, whereas the Hololens is a complete unit. So it’s not exactly an apples-to-apples comparison. These two product lines are, however, linked in one very important way – applications. Strictly speaking, if an app works on a Windows MR headset then it can be ported to Hololens and vice versa.

The Windows MR headsets are set to become mainstream products and that means much more incentive for developers to write applications for them. One day the technology in the Hololens will be cheap enough that almost anyone can afford it. It would be good if a mature and wide variety of applications is ready and waiting when Hololens technology is ready for prime time, but only time will tell if that’s how things will turn out. For now, most of us can only marvel that such a thing actually exists.