They say that the eyes are the windows to the soul. I don’t know about that, but it is true that our eyes are an important part of how we express ourselves and communicate. We can tell where another human being is looking, for one thing. It seems like a pretty mundane thing, but humans have an incredible ability to determine the gaze of another person.
This is not something that many other animals can do. In fact, dogs can follow the case of a human being, but cats can’t. Why? Because we’ve selectively bred dogs over millennia to care about what we are looking at. Cats have had no need to develop that ability, so Mittens literally could not care less about where you are looking.
Our gaze represents an entirely separate channel of communication. We are sending out and receiving valuable information on this channel all the time. So why not make use of it in a digital context?
That’s exactly what scientists and engineers have been doing for ages. They have used specialized camera sensors to measure the position of the eye and figure out where it is looking. The most common type of tracking is known as “pupil center corneal reflection”. Basically, near-infrared light is shined at the eye and then a special high-resolution camera geared at picking up that wavelength of light sees the reflection off your cornea and calculates the position of your pupil.
There are lots of different applications for this technology. For example, for people who design computer interfaces it can be incredibly useful to know where on-screen the user is looking. Where is the eye naturally drawn? Does the user take a long time to deal with something visually? There’s a good chance that just about every major app or operating system that you’ve used has benefited from eye-tracking as part of its design.
Eye-tracking technology has also been a major tool for market researchers, psychologists, and medical researchers. It’s helped us figure out mental processes, attitudes, and more. So you might be asking, what relevance does that have to VR? You’ll soon see.
Don’t Look at Me Like That
When you currently use a VR HMD such as the Oculus or a Gear VR, your “gaze” is essentially the center of the viewport. Some applications use this to interact with the world. You can rest the reticle on a particular object and then make it do something by clicking a button or simply waiting.
The problem with this is that just because the viewport is centered on a spot doesn’t mean that you are actually looking at it. Our eyes and head move independently of each other, so the way that current VR systems determine gaze almost makes it feel as if the user is wearing a set of horse blinkers.
It’s not just about accuracy either. Imagine what a VR developer could do if they knew exactly what the user was looking at. For one thing, other characters in the VR world could be programmed to respond to your gaze in a more natural way. Are you looking past them at an object in the distance? They might turn to follow your gaze. Stare at another character’s eyes for too long and you might make them uncomfortable and provoke them.
Apart from making the virtual world react in a more lifelike way, eye tracking can actually make the virtual world look better. You see, normally we render everything on the screen at full detail, since we don’t know where your eye will happen to fall. If we can track your eye movement with speed and accuracy then we can deliver rendering resources on the fly to sharpen up the parts of the scene that are in focus. Alternatively, you could use this to keep the apparent detail the same as we’re used to today, but reduce the horsepower you need to drive it. Either way, it’s a win-win situation.
In the Real World
So are there any VR HMDs in the real world that can actually do this? It turns out that the answer is “yes”. A company called Fove has been the pioneer in this area. Their HMD has already been out in the wild for a while as I write this. It’s a pretty good HMD overall actually, with pretty decent specifications. A little bulkier than some, but it’s not far away from HMDs we already know in terms of size and weight. The eye tracking is apparently pretty good, and for a first-generation product it’s an amazing start.
The problem is that most people who are going to get a VR headset already have one without eye-tracking built in. So it’s a good thing that veteran eye-tracking company Tobii is planning to provide custom modification to just about any VR headset out there. Tobii has been the world-leading name in eye tracking for as long as I can remember, and is set to be a strong challenger to Fove.
The Tobii site detailing its technology as applied to VR pretty much lays out the same advantage I discussed above. Including “foveated rendering”, which means only rendering sharp graphics in spots the fovea is focused. They also say it will make for much better 3D effects.
There’s no doubt that we should all be keeping our eye on VR eye-tracking tech.