Have you ever wondered exactly how it is that your smartphone or tablet knows which way you’re holding it? How about the gamepad controller that came with your console? Motion sensors are all over the place these days, but few people stop to think how they actually work. The technology that underlies the various sensors that make VR possible is truly fascinating, and we’re all walking around with these little marvels in our pockets every day.
So let’s have a closer look at the devices which translate your movements in the real world to those in the virtual world.
Feeling the Need for Speed with Accelerometers
The accelerometer is the tiny technical marvel that lets your phone know which way is up and it’s how it knows whether to switch to landscape or portrait mode. One accelerometer can measure force in one direction. For example, one accelerometer can tell you if it’s pointing towards the ground or not. Gravity exerts force on it when it points down, but not when it points in another direction.
If you increase the number of accelerometers you can measure movement along all three axes. Not only that, but you can also measure how strong the acceleration is. Which means you can do things like translate the strength of a virtual golf swing or some other movement that needs variability in speed and strength.
The accelerometers inside the chips you find in both smartphones and VR devices are actually electromechanical. They are known as a MEMS or “micro electromechanical systems” accelerometer. One well-worn design is a small mechanical structure made from silicon. It almost looks like a comb when magnified. Any movement along its axis pulls on that structure, converting it to an electrical signal that the software can use.
While accelerometers are great at measuring linear movement, they are pretty awful at measuring rotation. In fact, they are quite useless at it. If you had a whole bunch of them you could try to approximate rotation, but it would be very stuttery, which is why accelerometers are almost always accompanied by another type of sensor – a gyroscope.
You Spin me Right Round Like a Gyroscope
If it’s the measurement of fine rotation you want to achieve, then a gyroscope is what you need. Just about all of us have seen a gyroscope before. They’re sold as educational toys and usually consist of a set of concentric rings with a spinning disc in the middle. Once the disc is spinning you can turn the rings that house it any way you want; the wheel keeps spinning in the same direction. If you measure the direction of the wheel then you’ll always know which way is up. This is why airplanes have gyroscopes – to help them achieve level flight even when they have little or no visibility.
Inside your phone or VR controller there’s a MEMS gyroscope that helps to make smooth rotational measurement possible. It’s not, however, a tiny replication of that mechanical spinning gyroscope. There are in fact quite a few different takes on making a gyro that small, but it gets pretty weird to us normal non-engineering types.
A popular take on the mechanism works like a tiny tuning fork, using minute vibrations to measure rotation. I have no idea how it works, but we can all agree that the end results of these modern microscopic gyros is amazing. Motion controllers now have almost one-to-one translation of our movement.
It’s not just in VR where this technology has proven revolutionary. Lots of different devices can benefit from an accurate gyro. Modern drones and other radio controlled aircraft wouldn’t be such successes if it weren’t for their built-in gyros. Keeping a quadcopter level is basically impossible without modern self-stabilizing gyros, which is why full-scale quadcopters never really took off – literally or figuratively.
The Attraction of a Magnetometer
A magnetometer is, at the most basic level, a device that can detect a magnetic field. How exactly does that help you measure motion? Well, since the Earth is bathed in a magnetic field the magnetometer can act as an electronic compass. The magnetic north pole of the earth provides an absolute reference point which can be incredibly valuable when you need to do the math around positioning. It’s not the most reliable absolute reference point, but it can help.
The magnetometer has also been repurposed in an ingenious way by Google for their Cardboard VR design. The first generation of Google cardboard used a set of magnets as a wireless switch. By moving one magnet over another the magnetometer picks up a fluctuation in magnetic field strength. Google used that as a signal to the phone inside the cardboard headset frame to interact with stuff in VR. Pretty ingenious, if you ask me. Of course, the Cardboard 2.0 now uses a much more boring conductive lever button thing, but it was a fine demonstration while it lasted.
The Lidless Eye of Optical Tracking
If you’ve ever checked out the special features of a CG-heavy DVD or Blu Ray you’ve probably seen actors covered in little white balls and skintight suits. Think of Andy Serkis as Gollum in the Lord of the Rings or Bill Nighy as Davey Jones in the Pirates of the Caribbean.
What you’re seeing is optical motion tracking. Usually infrared light, which is invisible to the naked eye, is beamed out onto the scene and then captured by a special IR-sensitive camera. The dots are places on specific parts of the actor’s body and face, so that an accurate wireframe of their motion can be built and then mapped onto a CG character, which is what you see in the film itself.
Premium VR headsets such as the Oculus Rift use exactly the same technology, except instead of reflective balls there’s an array of IR LEDS on the headset which is detected by one or more IR cameras. In the case of the Oculus Rift there’s an array of these LEDs that help the system map the movement of your head more accurately and also lets it calculate lateral movement; in other words, when you lean in closer or lean back. By combining the sensor data from the IR camera with all the other sensors we’ve mentioned, almost perfect, jitter-free tracking is possible – a key component in truly immersive VR.
A Sixth Sense
As you can tell, sensors such as these are critical in making VR a practical technology. The more accurate the sensor or combination of different sensors, the more realistic the VR experience. The future will probably hold even more advanced sensors and we’re already seeing devices that measure muscle movement directly, such as the Myo armband that allows for finger and hand tracking. Some HMD makers are also building eye tracking technology into their products, with a company called Fove leading the market in this regard.
So expect to have your analogue self ever more digitized thanks to the ongoing work on the science of sensors.