Before ARcore, when Apple announced and then released their augmented reality developer kit, known as ARKit, it really took the VR and AR world by storm. This was some truly next-generation stuff unleashed on an unsuspecting market. It was augmented reality that did not need special markers. It can map out the dimensions of the room you’re in, find surfaces, remember them, and then project solid and convincing digital objects into the camera feed. ARKit is a quantum leap compared to basic AR apps we’ve already seen on smartphones and tablets.
Of course, we’ve seen this level of augmented reality before. Various outfits have been working on advanced AR systems for years now. Almost all of these have relied on specialized hardware in order to achieve the sort of spatial mapping and processing you need to craft such convincing digital illusions.
Google Does the Tango
The most famous project in this regard has to be Google Tango – a hardware standard created by the tech giant using multiple specialized sensors to do accurate range-finding and quickly create a virtual map of the 3D space around you. It worked fantastically, and the idea was that future phones would ship with all the Tango hardware built right in. The big downside to this is that it limits the number of people who can use this special AR platform to only those who buy phones designed for that purpose. It’s not exactly a recipe for mainstream success.
ARKit will run on just about any new Apple device. That makes it attractive to developers who know that they have access to virtually the entire install base of iOS machines. It becomes a virtuous cycle where people already have the platform so more developers make AR apps, which make money and then attract more developers. It also means that the quality of AR apps keeps getting better because of competition, just as the standards for app innovation have skyrocketed since the idea of a smartphone app was first introduced. There are no fart apps on the Top 10 list anymore!
ARCore is Ready to Rumble
It’s in this context that Google’s development of ARCore has been happening. Long before Apple was ready to release ARKit (which it now has) Google has been playing catchup. As I write this, ARKit has been out for some time. You can find plenty of writing about it on this site, and even app reviews and recommendations. It’s a mature and shockingly reliable platform. This is why I expect that Google will come out swinging with ARCore, the Android equivalent of ARKit. Unfortunately, given the nature of Android, they have a much bigger job ahead of them.
Why Android Makes ARCore Tricky
The big difference between iOS and Android is that iOS is a closed platform. Just like Mac computers, Apple has full control of both the hardware and the software. This is an approach that comes with pros and cons. In my opinion it’s the main reason that Apple computers never became dominant. Microsoft was wise enough to open Windows up to any PC maker, which meant the install base became huge and developers were lured to make software for the larger market. Apple was relegated to catering for niche creative markets like photo manipulation, desktop publishing, and film editing.
In this analogy, Android is like Windows. While Google is the custodian of Android, any phone maker can put it on their product. It’s also driven down the price of smartphones since it runs on low spec and high spec machines. The phone maker can concentrate on making the hardware and if they feel it’s worth it, they can customize the look and feel of Android, which is why Samsung Android phones have TouchWiz and Mi phones have MIUI, as an example. Since Google has given up control of the hardware their OS runs on, it makes it incredibly difficult to put out standardized hardware APIs similar to ARKit.
Apple knows every component in every one of its products. If you take two random Android phones there’s little chance they both will have GPUs, processors, or RAM that are anything alike in type or performance. You can’t count on a given Android phone having a specific quality of camera sensor or level of gyroscopic accuracy.
It’s also why gaming on iOS is a much better experience than on Android in general. How many times have you downloaded a new Android game only to find that it doesn’t play well with your phone model? It might be too slow, and crash or glitch out. There are simply too many hardware combinations to test for all of them. It’s for these reasons that Google has its work cut out in releasing a competitor to ARKit.
Before we go into the details of ARCore, let’s first explain exactly what it is.
Paint By Numbers
Like ARKit, ARCore is a software development kit. Developers who make applications use these as a way to standardize their software with various hardware and operating systems. It also means that they don’t have to reinvent the wheel every time they make new software.
In the case of ARCore, the idea is that a software developer who wants to use AR as a part of their application just needs to send the right inputs and outputs to the ARCore software and, presto, you have an AR application. Obviously it’s much more complicated than this, but it comes down to Google solving the hard problems of AR in Android on behalf of developers and then making that solution available so they can focus on creating cool stuff with AR rather than grappling with AR technology itself.
What Can It Do?
OK, now it’s time to talk about the actual features of ARCore. Whether you’re someone who wants to make AR apps or just use them, these are the abilities you can expect from the system.
First of all, ARCore is capable of motion tracking using the camera feed and motion sensor data. In other words, just like ARKit it can find “landmarks” and stay oriented, which in turn means 3D objects can “stick” to their location in the real world.
That wouldn’t be much help if it couldn’t tell whether something was a table or a wall. So ARCore also has something known as “environmental understanding”, a fancy way of saying that it can figure out from the camera picture whether something is flat, horizontal, or vertical – the basic building blocks of mapping a room.
Now that it knows where things are and what they are, it needs to actually draw 3D objects in the scene so that they look as if they’re really there. One of the main reasons that AR looks fake is because it doesn’t match the ambient lighting. If the 3D object is lit arbitrarily then it stands out like a sore thumb. It also needs to cast a believable shadow. It’s funny how you can tell immediately when these things are missing, but can’t always put your finger on why it looks so wrong.
Who Can Use It?
Just as with ARKit, there are minimum requirements for ARCore to work on a given phone.
First of all, the phone must run at least Android N or “Nougat”. For the preview version of ARCore the only phones that support it are the Google Pixel phones (obviously) and the top-class Samsung Galaxy S8.
ARCore is big, but still an underdog. With the release of ARCore, millions more people will have access to advanced AR experiences. Although Apple handily beat Google to this milestone, there are still far more Android phones out in the world than iOS ones. This means that ARCore is possibly more important for the AR industry overall than ARKit is. That’s not to say that ARKit means nothing. I’ve said myself that it’s an incredible achievement and a real game changer.
However, if only flagship-class phone owners need to apply when it comes to ARCore, the whole thing may be a moot point anyway. It may be that the average Android phone still doesn’t have the sort of hardware to make ARCore work as intended. One real risk is that ARCore gives AR a bad name thanks to all the inconsistency of the market.
In the end, I’m just ecstatic that AR is moving forward and that it’s not locked into elite, experimental hardware, because that means everyone wins.