Google Is Adding the Power of Augmented Reality to Your Smartphone

ARCore in Action

Google recently debuted ARCore, the Android equivalent to the Apple ARKit, in limited preview form. ARCore is a baked-in augmented reality (AR) platform for developers, intended to be less powerful than Tango’s custom hardware, but more accessible. For now it’s supported by both Android 7.0 Nougat and Android Oreo, on the Samsung Galaxy S8 and Google Pixel phones. Google plans for ARCore to work with 100 million existing and upcoming devices following an official launch this winter.

AR places computer-generated images into your field of view, making them part of the real world. For example, I might play a game using my smartphone that lets me capture digital creatures added to the real world around me, or I could train my smartphone on anything from a property that's for sale to a crime scene and get details about those locations. AR is different than virtual reality (VR): with VR, an entire virtual world replaces the real world, whereas with AR, virtual elements enhance the real world.

Since 2016, Google has been featuring Tango as its smartphone AR platform, but it requires specific requirements, such as custom sensors and other specialized hardware for its cameras. This has prevented Tango from truly going mainstream; it's even failed to gain a real foothold among tekkies. ARCore is Google's ticket to taking AR and Tango out of development and into the hands of Android users in a more general way: as part of devices that already exist.

 

ARCore has three basic components: Motion tracking uses video footage and internal sensors to estimate the phone’s relative location, so users can pin digital objects in one real world place and move around them. Environmental understanding detects flat surfaces with the camera, and light estimation adds a realistic touch. It also allows the digital pieces to fit in with their real world surroundings — by casting accurate shadows, for example.

AR in your phone

Clay Bavor, Google's head of augmented and virtual reality, spoke with The Verge about some potential future applications of the technology. Google expects interior design apps, and other tools that allow users to manipulate the real environment with digital objects, could be one avenue. Bavor told The Verge that an ARCore user at Google created an app to explain how to use Google’s complicated espresso machine, which shows users overlaid instructions such as “Don't touch this piece, it's hot," when they hold their phone up to it.

Eventually, ARCore might be combined with a visual search tool like Google Lens to transform the way we get information. For example, instead of Googling instructions for how to use an espresso machine, you could take a picture of the machine — which the visual search feature would identify automatically. Then, the AR feature would provide you with an overlay of instructions so you could actually see which levers to pull on the machine that's right in front of you — as opposed to something like a labeled photograph or video demonstration.

ARCore is evidence of Google's determination to help us all make sense of what we see around us, and to be part of the movement to digitally enhance our world.