Google has seen the future, and it is augmented reality (AR). To that end, Google and other companies are releasing more and more AR development tools.
ARCore is Google's core technology primed to make augmented reality a, well, reality. With the ARCore Software Developer Kit (SDK), developers can work with augmented reality on Android, iOS, Unreal, and the web.
Because of ARCore and similar tools, developing AR has become considerably easier, which means more developers can hop on board this particular bandwagon. If the trend continues, AR developers will be in considerable demand. According to IDC, worldwide spending on AR and VR products will reach $27 billion in 2018 (which is a 92% increase over 2017).
Read this cheat sheet to learn how ARCore works and how to use it. We'll update this article when there is new information about ARCore.
SEE: Virtual and augmented reality policy (Tech Pro Research)
What is ARCore?
ARCore is a Software Developer Kit (SDK) that, in conjunction with the Unity 2D and 3D content-creation engine, enables developers to work with augmented reality on Android, iOS, Unreal, and the web. ARCore is Google's answer to Apple's ARKit.
Augmented reality (AR) is defined as an interactive experience based on a real-world environment, with various elements that are augmented by computer-generated perceptual information; this includes visual, auditory, haptic (informs a user that input has been accepted), somatosensory (a system within the body that allows a user to register that haptic feedback has occurred), and olfactory elements. In other words, what you perceive in reality is enhanced via technology.
In order to use ARCore, developers need to work with the specific SDKs. Here are links to Google's quickstart guides to use with specific toolkits.
How does ARCore work?
ARCore does two things:
- It tracks the position of the mobile device as it moves; and
- It builds its own understanding of the real world.
Motion tracking for ARCore is handled by the phone's built-in sensors, and the device camera is used to pick out points of interest called "features." With the combination of location and features, ARCore is capable of determining the phone's position and orientation as it moves. ARCore can also detect flat surfaces such as tables and floors, as well as estimate the average lighting in an area. When you combine these capabilities, ARCore is able to understand its surroundings; because of this, developers can overlay objects, annotations, and other information in such a way that they seamlessly integrate with reality. With this ability, the possibilities are nearly endless.
Let's dive deeper into the key concepts and elements of ARCore.
As a device moves, ARCore uses a process called Concurrent Odometry and Mapping (COM) that allows the device to understand where it is in relation to the world around it. ARCore is able to detect distinct features—called feature points—in any captured image. These feature points are used to compute the device's change in location.
Feature points are used in conjunction with inertial measurements (from a device's Inertial Measurement Unit sensor) to estimate the Position and Orientation (pose) of the camera to the surroundings over a period of time. Thanks to a combination of the virtual camera pose and the device camera pose (provided by ARCore), developers can easily render virtual content from the correct perspective and overlay that content on top of the image displayed on the device.
ARCore also uses hit testing to get an x,y coordinate that corresponds to the phone's screen and projects a ray into the camera's view. This enables users to select and interact with objects in the environment.
ARCore looks for groups of feature points arranged on common horizontal or vertical surfaces such as tables or walls and makes them available to an app as planes. ARCore is capable of determining each plane's boundary and passes that information to an app, so that it's possible to overlay virtual objects on flat surfaces. A caveat: Untextured surfaces such as white walls may not be detected properly.
ARCore also uses orientation points, which allows for objects to be placed on angled surfaces. When an app performs a hit test that returns a feature point, ARCore will examine nearby feature points and use them to estimate the angle of the surface. Once the angle is calculated, ARCore will return a pose, taking the angle into account.
ARCore is capable of detecting information about surrounding lighting, providing the average intensity and color correction of a given camera image. With this information, virtual objects appear to be lit under the same conditions as the surrounding environment, thereby enhancing the sense of realism.
Anchors and trackables
To place a virtual object, an anchor must be defined so that ARCore can track the object's position over time. Anchors are most often created based on a pose returned by a hit test. Since poses can change, they require ARCore to update the position of all environmental objects such as planes and features over time. Because of this, planes and points are relegated to a special type of object called trackables—this simply means that ARCore can track objects over time. Virtual objects can be anchored to specific trackables to ensure the relationship between the object and the trackable remains stable as the device moves around.
Augmented images allow developers to build AR apps that can respond to specific 2D images; these images can be product packaging, movie posters, and more. With the help of augmented images, users can trigger an augmented reality experience when their device's camera detects such a 2D image. For example, a user could point their camera to a product package and see the actual product appear to show how it looks from multiple angles or how it functions.
How is ARCore different from Apple's ARKit?
The biggest difference between ARCore and Apple's ARKit is with mapping, which is ARCore's ability to gather and store localized information about a 3D world that can be utilized later. ARKit makes use of a "sliding window," which can only store a limited amount of location data that is constrained to a recent past. ARCore also has the ability to manage far greater map data, which, in turn, leads to a much more stable data set.
How is ARCore different from Google's Project Tango?
Google's other augmented reality project, the recently deprecated Tango, is quite different from ARCore. With Tango, specialized sensors and cameras are required; with ARCore, nothing more than a supported smartphone or tablet is necessary. Although there are a few phones that support Tango, those phones aren't in the mainstream and they have, for the most part, failed to deliver. With ARCore, any device running Android 7.0 will do.
Why should businesses care about ARCore?
The primary reason why a business should take a serious look at ARCore is because of how it can help create interactive worlds for users. For instance, if you are in the business of selling products that would benefit from users being able to see your products placed in a 3D overlay in their device screens (similar to Home Depot's "View In Your Space"), check out ARCore. Or, if you want to create interactive promotional materials to highlight your company, ARCore might be the tool for you.
What are examples of AR?
AR can come in many forms. One very basic augmented reality example is a sightseeing tour wherein the experience is informed via smartphone technology that allows users to garner more information about a particular piece of history than would be available in a typical installation.
Another example of augmented reality is using Google Street View to enhance the information you receive while viewing a location. Or maybe you need instructions on how to adjust the gears on your bike... augmented reality is there to help you in ways a standard video or how-to cannot. The digital information is seamlessly woven into the real world, such that the results are an immersive experience.
A great real-world example of AR is this "Take On Me" video, developed by Trixi Studios. Another outstanding and more hands-on example is the Google Measure app (Figure A), which helps you easily take measurements of larger objects with the help of ARCore.
Is ARCore available to use now?
ARCore was made available in 2017 and has been gaining steam ever since. With the correct SDK and supported hardware, any developer can begin working with Google's AR platform immediately.
How can developers start using ARCore?
Developers must first download and install the software developer kit of their choice. Once they have the SDK, apps can be developed for supported devices and platforms. For a complete list of supported devices, check out this hardware matrix. As for operating systems, Android 7.0 or later and iOS 11.0 or later must be used. ARCore also requires access to the Google Play Store, and iOS devices must be ARKit compatible.