What is augmented reality?

Augmented reality is the enhancement of real-world experiences with computer-generated information. Discover new possibilities in gaming, education, art, design, manufacturing, marketing, and more.

How augmented reality (AR) works.

AR can trigger your senses of touch, smell, or even taste, although it most commonly augments what you see and hear. By using virtual information to enhance your real-world experience, AR differs from virtual reality (VR), which simply places you in a 360-degree virtual world.

AR works by mapping 3-dimensional virtual objects onto a real environment. Some AR headsets display virtual objects over clear lenses, while others put a live camera feed between the viewer and the physical world. Computer vision — the programming that enables the computer to identify and process what the camera sees — makes sense of that 3D environment and places digital features within it. As this digital content is rendered ever more quickly and convincingly, it comes closer to looking and feeling like its real-world inspiration.

The types of AR devices.

There are two ways to put the digital camera between the viewer’s eye and the physical world:

style

Grid width 8, static links

style

Two up, grid width 10

Something on your face (the head-mounted display).

Mounting a screen in front of your eyes allows you to augment reality while keeping your hands free. Google did this in 2013 when they released Glass, but it wasn’t true AR in the sense that we think of it today. Glass presented digital information on a flat display in front of one eye instead of as 3D imagery presented binocularly as part of the environment. Google still sells Glass in an Enterprise Edition for use in manufacturing, health care, and other industries. Without interrupting their workflow, Glass wearers can access manuals, training videos, design specifications, and specially developed apps.

The Microsoft Hololens and the Magic Leap One are the most well-known AR headsets for gaming and entertainment. These headsets are getting better at blending the real-world environment with digital content. “When you wear the Magic Leap, it does a scan of the room and actually builds — in close to real-time — a rough model of the room,” says Seth Chaffee, Head of Immersive Production at AR/VR production company Giant Astronaut. “So when you start placing objects in your environment, it’s easier to map that experience into real-world space.” The more these headsets can integrate with the environment, including the sonic environment (so you hear sounds coming from particular points in space), the more real the experience will feel.

Some developers envision a world in which we’re all wearing smart glasses or contact lenses all the time, but we’re not there yet. Current AR glasses tend to be bulky, with a limited battery life. Worse, the wearer’s field of view is narrowed, limiting situational awareness in a way that may be dangerous.

Also, trying to make sense of 3D models overlaying your real-world vision can be exhausting. It takes fighter pilots six weeks of near-constant training to learn how to use their heads-up displays without being distracted by the overlays. “Other than the targeting reticle, they pretty much use AR only at night in order to see the landscape,” says Alex Kauffmann, Project Lead at Google’s Advanced Technologies and Projects. “When there are other things to see, overlays can distract you from what you need to be paying attention to, like other airplanes or a missile coming at you.”

There’s also the question of whether or not people want to wear AR glasses as a part of everyday life. “I don’t want to live in a world where my reality is mediated in that way,” Kauffmann says. A few bar owners in San Francisco don’t want their patrons to live in that world either, banning Glass wearers (“Glassholes”) to prevent recording by their video cameras.

style

Grid width 8, static links

style

Two up, grid width 10

Something in your hand (the smartphone or tablet).

Phones are great tools for using augmented reality because nearly everyone has one, and every smartphone has a built-in camera and access to mobile apps. Certainly part of the success of Pokémon GO, the AR game app downloaded more than 500 million times in its first year, had to do with the fact that people already had the tools to play it.

Like Pokémon GO, Ikea’s augmented reality app IKEA Place takes advantage of smartphone technology. With the app, users can see how a piece of furniture will look in their home before they buy it. AR might be useful for bulky items like furniture, but it’s less useful for products like toys, which don’t need to be previewed in an environment.

The biggest drawback of AR apps is that you have to hold your phone up so you can see the screen. “You’re having to lift your arm up, and it’s a lot of physical engagement,” Chaffee says. “It’s just not intuitive.” Also, with phones or smart glasses, battery life can be an issue. “With Pokémon GO, most people turned off the AR part pretty quickly, because it used up all their battery,” Kauffmann says.

AR glasses, headsets, and phones all have limitations to overcome, but the technology is rapidly improving.

style

Grid width 8, static links

What can you do with AR technology?

AR goes beyond simply slapping a virtual mustache on your face in Snapchat. We’re just beginning to explore the possibilities. “It may feel to a lot of people like it’s late in the game, but this is early Wild West times for AR/VR,” says Chaffee. “There’s still a huge opportunity for someone to jump in and make an impact.”

Both Chaffee and Kauffmann advise thinking beyond the fleeting “gee-whiz” excitement people feel when they encounter AR for the first time. Instead of using it for its own sake, imagine how you could use AR to help you tell a story. “Don’t ignore the emotional side of the technology,” Chaffee says. “So many people focus on the tech. They don’t think about the importance of human engagement. And that’s really where AR and VR are going to shine.”

style

Grid width 8, static links

Artist Estella Tse creates augmented reality artwork

Artwork by Estella Tse from Adobe Create

style

Two up, grid width 10

Artists like Estella Tse are already using AR to tell engaging stories. Watch a video about Tse and her half-AR piece “Two Sides of the Same Coin,” in which she portrays the internal struggle between conformity and creativity. She exhibited this project at the 2018 Festival of the Impossible, a show dedicated to work by AR and VR artists. The 2019 Festival featured exhibits that allow you to speak to an android in the future, watch a plant that feeds on social media “likes,” get swaddled in a compression carpet, and more.

One aspect of AR that Pokémon GO tapped into was the potential for generating shared experiences. “Getting a lot of people to see a thing,” as Kauffmann puts it. “That’s what Pokémon GO did that nobody else had done — this shared AR. You can put something in the environment that you and I can look at from different vantage points, and it’s exactly where it should be for both of us.”

style

Grid width 8, static links

Both Kauffmann and Chaffee think it’s creatives who will take AR into its most exciting and surprising directions. Chaffee credits storytellers with challenging technology to have purpose and help people improve their lives. And Kauffmann thinks creatives will be able to make AR without having to learn complicated coding skills.

To dig deeper into the possibilities, check out 5 innovative examples of AR in action, and watch Adobe product manager Chantel Benson talk about using augmented reality in Adobe Illustrator and Adobe Photoshop at Adobe Max 2018. Get inspired and start daydreaming. Then, start experimenting and see what you can make.

style

Grid width 8, static links