Sydney Butler · November 18, 2024 · 7 min read Screens are all around us. Right now you’re looking at a screen. It could be on a phone, a tablet or a desktop computer. Now, imagine if the digital world of the screen wasn’t just an object in the real world. Instead, digital imagery and data are superimposed and integrated into the space around you. It’s not virtual reality, it’s augmented reality.
In this article we’ll explore what augmented reality is and if this technology could replace all forms of screen as we know them today?
What Is Augmented Reality?
The term “augmented reality” means what it says. You’re augmenting the real world around you with some sort of additional information. The heads-up display in a modern car or a jet fighter is a sort of basic augmented reality. You’re looking at the world and information about things such as speed or altitude are projected into your line of sight.
When people use the term “Augmented Reality” these days it usually refers to something more sophisticated. For example, it could be a movie poster that springs to life or a sophisticated simulation that makes it look like fish are swimming around the room, as if the air around you was suddenly replaced with water.
How Does Augmented Reality Work?
Modern augmented reality comes in two flavors: marker-based and markerless. Marker-based augmented reality needs some sort of prepared object in the real world that tells the software where and when to project the virtual imagery.
A marker can be anything, but often it’s a card or a special QR code printed in a magazine. Many older AR games such as the PlayStation 3 game The Eye of Judgement and AR games that shipped with the Nintendo 3DS made use of card markers for their AR mechanics.
These days, thanks to more advanced technology, markers aren’t really needed. Sophisticated machine vision technology can make sense of the world around you. For example, it can recognize flat surfaces such as tables and vertical surfaces like walls. Accurately projecting digital imagery onto them. There are many AR apps that do this quite well.
Augmented reality software does all the math in the background to make the perspective of the images appear correct, solid and grounded in the real world. These days they can even take note of the current lighting in the room and make the projected image blend in with it!
The Extended Reality Spectrum
The technologies that can alter your perception of reality are beginning to multiply. They come in all varieties, which means our language has to adapt to accommodate them. Which is why we have the term “extended” reality.
Extended reality describes the entire spectrum starting with your normal, natural view of the world and then moving from augmented reality to mixed reality, ending with fully immersive virtual reality. Today, people tend to use augmented- and mixed- reality interchangeably, but mixed-reality usually refers to the more advanced experiences that incorporate the real-world space and objects around you.
Augmented Reality In Real Life
You can try augmented reality right now! If you have an Android or iOS phone you can download a game such as Pokemon Go and see what a digital game character projected into the real world looks like.
Android uses a technology known as ARCore, while iOS uses ARKit. Both let applications do sophisticated AR tricks using nothing more than the phone’s camera.
If you have a small mountain of cash laying around, you could also try something like the Microsoft HoloLens, which is quite possibly the most advanced augmented reality headset in existence.
The Future Of Augmented Reality
There are many exciting augmented reality projects in development that might give us some idea of what augmented reality might look like in the future. The biggest change would be moving augmented reality from your phone screen to a display device that you wear on your head.
Head-mounted augmented reality makes the most sense for the technology to flourish. It would be the most natural way to use the technology. Some of the most impressive applications of modern augmented reality use headsets. The problem is that these headsets are large and expensive. You couldn’t wear them all the time.
For augmented reality to truly come into its own, it needs to shrink down at least to a similar size and shape as standard glasses. You probably remember the Google Glass project, which used a tiny projector to display very basic information in your field of view. Future AR needs to be at least that compact, while offering a proper mixed-reality experience.
While no one knows if, how or when this will happen, we can make a few educated guesses. We expect the computer brain of this device to either be powered from the cloud or by a local computer box, such as a smartphone-class device in your pocket.
The display unit itself will probably use very small cameras for environment tracking and direct retinal projection to actually get the images into your eyeballs. Retinal projection technology already exists in various forms, and there are AR prototypes that make use of it.
In the very long term, we might see augmented reality delivered through devices similar to Elon Musk’s Neuralink device, but that’s far into the future.
Augmented Reality As a Universal Display Device
So, could augmented reality replace every screen you use today? The short answer is yes. In principle it could become the only digital display system you’d need. After all, you’d have the ability to conjure virtual “screens” anywhere you want. Need a massive 100-inch TV? Just have one appear in the desired spot!
In fact, augmented reality technology will probably break down the concept of a screen entirely. Perhaps your actual real-world desktop could also be your computer desktop. Applications can float freely in space around you, in whatever form is most useful. Some people might prefer their alarm app to look like an actual physical alarm sitting on a table. Others might like a time readout to be more abstract and simply appear as digital information floating in their field of vision.
In the long term we could find ourselves living in an “AR world”, where without an AR display things would look pretty strange to us in the here and now. Without AR glasses there would be no billboards, no advertising, no signs, no labels on anything at all! In the same way that everyone is expected to have a smartphone these days, you might need an augmented reality headset to get around the everyday world of the future.
For this to happen however, we need that AR headset to fulfill a lot of different roles. It needs to provide incredibly high-resolution images for a start. It also needs to use an as-yet developed universal wireless display standard. So that any device can interface with it. It has to be small, light and comfortable as well. It has to cater to normal users as well professional ones, though perhaps not in the same model.
Most importantly, it needs to effectively disappear from your awareness! A tall order, but that would make an augmented reality world possible.
