Augmented reality and virtual reality have already been part of the media industry for several years and are used in many areas. Now the way is being prepared for a new phenomenon: Mixed Reality. Mixed Reality is becoming more and more important for both users and companies, and the areas of application are increasing. What it is and what is behind this new idea is explained below.
The basis of Mixed Reality
To understand what is behind Mixed Reality, one must first understand Spatial Computing. This is a technology that digitizes activities of people, objects and machine as well as their environments to enable and optimize interactions. People’s movements are thus recorded and these and their surroundings are placed in a digital framework. This makes it possible, for example, in augmented reality and virtual reality video games to interact with the objects in the game as if you were in the video game yourself.
The technology in everyday life
It is often assumed that augmented reality and virtual reality video games are the biggest area where spatial computing is applied. This is because many don’t know where Spatial Computing is everywhere. Voice-controlled assistants, smart speakers, and even industry have been able to advance thanks to the technology.
Augmented Reality and Virtual Reality
In order to understand what Mixed Reality actually is in the following, Augmented Reality and Virtual Reality must first be distinguished from each other. If graphics, holograms or video streams are transferred into the real world, this is called augmented reality. In a video game, for example, this means that the room you are in still looks the same and different graphics are superimposed. With virtual reality, a fully immersive, digital experience is presented and the actual view is obscured. Physical obstacles from the environment are digitally represented to prevent collisions. Augmented reality and virtual reality are found on what is known as the mixed reality spectrum. The poles of this spectrum are the physical and digital worlds. Augmented Reality is more on the physical world side, while Virtual Reality is more on the digital world side.
What is Mixed Reality?
Mixed reality is the next step in spatial computing. Detached from screen-based experiences, instinctive interactions with data in one’s own living space and with friends become possible. This is how transitions between augmented reality and virtual reality take place: the physical and digital worlds are mixed. Especially in social media, the use of mixed reality via handheld devices is increasing. The AR filters that can be used on Instagram, for example, fall under mixed reality experiences. With the new application possibilities, a digital object can now be placed in the physical world as if it were physically present. An avatar of a user can be used in the physical world, which can collaborate asynchronously with others at different times, and much more.
How does mixed reality work?
Advances in machine vision, graphics processing, and display technologies, can create this new mix of physical and digital worlds. Sensors have been able to improve so that computers can more fully sense the environment and human data input has become more diverse. What first began with keyboards and computer mice evolved through touch, freehand, and speech now to skeletal tracking. This discipline is referred to as Human Computer Interaction (“HCI”) or human-computer interaction. Data that can be obtained from the environment include the person’s body position, such as head movements, spatial boundaries, such as objects and walls, ambient light, ambient noise, and physical locations. Together with computer processing supported by the cloud, the input methods and perception of the environment form the basis for true mixed reality experiences.
The evolution of mixed reality
Back in 1994, the term mixed reality was introduced by Paul Milgram and Fumio Kishino. They discussed the concept of a virtuality continuum and the taxonomy of visual displays. Development progressed and besides displays, the application of Mixed Reality now includes environmental understanding, human understanding, spatial sound, locations and positioning in physical and virtual spaces, and collaboration on 3D resources in Mixed Reality domains. With these advances, it is now possible to move through the physical world while movements are mapped in a digital world.
Devices and experiences in mixed reality.
Mixed reality can currently be experienced through two main types of devices. The first type are holographic devices that can display digital objects as if they were present in the real world. Some of these devices have a transparent display so that the physical environment can still be seen. The second type are immersive VR devices that can create a sense of presence by completely blanking out the physical world and replacing it with a fully immersive, digital world. These devices often feature an opaque display. The real physical environment is completely blocked out while worn. Both types of devices can be made to connect to a separate PC via USB cable or WLAN, or they can be made to not connect. Currently, the devices can only cover a certain range within the mixed reality spectrum. There are still no devices that can call up the entire spectrum.
Mixed reality in industry
Spatial computing is often associated with video games and social media, but Mixed Reality can also be applied in industry. Starting with employee skills, new inspection content can be quickly transmitted to individual sites, and production can also be improved. Employees receive context-relevant information projected directly into their field of vision and at the respective location of the product.