HoloLens: But is this “mixed reality” really useful to us?

Microsoft with the HoloLens promises to change our lives. The problem is that, in the field of technology, not always the good intentions have turned into successes.

In this article, we try to understand how much HoloLens could change our habits.

HoloLens is a PC-independent unit with an integrated holographic processor unit (HPU). The HPU manages the data collected by the many sensors lightening the load on the CPU and GPU.

When we start looking around with Hololens, they constantly map the environment.

The operating system is obviously Windows 10, so the start menu is the Start menu of the above system.

To get the hologram of the Start menu we need to execute the gesture “Bloom”. To execute the Bloom we hold the punch and then open our hand (for basic principles of HoloLens let click ).

This is where the first perplexities begin. The Bloom is not always recognized by the system and it may happen that it is necessary to repeat it more than once.

The Start menu is one of the few holograms that is not placed in relation to the environment. It follows the gaze.

We can instead anchor the application windows, such as the browser, to a wall of the room we are in. You can run up to three applications simultaneously. Fixing windows to the wall is like having several virtual screens in various parts of the room.

To click one of the Start menu items, place the cursor moving your head (“gaze”) and perform the gesture called “Air-tap”.

This gesture is very intuitive because it simulates the use of touch screens.

HoloLens allows you to bring an object displayed on a desktop PC outside the display. In this way, the object floats in the air and you can move it with the mouse in the real environment! Good is the ability to modify the design elements of a lamp or a car, overlaying them to the real product to test its impact.

There are many interactions between users and holograms as well as between holograms and real environment. For example, you can create a robot and drive it into your room among the furniture. Moreover, you can create a ball that can bounce on the floor. We can also play a soccer game with the holograms that kick the balls on our table or on our floor (without the risk of breaking our ornaments!).

You can take pictures of the environment as it appears in mixed reality. This feature has a great potential in prototyping. In fact, it is very useful to share experiences through photos or videos of what’s being created.

The environment mapping of HoloLens can be useful to create apps. For example a 3D scanner app. HoloLens is able to create a 3D (not very accurate) model of our room by “looking at it”. This functionality is “spatial mapping”. Of course before to consider it as a 3D scanner there is a bit of road yet to go.

There is also the ability to impart voice commands without the classic “Ehi Siri” or “Ok Google” addressing commands.

HoloLens can also be used as a PC extension. For example, Holographic Workstation uses HoloLens to create 3D real-time financial data holograms.

A three-level system allows traders to interact with large amounts of data in a 3D and 2D combined environment. The user can share information with other HoloLens holders. Interesting is the ability to use HoloLens even sitting at PC. This offering an integration with traditional tools.

There are many design-oriented applications such as NASA Mars. NASA Jet Propulsion Laboratory uses Hololens to help scientists on the Mars projects. The images captured by Mars Curiosity Rover and the Mars Reconnaissance Orbiter are displayed on the HoloLens. It provides a mixed reality view where virtual objects can appear along with real ones, recreating the Martian setting in the room. Designers working on the next Mars rover use HoloLens to view their 3D plans in scale, before any real manufacturing.

It’s very interesting the use of HoloLens for rover design. Coming back down to earth there are also applications for architectural design. An example is a planimetry you can extrude, rotate and scale to the desired size. You can enrich the models with a lot of information, images, and videos floating in the air. The models can be placed anywhere in the room and you can make selective viewing of some layers. You can ask for an overview as well as look at the interior of individual environments. You can enlarge the model until you find yourself inside. You can also make changes directly to the hologram. Anyone who has worked or works in the 3D field knows how uncomfortable is to manage a model on the traditional screen.

Remaining in the field of architecture, we also find applications in interior design. For example, the user can see his new kitchen at the design stage. Moreover is possible to enter in the new virtual kitchen even overlaying it on the real kitchen. The user has the ability to change the furniture in the design, shape, and color. The user can also change the color of the walls, add or remove elements and decide their size. HoloLens can record the user’s experience. The application creates reports showing the parts on which the user has focused more and the parts that the user liked more (the “feeling”). In the same way, the user’s commentary words statistics (spoken during the experience) can be shown. This is an effective application for both design and marketing.

Stryker uses HoloLens for the design of operating rooms. Physicians and nurses, with HoloLens, can preview how the operating room would be design. It permits to understand in advance any changes to make. In this way is possible to achieve the highest level of efficiency, comfort, and operation. There is another company, the ThyssenKrupp, which uses the HoloLens for the design of lifts for the disabled.

These are examples of companies who strongly believe in HoloLens.

Interesting is the use by Japan Airlines that uses HoloLens to train its mechanics.

Exists an application for the performance analysis of PGA Tour golf tournament players. The golf course is displayed in 3D and you can ask for statistics of any tournament player in a specific hole. You can display the trajectories of individual shots and compare the various players. Who has never wanted to analyze the performance of the PGA tour participants? All joking aside it seems a very powerful tool for industry professionals. But I do not really know how much the use of HoloLens can be in this case an added value.

Noteworthy is the use of HoloLens by Cirque du Soleil in the design of the show scenery. People in different parts of the world can work together in front of the same virtual stage. It is as if they are all in the same theater. An application that is the case of saying “spectacular” that can be used for other forms of entertainment as well.

To conclude, it seems obvious that this is a technology at the dawn but with really important potentialities. Its success will depend greatly on the type of applications that will be developed. The current applications are in part fun toys and partly designed to provide the basis for future important applications. We hope that the futures HoloLens applications will revolutionize the world of current technologies.

Let’s see what happens…

Francesco Pallotta is a senior software engineer expert in software design and development. He works in the field of Space and Defense and deals with application development techniques for virtual reality and augmented reality.

Do you want to read more about Virtual Reality, Augmented Reality and Mixed Reality? Follow me on and .

Did you enjoy this post? Recommend it, by clicking the heart ♡ icon.