As Ux engineer, I love watching science-fiction movies to imagine what could be the user experience in the future. In these kinds of movies, technologies seem to be very easy to use. Tasks that should take an eternity in real life are so quickly achieved in science-fiction thanks to efficient human-machine interfaces. You need specific information? No problem, just ask to your personal AI. You need to find a needle in a haystack? Easy, use your enhanced vision.
You might say it is just a scenario simplification in order to focus on what is significant for the story. And that’s right! However, I also think that by improving the communication between human beings and the computer, we will be very more efficient.
A lot of technologies are already powerful but due to a lack of ergonomy, they lose most of their efficiency. By improving the user experience, we will boost their impact on users.
Flat screens are no more relevant
Until now, most of the human-computer interfaces were composed of screens, keyboards, mouses, touchpad and/or touch screens. This configuration is very efficient to interact in a one-to-one with a computer.
However, whatever you use a computer, tablet, or smartphone, it is no more relevant when you do something else at the same time as:
- driving a car
- talk to your friend
- discover a beautiful landscape
- eat a delicious meal
In all these situations, you have to choose between your current action and your computer or your smartphone. Most of the time, you alternate between the two of them. And it is too bad because you lose the efficiency and the authenticity of the situation — even with a smartphone. Flat screens might be enough in the past but we need to find something better to really start valuing every moment of life.
From Science-Fiction to Reality
I love user interface technologies and I always keep myself up to date. I think we are now living an important moment because a lot of technologies in science-fiction movies are becoming reality. Technologies previously imaginable only in science-fiction are now ready to use. They bring so many new opportunities that I absolutely wanted to present them to you.
First of all, I have chosen 10 technologies that I have already seen once in a science fiction movie or series. Besides, I have selected only technologies that may be totally popularized for the general public in the next years thanks to recent scientific breakthroughs or fashion trends.
Notice that I study these interfaces from a Ux engineer point of view. It means that I grouped together technologies that bring same user experience event if the manufacturer or the technologies behind is different.
These technologies will be ordered according to their distance to the market of the technology. The first ones are already available for everyone whereas the last ones are at the research stage.
Using simple hand movements to control your house is an old dream. Clap your hands to switch on the light. Swipe to the right to change the channel on the television. It is very useful to avoid remote controller. Using 3D gestures may be disturbing when you don’t know them but it is very efficient to control a building for example. Moreover, your gestures could be very discrete and not intrusive during a conversation with your friends in contrast of using vocal commands — that we will explain later.
The first science-fiction movie introducing gestures is surely Minority Report.
In 2002, it was considered futuristic. Now a lot of devices capture motion in 3D like Leap Motion or Kinect. However, the captured area is currently not very large and doesn’t allow us to detect gestures in an entire building with enough accuracy. But it will definitively come in the future to control at least some tasks in buildings.
This is the typical user interface technology directly coming from science-fiction. In Blade Runner Rick Deckard (Harrison Ford) uses voice orders to manipulate a device.
First of all, voice commands are very useful when you don’t know how to use an object. You don’t know which button to use? No problem, use your voice and the program will do the rest. A second use case is when you cannot physically use a remote controller. It is for example the case in 2001: A space odyssey when users are wearing suits in the space and can only use their voice to interact.
10 years ago, voice commands were totally inexistant and we couldn’t imagine it would expand at this speed. After a furious war between Microsoft, Google, Apple, and Amazon, we have now several voice assistants with Cortana, Google Now, Siri, and Alexa.
This is available right now in all modern smartphones or connected speakers. It is very useful for multitasking, when you are driving for example. Even if they are called smart assistants, real complex conversations are quite hard with these things whereas simple independent questions or orders are very accurate. It is why I classify them as voice commands systems. But they do the job and I think we will soon be able to use these voice systems for all actions even in professional situations.
I surf on Google Now and Cie to talk about artificial assistants. Even if Google Now, Cortona, Siri, and Alexa are still not ready to be real assistants, it is their final goal and they already have several features to assist their users. They can remind agenda events or purchase things for you. Virtual assistants differ from voice commands by being able to follow the user in his daily life in order to understand the context. Moreover, assistants are not limited to speech interaction. It might have physical or writing capabilities to interact with human beings.
A good example of a virtual assistant in movies is Jarvis in Iron Man. Jarvis can handle a conversation as any human would. He can also understand the context and assist Tony Stark (Robert Downey Jr) in all his actions.
Artificial assistants appear regularly in science-fiction because it is the Holy Grail of efficiency. Thanks to an assistant, we will multiply our abilities by ten without any problem. Each human being is to be the supervisor while the virtual assistant will operate low-level tasks. Virtual assistants could also be very efficient for very specific tasks just as administrative paperwork. Virtual assistants like Google Now have some way to go before they can be as efficient as Jarvis but they are progressing step by step.
In this technology, I regroup face, shape, and eye recognitions. The benefit of this technology is to be automatically identified. The real achievement of this user-machine interface is the identification at medium range — when you approach a vehicle or enter in a building. It would be totally transparent for the user (he wouldn’t need to put his face in a device) and secure access in the same time.
Technically the face recognition is far easier than iris recognition at medium range but I put them together because the goal is exactly the same: identify someone thanks to a signature of body parts.
One of the best examples of body recognition in movies is definitively Minority report.
In the movies, all places scan the iris of their visitors in order to identify them. Door locks are no more necessary. People may simply walk, doors will open or stay closed according to the person's authorization. The movement is so much more fluid and easier.
Minority report describes also potential abuses like intrusive targeted advertising. It is a problem we should really consider. Limits about this technology in public places might be useful.
Facial/Iris recognition is already operational for several years now.
However, it only became popularized in daily life with the release of the iPhone X and the FaceID in 2018. This new smartphone doesn’t have any physical home button, so Apple has replaced the TouchID using fingerprints by the FaceID using … the face. However, the detection is only available at short distance — which may be understandable for iPhone, and seems to be not mature with some false-positive and false-negative unlockings.
With this technology, no more keys are needed. You just walk and the doors open according to your authorizations. It may no be mature yet but no doubt it will be expanding in the next years.
Virtual reality is a user interface based on a virtual world where the user can move, interact like in the real world. The main benefit of this technology in comparison of 2D screen is immersion. By using the classic head movements for the view and arms movements for objects manipulations which are more instinctive, you are more efficient.
It was quite unknown several years ago, it is now popular in the world of video gaming. These technologies currently work with an adapted headset like Oculus, HTC Vive, PlayStation VR, etc. It is also available with motion detectors to detect body movements.
For now, virtual reality is cool for gaming but keeps real inconvenient like the screen quality and the latency. Things are improving it is still tiring for long use. Moreover, controllers still lack of accuracy for professional uses.
Whatever be the case, the current craze about virtual reality will certainly accelerate the development and open new opportunities. In the future, we could discuss business with people in virtual rooms instead of using air planes and lose hours of work.
My favorite movie showing the virtual reality in action is Matrix. It shows the maximum achievement of this technology — may be too much in fact.
The user would interact easily with the virtuality and could as efficient as in reality but with several extra powers — otherwise, reality would be enough. Why not training for astronauts or piloting drones? Virtual reality will definitively continue to expand in the next years to create magic immersive experiences.
Augmented Reality and Mixed reality
This time I cheat because I give you two user interfaces in the same time. These two technologies are very close from a technical point of view even if the use case is slightly different. Augmented reality projects data on the user field of view about what he is currently looking at. A good example is Iron Man HUD.
The goal is to enhance the capabilities of the user by giving him more information about his environment.
Mixed reality is a mix between augmented reality and virtual reality. It projects virtual images on reality like advertising. A recent movie showing mixed reality in action is Blade Runner 2049.
Blade Runner 2049 is a great example since it explores the case of a mixed reality almost impossible to differentiate from the reality with the AI companion called JOI.
Augmented reality and mixed reality are also becoming reality with Google glasses, Hololens or Intel’s Vaunt.
These technologies are really buzzing right now. Apple and Google are fighting to create the best AR kit for smartphone.
This technology is mostly used to control arm prosthesis. This interface is plugged into the flesh body and detects the nervous signals to reproduce the movement of a missing limb.
The first movie I saw where we could see this technology is obviously Star Wars The empire strikes back where Luke gets a synthetic arm after his fight with Dark Vader.
It happens a lot of time ago, in a galaxy far far away but it also happens now. We are now able to build this kind of robotic arm.
These innovations will change the day life of thousands of people all around the world. It is currently at the research stage but slower or later it will be available on the market.
This human-machine interface is quite similar to nervous control but has far more potential. The previous innovation detects the electric signals between existing nerves in limbs to understand what movement the user would like to do. Whereas the neuronal control is directly listening to the brain. So you can create custom signals with your thoughts which are limitless. And this now exists:
A good example of control using the brain is Avatar where Jake Sully (Sam Worthington) controls an alien body using his brain.
A robot is more than a simple user interface since it is a platform capable of physical movements. Since human beings show more empathy for animated beings, the interaction between humans and robots could be deeper than with inanimate objects. And if the robot can reproduce facial expressions, it is even better to share information with users.
In this way, it might be more efficient to use a robot than a simple virtual assistant.
The very recent series Altered Carbon on Netflix is a good example. In this series, artificial intelligence has a building as its body. The AI owns and manages a hotel. However, in order to better interact with the clients, it constructs a humanoid body on the fly.
Imagine a robot moving and speaking like a human being. It would be the perfect human-machine interface. Users wouldn’t be lost because they could interact with it exactly as for another human.
This kind of robot is available only in science-fiction. However, companies like Boston Dynamics are hard-working to create robots.
These technologies are the future of Human-Computer interfaces. They bring so many new and cool opportunities. Great challenges are still waiting to be solved. Most of these challenges are very technical and need specialists in mathematic modeling. However, since these technologies will totally transform interactions between machines and humans, we need to work on their ergonomy. I noticed that ergonomists/Ux designers/Ux developers are almost never involved at the research level. And I think it is a shame for so innovative user interfaces.
That is why, as a Ux developer, I have decided to focus my attention on these new interfaces for all my new side-projects. Because I want to be part of the future. And you do you want to be part of the future?