Smart displays are here. Amazon paved the way with its Echo Show launched back in mid-2017, then Google and Facebook released their own versions, and a slew of third parties like Lenovo and JBL quickly followed suit — will Apple be next?

Each of these smart displays hopes to become the center of our smart homes and tie us into an ecosystem of one of the GAFA companies (Google, Amazon, Facebook, Apple) or — if you happen to reside in China — one of the BAT companies (Baidu, Alibaba, Tencent). They are deceptively simple and easy to use but pack a serious punch in the amount of clever technology they pack under the hood.

Amazon Echo Show

Amazon Echo Show (Source: Amazon)

The smart displays market is growing

We can think of smart displays as the evolution of smart speakers, taking the existing voice assistant interface and AI capabilities and then adding a screen for visual output and sometimes also a camera for visual input. The smart display can now let you do tasks that you’d previously have used your phone or tablet for, such as making a video call or accessing information that’s better displayed as an image or video rather than read out as audio. Voice is great for many things, but sometimes, you need visuals. For example, consider the difference between asking your smart display to “show me how to make a paper airplane” as opposed to asking an audio-only device to “tell me how to make a paper airplane.”

The market potential is huge. The overall smart home market is estimated to top $150 billion by 2023, and although smart displays have gotten off to a slow start — with only a few million sold so far — they’re likely to take a fast-increasing share of that opportunity.

One area that has caused concern for some is privacy. Are consumers happy to have their movements observed and recorded 24/7, with a microphone and camera in their home potentially sending their conversations and whereabouts to a faceless internet giant? Vendors will need to reassure users that they won’t exploit their position in every living room, bedroom, and kitchen. Local processing of data (as opposed to sending it to the cloud) is one way to mitigate these concerns.

Sophisticated video and voice technology

The technology in smart displays is surprisingly advanced for such relatively low-cost devices. For video calls, many of them include specific features to make the call intuitive. For example, Facebook’s Portal has software that can recognize a face and then automatically crop the camera’s image and zoom in on the speaker. Move around the room, and it follows you. This is one area in which sophisticated video processing can create one of those “it just works” moments and make the video chat more enjoyable.

Voice interfaces also deploy some seriously clever technology. We’ve become used to always-on listening, wherein the speaker or display leaps into action when a keyword is spoken. To make this work effectively, the devices use far-field audio detection, which can pick your voice out of background music or audio. Adding AI and neural networks could enable a smart display to identify sounds like a doorbell or barking dog and act accordingly — potentially serving as an alarm system of a smart home.

Upcoming technologies could harness the smart display camera input for facial recognition. This will allow voice commands to be issued without the need for a wakeup word just by looking at the device, thereby making the operation even more seamless.

Smart assistance is becoming pervasive

Over the next few years, smart displays are going to get smarter, cheaper, and more popular in our homes. Smart assistance capabilities will also spread to more devices; for example, our TVs will effectively become big smart displays.

Taking it a step further, another product that might well take off is the personal assistance robot, such as the recently announced Temi. Basically, a personal robot is a smart display on wheels with a few extra motion-related features. Once we’re happy talking to an Echo speaker or display, holding a conversation with a robot is not really much of a leap, and with AI getting better all the time, it’s going to feel more and more natural.