Sony
Sony's Any Surface UI is an innovative user interface that revolutionizes the way we interact with IoT devices. It achieves this by creating an intuitive, dynamic, and personalized experience, effectively transforming our relationship with technology.
The process of translating that vision into an actual UI design was a complicated effort, though. We needed to make sure that the focus on simplicity didn’t result in a cold, dull feel—the quality of the interface depended on interactivity that would give the design a kind of warmth, nestling the interface into the user space as harmoniously as possible.
“A musician aims to play sounds that transcend silence, while a painter strives to paint beauty that transcends a white canvas.”
Transcending the state of nothingness—that’s what art is. In the creative process, the moment that defines a work often comes at the initial encounter between the artist and the medium: The first measure of a composition or the first stroke of a painting often determines whether the work will succeed. At Sony, hardware design operates on a similar basis—the first basic line sets the course for the entire design.
Searching for the best way to make that first, crucial mark, the UI design team eventually made a breakthrough that would give real shape to the Any Surface concept.
We decided to use the most basic, fundamental element of all: the single point.
A point, having no height or width, represents zero-dimensional space. In our day-to-day lives, points often signify locations, serve as markers, or indicate origins. Points have a deeper power, too: Like the first drop of rain hitting the ground or the center of a ripple spreading across the surface of a still pond, points also have a more symbolic, suggestive connotation hinting at something more to come.
A Paradigm Shift
Given all the shifts reshaping our social realities and day-to-day lifestyles, we have been inhabiting a world where the IoT—a networked web of smartphone-controlled electronics ranging from audio devices and projectors to lighting, door locks, and other home appliances—will form our spheres of daily existence. Developments in artificial intelligence (AI) technology, meanwhile, are quickly making devices more responsive and convenient than ever: Instead of relying on manual user input, devices are now able to perform a multitude of tasks automatically at the sound of a voice command.
Most home electronics obviously need to be within a certain size range, have access to power, and accommodate the other demands of regular domestic use. What it comes down to, then, is how products can blend simplicity, communication, and user-friendliness. As those needs started to come into view, Sony knew that it had to push its UI to the next level.
The goal was to create a UI that would let users focus exclusively on what they wanted to do and know without having to give any thought to the devices or apps they’re using. The ideal UI would play a non-intrusive role, only showing up when there was a clear need; it would do its job and then get quickly out of the way. To develop an “Any Surface” UI language capable of rendering a viable interface out of whatever the user was looking at, we set four core goals.
Keep high visibility in environments covering various ranges of information
Make the UI scalable to IoT devices that operate under capacity and power-consumption restrictions
Establish a uniform process for ensuring natural conversation-based agent operations
Enhance usability through smooth, logical screen transitions
We decided to build the Any Surface interactions around the idea of a “point”—a dot—appearing in empty space, a concept that would reflect the intelligence of the UI’s dialogical capabilities.
When the dot in the Any Surface UI moves, the interface gains the visual components of lines and motion. The lines define the width of the display area, preparing the user for incoming information, and then immediately disappear to foreground the target information. The motion of the point, meanwhile, not only lends the interface a dynamic dimension but also shapes the Any Surface’s distinctive personality. To give that visual character an organic, welcoming identity, the team meticulously designed the motion to exude visual warmth and evoke a sense of comfort.
For the final stage of the information-display process in the Any Surface UI, we decided to use a “surface” concept. The rectangular surface visualization chimes with the gentle, intelligent personality of the entire UI, as well, featuring rounded corners during the transition to highlight the information within its borders and imply convergence.
If two people are just standing together and talking at the same time, they’re not having a dialogue; there’s no back and forth, no real communication. That basic understanding—the need for both talking and listening—was vital in shaping the Any Surface framework, which includes numerous features that help the agent listen to the user and encourage the user to wait for responses.
First of all, the interface uses visual cues to indicate what the agent is doing. When the dot is on a small, stationary circle, the user knows that the agent is in standby mode and waiting for voice input. When the display shows larger, pulsating circles around the dot, the interface is telling the user to wait while the agent is thinking.
Some devices also feature illuminations and tactile feedback. To integrate those elements into the interface, the design team worked to make sure that the timing and intensity of any additional signals would seem as real and natural as possible during user-agent interactions.
The market is actually home to many products that do searches based on AI-based voice commands—but their interactive styles are still a long way away from being as natural as two people having a conversation. A lot of that awkwardness comes from how the AI handles situations where it misunderstands something or can’t make out what the user is saying. In those cases, the products tend to switch immediately into overly mechanical, impersonal responses or rely so much on actual audio input that they fall into annoyingly redundant patterns.
Unnatural interactions were the last thing we wanted. To make the interface as smooth and intuitive as possible, we decided to bring audio together with text and graphics for an optimal, communicative blend of interactive elements.
It’s no rare occurrence to mishear or misunderstand the person you’re talking to. The same goes for the Any Surface interface; there’s no guaranteed way to get around it. When a human-to-human dialogue runs into that kind of problem, the speakers often turn to text and pictures to streamline the communication experience. Any Surface follows that same, natural pattern. The UI also presents choices to users in a text-based format and opts for graphics—maps, graphs, gauges, and more—when a visual representation would be more effective than voice-based communication. Altogether, the dialogue system is a true Sony original with unprecedented usability.
Another feature is the “modeless” process flow, which enables users to combine voice commands with tactile operations at any time and even delegate remaining operations to the agent if the need arises. Giving the user as much freedom as possible, the interface embraces a thoroughly user-centric design.
For adults who’ve grown accustomed to the traditional app style, the Any Surface might take a little getting used to. Children, however, don’t have any preconceptions about how interfaces are supposed to work. For them, the UI will be a natural fit—and future generations will find it hard to believe that there was ever something other than the approach that Any Surface epitomizes.
The simplicity and flexibility of the Any Surface experience makes for a versatile UI language capable of flourishing in any application and any setting: the devices people use, the spaces people inhabit, and even next-generation products that currently lie beyond our imaginations.
We’re getting closer and closer to a new technological landscape, a world where whatever we need or want is just a simple flick of the finger or voice command away. Any Surface is getting a running start on making that vision a reality.
As we tweak the interface in accordance with changes in product identities and applications, Any Surface will gradually roll out as the new UI layer for a wide variety of Sony products. The entire Any Surface experience converges on a single point. The future of UI design lies right there in that elegantly simple, zero-dimensional concept.