Real Art

Emotional Tech

Emotion-tracking algorithms give technology the ability to understand our inner lives.

Joe’s having a bad morning; he wakes up late, stubs his toe, and burns his tongue while rushing to finish his coffee. He heads out the door frustrated and anxious. But after hopping into his car, Joe’s day begins to turn around. His favorite song pumps through the speakers, the air temperature automatically lowers, and the dashboard reminds him not to worry—it’s found a route that will get him to work on time.

This is just one subtle scenario illustrating the future of technological interaction: emotional-responsiveness. Not long from now, our devices and environments will understand how we’re feeling and react in powerful ways. Already, emotion-tracking company Affectiva has developed sophisticated algorithms to analyze facial expressions in order to identify and log human emotion. But this is only the infancy of emotion-enabled technology. Soon, it will transform the ways we live and work, permeating every facet of our digital lives.

At first, the technology might be as simple as an app prompting you to take a picture when sharing a laugh with friends. We’ll use smart devices to track our emotions as closely as Fitbit users track their steps, examining graphs of our emotional timelines to get granular insight into what really makes us happy. A user might notice that coffee-fueled mornings leave them feeling blue in the afternoons and decide to cut back on the caffeine. Or that waking up later has the surprising effect of raising their base mood for the rest of the day.

It might mean clothing that alters its shape and color in order to broadcast your mood to others. Or a smartphone application that sends you adorable puppy videos until your anxiety level falls within a healthier range. It could mean a streaming music service that tailors its tunes to how you’re feeling or want to feel, a dating site that matches couples based on compatible emotion profiles, an e-reader that tracks your emotional response alongside your page count, showing you how it compares to the thrills and yawns of others readers.

As more data is compiled and incorporated, the venues for emotion-reading technology will grow. Beyond facial expression, emotions will be measured through heart rate, skin conductivity, brain chemistry, behavioral patterns, and more. Every device, application, and environment will become an input for emotion and an output for emotional responsiveness. Every kind of experience will be enriched with what Affectiva calls the “emotion data layer.”

Restaurant and movie reviews will be accompanied by emotion graphs showing the average rise and fall of feelings throughout the dining experience. The facades of theaters and concert halls will light up with vibrant visualizations of the real-time emotion levels of the audiences within. Your corner cafe will be able to know immediately if you aren’t happy with your latte and send you a notification offering the next one on the house. Cars won’t let people drive when they’re overly upset. Doctors will receive notifications when a depressive patient’s emotional state crosses a worrying threshold.

The entire movement will be powered by what Microsoft CEO Satya Nadella has referred to as “ambient intelligence.” That is, the automated ability of computers to convert massive amounts of data into responses that feed back into human culture. Incorporating emotion information into Big Data will provide unprecedented insight into emotions all over the world. Researchers will have a field day correlating the emotional reactions of sets of individuals with international events, weather patterns, geography, and more.

Emotional expression is the first form of human communication. It’s a near instant way of telegraphing meaning to another person. Think of an infant’s cries or the instant power of a seductive smile. We’ve evolved to slightly mirror the emotions of the people around us and this mirroring creates empathy. Emotions move from person to person like a telephone message transmitted along the wires.

Our first interfaces with computers were through keyboards and punch cards. Now we’re using voice recognition and gesture interfaces that can better respond to the user’s needs and wants. The real revolutionary potential that emotionally responsive technology holds is the ability to instantly know how we feel and then deliver on it. Without typing, voice-to-text, clicks, taps, or gestures.

Emotional expression is one of the most powerful ways we communicate with each other. Soon it will be the most powerful way we communicate with the world around us.