When former governor and current U.S. Rep. Mark Sanford admitted during a press conference that he'd been unfaithful to his wife, his 20-minute apology was panned as a "rambling," "meandering" "train-wreck" that was "almost as messy as the mess he was apologizing for."
It turns out, even computers can tell it was bad: According to Moodies, an app capable of identifying a speaker's mood, attitude and personality from his voice alone, Sanford's mood during his press conference was "authoritative," but "short-tempered," and the tone of his voice betrayed his feelings of "dislike" and "impulsive rejection."
Moodies, released Sunday, was created by Israeli startup Beyond Verbal, a company developing software that can be used to recognize emotion in people's speech. Beyond Verbal describes its technology as an "emotional analytics engine" and aspires to license its algorithms to third-party firms, from app developers to hardware manufacturers, to create a new breed of devices capable of detecting not only what we say, but also how we say it. The first such third-party app is slated for release in the coming weeks.
"Our goal is to introduce emotional understanding into practically everything we do, including human-to-human interactions, driving cars, playing games, improving service, or getting better commercials," said Dan Emodi, Beyond Verbal's vice president of marketing and strategic accounts. "We’re going from our cognitive communication -- what we say -- to our emotional communication, which is our body language and vocal intonation."
Emodi describes how human resources departments could one day use Beyond Verbal's technology to screen candidates. An applicant might be asked to read, out loud, the job description for the position she's applying for, and Beyond Verbal algorithms could rank her enthusiasm for the role. The software might even gauge whether she seems more introverted or extroverted, more curious or more methodological, and whether her personality would be a good match for the company's.
Apps could also use Beyond Verbal's software to evaluate a person's mood and suggest a playlist of songs that suit how he's feeling in the moment, or listen in on someone's voice during a call to track which people leave the speaker in a good mood -- and which leave her feeling drained and demoralized.
Moodies, which Beyond Verbal built to showcase its technology, asks users to speak into their computer's microphone for 20 seconds, then promises to identify their primary and secondary moods. Since the software analyzes vocal cues and intonations, not the content of the speech itself, the startup claims it can analyze 26 different languages.
Beyond Verbal belongs to a growing number of companies that seek to help computers better understand what humans are feeling, a field known as affective computing. Affectiva, for example, uses facial cues and physiological responses to measure how viewers respond to marketing messages, and nViso tracks eye movement and minuscule movements in people's faces to track peoples' emotional response to what they're seeing on a screen. Though Beyond Verbal can't capture body language, Emodi claims devices can more seamlessly track moods through vocal signals, as they don't require careful positioning in front of a camera.
We decided to put Beyond Verbal to the test by having it assess the moods and emotions of unfaithful politicians apologizing publicly -- or making excuses for -- their marital indiscretions. We played 20-second clips from YouTube videos, and let the Moodies app assess the men's feelings.
In nearly every case, Moodies picked up on the fact that the men were under duress. Yet it seems that politicians everywhere should perhaps study Bill Clinton when preparing to apologize for their infidelity: Clinton, who not only recovered from the scandal but was recently rated more popular than ever before, managed to exude "happiness," "warmth," "love" and "satisfaction" in his apology. Other politicians, like Sanford, Anthony Weiner and Eliot Spitzer, were judged by Moodies to be "resentful," antagonistic" and "short-tempered."
At a White House press conference in 1998, Bill Clinton famously denied having sexual relations with "that woman, Ms. Lewinsky."
According to Moodies, he demonstrated a "strong internal conviction" and "desire to convince," along with a "need to fulfill creative or primordial cravings" -- an interesting and quite apt conclusion given the nature of the scandal and Clinton's desperate need at the time for Americans' approval.
When Clinton addressed the nation several months later with prepared remarks -- and an apology -- he was in a better mood, Moodies found: his emotional state was characterized by "happiness" and "harmony."
Moodies captured the speech following the 48-second mark of the video.
New York Representative Anthony Weiner admitted to, and apologized for, sending graphic photos to women on the internet during a strained press conference in 2011. "This was a very dumb thing to do and it was a destructive thing to do, but it wasn't part of any plan to be hurtful to my wife, deceitful to you," Weiner said during the clip we played for Moodies. "It wasn't part of a plan. It was a destructive thing I did that I accept responsibility for." Beyond Verbal, it should be noted, cannot detect lies, but only measure emotion.
Moodies captured the speech following the 6:57 mark of the video.
In his first media interview following his arrest in New York City on rape charges, Dominique Strauss-Kahn addressed the charges and admitted regret for what he saw as not only a moment of weakness, but also as a moral failing. Though Strauss-Kahn's words were contrite, even in French, Moodies picked up on the fact that he was "assertive" and exhibited "great self-confidence" and "possessiveness."
Moodies captured the speech following the 1:15 mark of the video.
Unlike Weiner and Sanford, Moodies listened to Eliot Spitzer delivering prepared remarks, not speaking off the cuff. Though Spitzer seems quite composed during the video, Moodies picked up signs of "emotional outburst" and feelings of "jealousy or hatred." The company notes, however, that its software is still being refined to recognize and judge actual emotional outbursts, such as crying or laughter.
Moodies captured the speech following the 40-second mark of the video.
Moodies delivered the same analysis both times it listened to the same part of Sanford's speech.
Moodies captured the speech following the 7:34 mark of the video.