Google’s Robot Reservationist Is Unnervingly Good

Google’s Assistant can already order Starbucks, estimate restaurant wait times, and make restaurant reservations online — and soon, its newest technology, called Duplex, will be able to automatically make reservations by phone using a human-sounding voice. The search engine giant excitedly announced the new feature in May, and spent the past month refining it in preparation for a limited launch later this summer. TL;DR: It works convincingly well.

Here’s how it works: A user asks Google Assistant to make a reservation at a restaurant on a specific day. The Assistant follows up with questions to confirm relevant information: How many people, for what time, and if that time is unavailable, is there a window of time they can work within? (If the restaurant uses OpenTable, the Assistant will automatically make the reservation that way, and confirm via a notification.) Otherwise, it will dial the call: “Hi, I’m calling to make a reservation. I’m Google’s automated booking service, so I’ll record the call. Can I book a table for Thursday?”

Nick Fox, Google VP of product and design, explained at a recent demo that the goal of these opening lines is to: set a clear purpose for the call, announce the source of the call, and, in order to follow various federal and state laws, disclose that the call is being recorded.

Based on initial reactions to Duplex — “terrifying,” “worrisome,” “deceptive” — Google is now approaching its new offering with caution and optimism rather than pure curiosity. “A lot of technology can be used for good purposes,” Fox says. “Some can be used for nefarious purposes... we think it’s important to be up front about [our goals].”

Google’s Assistant sounds strikingly human, with speech patterns that include disfluencies, or the ums and uhs uttered in normal conversation. Though its eerily human-sounding voice scares a lot of people, it’s convincingly human-sounding by design. “People thought we were trying to trick them,” Scott Huffman, VP Engineering, says, “but we were studying the pitch, speed, pace of human voice” so that the actual human on the other end would “get the message.” Early tests of Duplex failed in their missions, because when the voice on the user’s end sounded like a robot, a lot of businesses hung up. Huffman says that no matter how many different ways they tried it [with the robot voice], the “end result is that it didn’t work.”

Huffman clarifies that Duplex is a learning program, and learns each time it’s used — engineers are listening to calls in order to help make tweaks — but that it has barriers. It can’t strike up a conversation, or answer off-message questions such as, “What’s the weather like?” It sticks to its script: When the restaurant’s employee asks for relevant information, such as party size, reservation time, name for the reservation, and contact information for confirmation, the Assistant uses the information it has (and is allowed to share) to complete the reservation.

In the event that it cannot understand the human on the other end of the phone, is unable to answer a question it’s asked, or otherwise gets stuck, it backs out of the conversation by saying something like, “I’m sorry, I didn’t catch that. I’ll have someone call you back.” In those cases, a live human operator employed by Google will call the restaurant back to complete the reservation. Huffman says that four out of five calls can be made entirely by Duplex, needing no human interaction, but for the lingering few, operators are standing by. “There will always be things that will be odd or different,” he says, “and having operators just makes the experience better.”

The public launch will happen in two phases: First, Duplex will begin helping users call businesses to confirm holiday or special hours of operation. After a feedback period, it will roll out the ability to make appointments at hair salons and restaurant reservations. Fox wouldn’t specify how many users would be part of the initial test, only saying it would be “a limited set of businesses and a limited set of users,” and noting that all parties would be privy to what was happening. Even after the initial launch, users will not be able to ask Duplex to call any restaurant, only restaurants that opt-in to receive calls from Google Assistant.

Listening to the calls Duplex makes, it’s easy to imagine a world of bots talking to bots. Fox says Google isn’t yet working on a business-side version of Duplex that would answer user calls. “There are ways in which we can imagine extending the technology...” he says. “We’re not doing those yet. We’re focusing on these specific cases... we’ll see where we go from there.”

For businesses concerned about no-shows, the technology offers several opportunities for the user to cancel. Fox also outlined how much positive feedback the team received from the accessibility community, and noted that language barriers are something it is actively working on. But this summer’s launch is still a test run, and there’s no indication of how many users will eventually be invited to use Duplex, or if it will remain free. “Our intent is to make all of our technology available broadly,” Fox says, noting that it could be a service Google might charge for one day. “We’re too early to say how or when that will happen.”

According to Fox, the businesses they’ve approached as initial partners are “excited” to test it out, and “hopeful about it... they’re hopeful that it generates new business.”