UPDATE: March 17, 5:55 p.m.: Square contacted Eater to confirm that the company first posted an official response 19 minutes after the outage began, not an hour, as previously reported.
Nate Snell, owner of Pip’s Original Doughnuts and Chai in Portland, was inventory shopping Thursday morning when he got a strange text from one of his employees. The system on which the store’s entire checkout process was based had logged the workers out of the backend and would not reconnect. Staff could not process any payments. On a typical day, the line at Pip’s — which sells miniature doughnuts, fried to order, and a variety of chai teas — is out the door. But the ordering and checkout flow was interrupted by the system at the center of the entire process — Square, the mobile credit card scanner and point of sale software, which functions as a business’s de facto cash register.
A Square service outage yesterday lasting roughly two hours forced restaurants, coffee shops, and food carts around the country to turn away customers and lose sales, bringing into question whether relying solely on new technology and software to make business transactions is a good idea.
At Pip’s, where Square handles everything from credit card transactions to printing receipts, Snell and employees had to quickly re-think their checkout strategy, writing and processing all orders by hand. “It was a challenge, for sure,” Snell says. “Some of the customers were pretty frustrated obviously.” Patrons who didn’t have cash were turned away with Pip’s apologies and a voucher for a free order of a dozen doughnuts. In the end, the shop lost about $900 in revenue, not including the 60-70 free vouchers, which were worth $6 a piece.
As complaints started to roll in about Square yesterday, with many business owners venting their frustrations on Twitter and Facebook. Square later posted an official response, confirming they were aware of the problem and were investigating. Fifteen minutes later, Square identified a cause — a system update.
“Today’s outage was due to a change we made to one of our back-end systems that resulted in capacity issues,” a representative told Eater in an email. But the company would not respond to requests for more details, such as how widespread the outage was or how many users were affected, only to say they were working on an analysis, which they plan to make public at an undisclosed time.
In the meantime, some users want compensation for their losses. Snell says he reached out to Square, but hadn’t gotten a response. Square did not confirm with Eater if or how patrons would be compensated.
Any benefits Square doles out to hurt patrons could potentially be costly for the business. Square boasts millions in meals sold from restaurants that use its suite of services (though it declined to disclose the exact number of its customers). That service suite includes, most notably, devices that restaurant owners can insert into smart phones, tablets, and scanners that allow them to accept credit card payments. Last year, Pip’s processed $1.3 million in payments using Square technology, Snell says. Like Pip’s, many restaurants, bars, and cafes structure their entire systems around Square products, sometimes without backups.
Matt Gingles, the owner of the Burrito Shack, a small Mexican restaurant with two locations in Kentucky, pulled out a calculator and a menu and added the afternoon’s sales and taxes for each order by hand. The outage, for him, occurred during the busiest time of the day, the lunch rush. Without Square, he was unable to track any of the transactions he lost, but estimates a $600 hit. He says customers were “irked.”
“They either paid with cash if they had it or turned around and left,” Gingles says. He added that after three and half years of using the service, this is the first incident of this caliber he’s had, and was shocked that something like this could happen at a company like Square. “You should have safeguards against things like that,” Gingles says. “Because when that happens, we lose money. We had no way to track sales and my business just kind of collapses.”
The company does offer an “offline mode,” and during the outage, Square Support suggested businesses use it. But that option was useless to people already logged out of the system.
For users who couldn’t use the offline mode to at least track sales, they had figure out another way to collect payment, or turn customers away. That was the biggest lesson learned, Snell says. He has emergency plans for greasy spills and fires, but was caught off guard by the technical glitch. “I don’t think we realized that the entire Square system nationwide would go down,” Snell says. “I immediately got on Amazon and ordered an old-fashioned [credit card] swiper.”
“It just teaches you technology doesn’t guarantee reliability. It guarantees convenience,” Snell adds. If this incident has taught restaurant owners anything about technology, it’s that they can’t depend on it too much, especially in an age where even large companies are victims of cyber attacks and hacking. “Call me a pessimist, but I think it’s realistic to assume that these kinds of issues will become more prevalent and not less,” he says.
As for how much this incident will impact his future relationship with the service, Snell says that while he is disappointed with what happened and Square’s response, he thinks the system is still the best one for his business. “But I’m going to be prepared for this next time,” he says. It will become a part of our reality and our training.”
• All Square Coverage [E]