A privacy-first chatbot that’s anything but secretive

• Amazon chatbot released at the beginning of December – a year after some competitors. 
• The chatbot has run into problems, and could leak confidential data. 
• It’s also reported to be hallucinating significantly.

Amazon recently launched an AI chatbot called Amazon Q. As has been the case with almost every chatbot launch in the year of undue haste that was 2023, problems have been identified very quickly after its launch. In this instance, the Amazon chatbot is “experiencing severe hallucinations and leaking confidential data.”

That’s right – far from being a weapons and gadgets expert, Amazon Q is a spy on LSD.

The revelation came, not – as if often the case with malfunctioning chatbots – from internet trolls posting their getarounds, but from leaked internal documents (andgiven its habit of radical indiscretion, you have to wonder whether Amazon Q leaked them itself!). Posted to Platformer, they say that the leaked data includes the location of Amazon Web Services (AWS) data centers around the world.

An Amazon spokesperson told us, contrary to the report, “Amazon Q has not leaked confidential information.”

Amazon’s chatbot also allegedly revealed internal discount programs and unreleased features. The incident was marked as “sev 2,” enough of an issue to warrant paging engineers at night and making them work through the weekend. Observers could be forgiven for acknowledging that Amazon warehouse working patterns, finally enforced for the higher ups, is how you know the problem was significant.

“Some employees are sharing feedback through internal channels and ticketing systems, which is standard practice at Amazon,” a spokesperson said. “No security issue was identified as a result of that feedback. We appreciate all of the feedback we’ve already received and will continue to tune Q as it transitions from being a product in preview to being generally available.”

It’s worth remembering that Amazon pitched Q as a more security- and privacy- focused alternative to other generative AI chatbots.

Adam Selipsky, CEO of Amazon Web Services, told the New York Times that companies “had banned these AI assistants from the enterprise because of the security and privacy concerns.” In response, the Times reported, “Amazon built Q to be more secure and private than a consumer chatbot.”

Whether these features were meant to include the security and privacy of AWS is another story. Internal documents claim Q has given misleading answers on digital sovereignty and other issues.

““Expect the Q team to be very, very busy for a while,” one employee said in a Slack channel seen by Platformer. “I’ve also seen apparent Q hallucinations I’d expect to potentially induce cardiac incidents in Legal.”

Amazon is very cagey about the locations of its vast data center footprint, which is made up of its own builds and wholesale leases. As of December 2022, AWS revealed it owns 15.4 million square feet of data center space around the world, and leases another 18 million.

A few years ago, WikiLeaks published internal Amazon documents showing the locations of its data centers as of 2015. A 20-page document included a map tagged with locations for easier viewing.

Is the Amazon chatbot running late?

The issues with the Amazon chatbot come at a time when the company is fighting against the perception that Microsoft and Google have beaten it in the AI arms race. After announcing it would spend $4 billion on AI startup Anthropic, Amazon revealed Q at its annual Amazon Web Services developer conference.

As these things go, chatbots are arguably fairly passe by now. Microsoft’s involvement in ChatGPT, which began making headlines just over a year ago, means Amazon might be thought of as falling (way) behind. To successfully launch an initial chatbot in December, 2023, potential business customers need a cogent answer to the wuestion of what it does over and above the chatbots that have been out there, developing and overcoming their data nightmare teething troubles for a year.

The chatbot offering from Amazon has been presented as an enterprise-software version of ChatGPT. Initially, it would answer questions from developers about AWS, edit source code and cite sources. Yes, it’s competing with Microsoft and Google, but it’s priced lower. ChatGPT Prime, if you like.

Another selling point would have been its increased security – but the information leaks reported internally make that significantly harder to sell.

Still, the risks presented by Q, outlined in the document, are actually ypical of LLMs, all of which return incorrect or inappropriate responses some of the time. So, it’s not as though Amazon’s chatbot is worse than the others – it’s just that its at least as notably deranged, in a very public way, from a company that’s a year late to the party, and that marketed it on the basis on increased security. The equation of saleability there is hard to solve without significant reparenting of the model.

And, of course, there’s that inconvenient detail that its responses threaten to spill one of the best kept secrets in the data center sector: where AWS is hidden.

What happens next is anyone’s guess – but there will clearly be some extensive overtime worked in what we like to think of as Q Branch at Amazon, trying to get the privacy-centered chatbot to shut up.