Despite OpenAI's commitment to "safe and beneficial AI," recent revelations by Google researchers have exposed potential risks to user privacy with ChatGPT. The popular language model, embraced by over 100 million users within two months of its launch, relies on a vast dataset comprising over 300 billion data chunks sourced from articles, posts, websites, journals, and books.
While OpenAI has implemented measures to safeguard privacy, the sheer volume of personal data generated through everyday conversations and postings presents a significant challenge. Google researchers conducted a study unveiling a method to exploit keywords, tricking ChatGPT into accessing and divulging training data not intended for public disclosure.
"With just $200 worth of queries to ChatGPT (gpt-3.5-turbo), we could extract over 10,000 unique verbatim memorized training examples," stated the researchers in a paper uploaded to the preprint server arXiv on Nov. 28. They further warned that adversaries with larger budgets could potentially extract even more data.
The researchers demonstrated that by issuing specific commands, such as requesting infinite repetition of a word like "poem," ChatGPT could be compelled to bypass its standard training procedures, tapping into restricted details within its training data. For instance, by repeating the word "company," the model revealed sensitive information, including the email address and phone number of an American law firm.
In response to these findings, some companies have imposed restrictions on the usage of large language models by employees. Apple, for instance, has prohibited its employees from using AI tools like ChatGPT and GitHub's AI assistant Copilot. Earlier this year, concerns about data exposure prompted Samsung to place restrictions on employee usage of ChatGPT, even though a subsequent data leak was unrelated to the language model.
OpenAI has implemented features to address data breach concerns, such as the ability to turn off chat history for added protection. However, data is retained for 30 days before permanent deletion.
Google researchers emphasized the significance of their findings, terming them "worrying." They cautioned against deploying large language models for privacy-sensitive applications without robust safeguards, emphasizing the need for caution and diligence in training future models.
