When seeking answers from ChatGPT, there is a potential pitfall — the model may provide entirely fictional "facts" that, while sounding credible, are fabricated. A recent incident involving a New York lawyer highlighted this susceptibility, as ChatGPT generated imaginary judicial opinions and legal citations during court proceedings, much to the dissatisfaction of the presiding judge.
Despite being trained on extensive factual data, large language models (LLMs) such as ChatGPT are susceptible to generating false information, commonly referred to as hallucinations. This can occur when LLMs are tasked with generating text on less familiar topics or when they inadvertently combine information from various sources.
To address this challenge, a team of researchers, including doctoral candidates Orion Weller and Nathaniel Weir, developed a method inspired by journalistic practices. The researchers conducted a study to assess the impact of incorporating the phrase "according to" in LLM queries. Their findings revealed that prompts using "according to" successfully guided language models to anchor their responses in previously observed text. Rather than producing hallucinated responses, the models were more inclined to directly quote the specified source, akin to journalistic quoting.
"Language models excel at interpreting syntactic and semantic cues," explains Weller. "Given that 'according to' is commonly used online when quoting sources in news articles, an LLM may interpret the prompt as a hint to search for quotations from its training data."
Using Data Portraits, a tool developed by the researchers, the team verified whether an LLM's responses could be traced back to its original training data. Referred to as the "QUIP-Score" (quoted information precision), this metric increased by 5% to 15% when grounding prompts, such as "According to Wikipedia...," were employed. The team found that incorporating a grounding prompt enhanced the model's ability to quote text, leading to more detailed and accurate responses.
The researchers aim to enhance knowledge grounding by encouraging LLMs to quote directly from reliable resources encountered during training. This strategy aims to ensure that models access information from high-quality or trusted documents, improving the accuracy of their responses.
While reminiscent of virtual assistant functionality, the key distinction lies in the Hopkins team's implementation. Their LLM operates without internet access, relying solely on its internal knowledge distribution learned from previously observed sentences, eliminating external data from live searches.
The "according to" prompting technique proves effective across various LLMs, with optimal results achieved in conjunction with larger models and instruction tuning. The latter involves training the model with explicit instructions, such as "Answer the question with the correct answer," in addition to typical question-answer pairs.
It's crucial to note that the generated text's presence in sources like Wikipedia doesn't automatically validate its correctness with respect to the given question, cautions Weller. The model's accuracy still hinges on the quality of its training data, prompting the team to incorporate measures to filter out information from unreliable sources.
"We demonstrate the ability to explicitly instruct ChatGPT not to cite specific sources, providing further evidence of its understanding of grounding instructions," notes Weir.
While acknowledging that their method isn't a complete solution, Benjamin Van Durme emphasizes its contribution as a step towards helping LLMs generate more factual and accurate information by leveraging their training data effectively.
More: https://techxplore.com/news/2023-08-fake-facts-words-technique-ground.html
