Humans are reading ChatGPT users’ prompts to improve OpenAl’s models, and those chats can include sensitive, personal information, according to leaked internal documents and real prompts seen by 404 Media.
The news presents a major privacy risk for ChatGPT’s users, with people often using ChatGPT as a therapist, professional assistant, or digital friend, and providing it with all sorts of intimate details about their lives. The contractors don’t see ChatGPT usernames, and OpenAl says it tries to remove personal information before prompts reach the reviewers, but the company acknowledged sensitive details can still get through.
The news also dispels the misconception that these models are improving only because of OpenAl’s mass scraping of the internet, the talent of its well-paid engineering and Al teams, or the power of its newer models. An important and overlooked part are the outside contractors paid to read and review ChatGPT responses to real prompts over and over again. Anthropic confirmed to 404 Media it is also using human review to improve its models.
“No,” someone who works with the prompts said when asked if they think ChatGPT users know that humans are reading their chats. “I don’t think they would imagine some contractor somewhere […] is analyzing the conversations.”
Unpaywalled link here - https://archive.ph/98Wr5
First off, the label AI is bad - it’s Artificial, but Intelligent - or smart - it is not.
LLM - Large Language Model + scaffolding. They match patterns. They build an unimaginably complex “context window” of 200,000 tokens and from that they synthesize their responses to prompts of usually a few dozen tokens based on trillions of model weights trained on hundreds of trillions of tokens. Optical character recognition has been doing this to automatically “read” the routing numbers printed on the bottom of checks since the early 1960s, that old OCR just has much much lower dimensions of patterns to match.
The “Mechanical Turk, one step removed” is helping to tune those trillions of model weights by reinforcing when it “gets things right” and flagging when it “gets things wrong.” The LLM has to find this out somehow, that’s what the employees are doing - training it, teaching it right from wrong.
AlphaZero taught itself to play chess, Go, and other relatively simple games nearly 9 years ago now - it mastered those with “best next move to win” pattern analysis / generation all by itself because the rules of those games are crisp, well defined, simple. What makes “a good response” in a conversation is a hugely different animal with orders of magnitude more dimensions and levels of nuance.
After all this training, the next generation model should, statistically, generate “correct” conversational responses more often, according to what the inputs of the employees consider and tell the model is correct and incorrect for a given situation.