OpenAI is reading ChatGPT chats, humans look at what people say to AI

OpenAI is reportedly using hundreds of contract workers to read and review real ChatGPT conversations to improve its AI models. According to a latest investigation by 404 Media, based on leaked internal documents, Slack channels, instruction guides and user prompts, the company is using these workers to evaluate ChatGPT responses and help train and improve its AI systems.

According to the report, the review of ChatGPT conversations is part of OpenAI’s effort to improve its AI models. The work is being carried out under the codename Project Lily, with reviewers sometimes seeing entire exchanges between AI and ChatGPT users that can include sensitive personal information. The review process is said to be separate from publicly disclosed safety checks, such as chats flagged for threats of harm. Instead, contractors are asked to assess how well ChatGPT responds to prompts, summarise what the user is asking for, and rate multiple model outputs.

One worker told investigators that they did not think most users knew that humans may be reading their chats, while some prompts reportedly showed users asking ChatGPT to keep the contents private. OpenAI contractors reportedly use a dashboard that first shows a real user prompt. Reviewers then write a brief summary of the user’s intent and assess four ChatGPT responses, marking which parts are aligned or misaligned with the instructions for the model being trained.

Reviewers are reportedly required to highlight at least three parts of a response and explain their reasoning. They then score each response from one to seven, ranging from “unacceptable, unusable” to a response that “would be hard to meaningfully improve.”

Documents reviewed by 404 Media say the reviewers are helping train ChatGPT to sound less human and be less likely to blindly agree with users, what we call sycophancy. They check whether responses are clear, natural and appropriately warm, while avoiding things like excessive emojis, “AI-speak”, forced attempts to copy a user’s style and claims of having real-life experiences. The model is also expected to match a user’s tone without suggesting that it is a human or has emotions.

The report highlights that while usernames are not shown to reviewers, the chats can still contain personal or sensitive information. In some cases, reviewers can also see a “user memories summary” showing what a person has previously used ChatGPT for and, at times, details such as where they may live. OpenAI says chats pass through a Privacy Filter that is designed to remove personal information before reviewers see them, but the company admits the filter can sometimes miss sensitive details.How ChatGPT users can stop OpenAI from reading chats

When the investigators asked OpenAI where they inform users that humans may review their chats to improve its AI models, the company did not initially provide an answer. OpenAI later pointed to a section on its website that says humans may review some content. The company also said users can stop their chats from being used to improve its models by turning off the “Improve the model for everyone” setting. This setting is on by default for Free, Plus and Pro users and applies only to new conversations, while it is off by default for Enterprise, Business and Edu accounts.

When India Today checked the settings, the “Improve the model for everyone” option appeared to be turned on by default in both App and Website. To turn it off, go to ChatGPT Settings > Data Controls and switch off the toggle.All AI companies read chats of users

And notably, OpenAI is not the only company using human reviewers to check AI chats. According to the report, Anthropic also confirmed that it uses human reviewers to improve Claude, while Google’s Gemini warns users that some saved chats may be reviewed by humans.- EndsPublished By: Divya BhatiPublished On: Sep 15, 2026 10:35 ISTAlso Read | Not just OpenAI and Anthropic, even Zoho wooing developers with AI toolsAlso Read | Government issues alert for Android users, asks them to update phones ASAPAlso Read | Samsung launches Galaxy S26 FE in India, says it brings flagship features to more people