Is ChatGPT Private? What Really Happens to Your Chats

By Anamika Dey, editor

What This Article Covers

What Project Lily is, and what 404 Media’s investigation actually found
What OpenAI says publicly versus what the new class-action lawsuit alleges
Whether your chats are being used for training right now, by default
The opt-out setting that doesn’t do what most people assume
How ChatGPT compares to Claude and Gemini on human review
Practical steps to keep sensitive conversations out of anyone else’s hands

Is ChatGPT private? It’s worth asking honestly, because most people use it like the answer is yes — typing out a work conflict they haven’t told their manager about, drafting a message to an ex, asking it to help make sense of a diagnosis before their next appointment. The interface looks and feels like a blank notebook that only you can see. For years, that’s more or less how OpenAI let people think of it.

That assumption took a direct hit this month. Reporting on an internal OpenAI program called Project Lily showed that real human beings — outside contractors, not OpenAI employees — read a stream of actual ChatGPT conversations, some of them containing exactly the kind of personal detail people assume stays private. Days later, ChatGPT users sued. Here’s what’s actually been confirmed, what OpenAI disputes, and what you can do about it right now.

Jump To Section

In the article

What Is Project Lily, and What Did the Investigation Find?

On September 14, 2026, the outlet 404 Media published an investigation into a review program OpenAI runs internally under the codename Project Lily. Citing leaked reviewer instructions, internal Slack messages, and real prompts, the report described hundreds of contractors — hired through outside staffing firms — reading full ChatGPT conversations to help train the model.

The job itself sounds mundane: contractors summarize what a user was trying to do, compare several possible ChatGPT responses, and score them so OpenAI can fine-tune the model’s behavior. Leaked reviewer guides reportedly asked contractors to flag stiff, checkbox-heavy “AI-speak,” catch the model claiming human experiences it doesn’t have, and dock answers that flatter or validate a user too readily — a known problem that has drawn legal scrutiny elsewhere over user safety.

What made the story land wasn’t the mechanics of the review process. It was that reviewers work from entire conversations, not isolated snippets, and that those conversations can include exactly the kind of detail people assume a chatbot keeps to itself.

What OpenAI Says vs. What the Lawsuit Alleges

OpenAI’s position is that reviewer screens don’t show usernames, and that an automated system it calls the Privacy Filter strips identifying details before a conversation reaches a contractor. OpenAI’s own documentation for that filter acknowledges it can miss uncommon identifiers or under-redact when context is limited — and 404 Media reported that the dashboard shown to reviewers can still include a “user memories summary,” a block that can reveal a person’s general location, occupation, or ongoing personal context even with names removed.

On September 16, 2026, the law firm Almeida Law Group filed a proposed class action — Vredenburgh et al. v. OpenAI OpCo, LLC, in the U.S. District Court for the Northern District of California — on behalf of ChatGPT users. The complaint’s central claim isn’t that human review happens; it’s that OpenAI’s Terms of Use, Privacy Policy, and its page describing how data improves model performance describe retention, automated redaction, and machine training, but never state that a person reads what a user typed. The filing also points out that OpenAI’s Privacy Policy lists categories of outside companies that receive user data, and that a data-labeling or human-review vendor isn’t among them — while OpenAI does disclose human review elsewhere, for flagged safety content and for business-account administrators.

None of this is settled law yet, and OpenAI hasn’t answered 404 Media’s specific question about where it discloses human review to consumer users. Treat the lawsuit as an allegation, not a verdict — but the underlying facts about how the review program works are drawn from OpenAI’s own leaked materials and public documentation, not just from the complaint.

Is ChatGPT Training on Your Conversations Right Now?

For most people, yes, by default. On Free, Plus, and Pro accounts, the setting labeled “Improve the model for everyone” is switched on unless you’ve turned it off yourself. ChatGPT Business, Enterprise, and Edu accounts have that setting off automatically, and OpenAI has spent recent months pitching a zero-data-retention option to enterprise customers specifically — a tier of privacy consumers aren’t offered.

To check or change it on a personal account: open ChatGPT, tap your profile icon, go to Settings, then Data Controls, and look for the “Improve the model for everyone” toggle.

The Opt-Out Loophole Almost Nobody Mentions

Turning that toggle off feels like it should settle the matter. It doesn’t, in two specific ways worth knowing.

It isn’t retroactive. Conversations you had before flipping the switch stay in the training pipeline; the setting only protects conversations going forward.
A thumbs-up or thumbs-down on a response can pull that exchange back in. Rating a reply — something most people do without a second thought — is feedback OpenAI’s own documentation says can put the conversation tied to it back into the training pipeline, even with training otherwise switched off.

The opt-out also does nothing to stop automated scanning for Terms of Service or safety violations, which OpenAI runs regardless of your training settings — and reasonably so, but it’s a separate system from the one this controversy is about.

How ChatGPT Compares to Claude and Gemini

Project Lily isn’t evidence that OpenAI is unusually invasive compared with its rivals — human review of AI conversations is standard across the industry. What differs is the default setting a user starts on, and how clearly each company says so.

Platform	Human review	Free/personal default	Business/Enterprise default
ChatGPT (OpenAI)	Yes — outside contractors under Project Lily	Training ON (Free, Plus, Pro)	OFF by default; ZDR piloted for enterprise
Claude (Anthropic)	Yes, for users with training enabled	Opt-in, not on by default	Not used for training
Gemini (Google)	Yes, for some saved conversations	Activity setting ON unless changed	Workspace accounts excluded

Anthropic has confirmed it uses human reviewers for Claude conversations, but only for users who have left model training enabled, and says account identifiers are removed first. Google discloses on Gemini that human reviewers may read some saved conversations as part of quality review. The practice is common; the opt-in-versus-opt-out framing is where the platforms genuinely differ.

How to Actually Protect Your Privacy on ChatGPT

Turn off “Improve the model for everyone” in Settings → Data Controls if you haven’t already — and know it only protects chats from that point forward.
Skip the thumbs-up/thumbs-down buttons on anything you’d rather a stranger not read, even with training turned off.
Use Temporary Chat for one-off sensitive conversations — it isn’t saved to your history or used to train the model, though OpenAI says it may still keep a copy for up to 30 days for safety review.
Delete old conversations you no longer need — OpenAI’s help documentation outlines how deletion interacts with data already queued for training.
For anything genuinely sensitive — medical, legal, financial, or something you wouldn’t want a contractor summarizing — treat the chat window like email, not a diary: assume a human could eventually read it.

The Bottom Line

ChatGPT was never fully private, and neither are most consumer AI chatbots — human review is standard practice across the industry, not a scandal unique to OpenAI. What’s new is how visible that gap between assumption and reality has become, and that a court will now decide whether OpenAI told users clearly enough. Until then, treat sensitive conversations with any chatbot the way you’d treat an email you wouldn’t want forwarded.

Frequently Asked Questions

Does turning off ChatGPT’s training setting delete my past conversations from the system?

No. Turning off “Improve the model for everyone” only stops future conversations from being used for training and review. Conversations from before you changed the setting remain part of the existing pipeline.

Can I still be identified even with usernames removed?

It’s possible. OpenAI’s automated Privacy Filter can miss uncommon identifiers, and the summary shown to reviewers can still surface details like your general location or occupation drawn from your chat history, even without a name attached.

Is this only a problem with ChatGPT?

No. Human review of AI conversations is standard across the industry — Anthropic’s Claude and Google’s Gemini both use it in some form. The distinction is that Claude’s review is opt-in, while ChatGPT’s consumer tiers have it on by default, and Gemini discloses review of saved chats directly in its settings.

Have you checked your ChatGPT data settings since this story broke — or is this the first you’re hearing that a human might read what you type? Tell us in the comments.

Editor’s Observation

The detail that stuck with me isn’t the contractors — it’s the thumbs-up button. People use it reflexively, as feedback for the product, not as a legal signal about what happens to their words next. That gap between what a gesture feels like and what it actually authorizes is, at this point, the recurring theme of AI privacy stories in 2026 — the same gap we’ve written about with surveillance pricing and with phone permissions. The interface is designed for trust; the fine print is designed for coverage. Until those two are the same document, read the fine print.