What Never Goes in a Prompt: Samsung Leak Checklist

TL;DR

Treat every prompt as a message leaving the building. Source code, customer files, credentials, and health or payment data never go in a public model. Samsung’s 2023 leak and Air Canada’s chatbot case show why a one-page policy has to name those bans.

Banned in a public prompt: source code, client files, secrets, and health data.

Key Takeaways

• Safety = protect people from the model (accidental harm: bias, hallucinations, leaking PII, over-reliance) — guardrails. Security = protect the model from attackers (adversarial: prompt injection, breaches, exfiltration). Intent is the key distinction.
• A deployer owns what staff paste and what the model tells customers.
• Download the free 1-page AI safety policy template — email is optional, not a gate.

What never goes in a ChatGPT prompt?

If you would not print it on a postcard, do not paste it into a public model. That rule covers source code and design files, customer contracts, credentials and API keys, health or student records, payment data, unpublished financials, and anything under NDA.

In 2023, reporting described Samsung semiconductor staff pasting confidential source code into ChatGPT. The company then restricted generative AI tools. The engineering lesson is simple: a prompt is an egress.

The company still owns the answer

In Moffatt v. Air Canada (2024), a chatbot invented a bereavement-fare policy. The Civil Resolution Tribunal treated the airline as responsible for the statement. A deployer cannot hide behind “the model said it.” If the bot can change a customer’s rights, a human review path belongs in the policy before launch.

How to write the ban so people follow it

Name the tools that are allowed. Name the data classes that are forbidden. Name who to ask when the task is gray. Keep it to one page. Module 0 includes that template as a free download. Write and quiz stay in the classroom after login.

Enter the free classroom

Watch Module 0 — Why AI Safety Is Not Security — with no login. Write and quiz stay in your classroom. Unlock M1–M5 for $199. If you are a deployer, also read the EU AI Act deployer page.