The app is the easy part. The foundation is not

Rafael

• 2026-10-18

In my last post I listed three apps that leaked data within days of shipping. They have one thing in common: none of them failed in the part anyone was looking at. The interface worked. What broke was underneath: a database key in the browser, a view that skipped the access rules, a check that ran backwards.

So we went looking for more, and wrote them up in a new whitepaper, The app is the easy part. The foundation is not. It is written for CIOs and security leads whose teams now build internal apps from scratch or generate them from a prompt. Every incident in it is public and dated, and the paper sorts them by the layer that was missing.

Data leaks through the basics

McHire, the hiring chatbot Paradox.ai runs for McDonald's franchisees, exposed up to 64 million applications in 2025. Ian Carroll and Sam Curry got into the admin side with the username and password 123456. The Tea app left about 72,000 images, 13,000 of them ID photos, in a Firebase bucket anyone could read. An exam app hosted on Lovable exposed 18,697 user records because its generated access check blocked signed-in users and let anonymous ones through. Escape scanned 5,600 public apps built on prompt-to-app platforms and found more than 400 exposed secrets.

None of these is a model failure. Each is a credential, a storage setting or an access check that the app had to get right on its own.

Assistants do what a stranger asks

PromptArmor showed Slack AI putting a secret from a private channel into a link, following instructions someone had planted in a public one. EchoLeak did the same to Microsoft 365 Copilot with a single email and no click. Invariant Labs had an agent post private GitHub repositories in a public pull request because an issue told it to. And during a code freeze, Replit's agent deleted SaaStr's production database.

Simon Willison's lethal trifecta explains the first three: private data, untrusted content and a way to send data out, in one assistant. Nobody has made prompt injection impossible. What a foundation decides is how much an injected instruction can reach, and whether anything changes without a person saying yes.

Stacks drift as they grow

The scale problems are quieter. Martin Kleppmann wrote about dual writes in 2015: "you probably won't even notice that your database and your search indexes have gone out of sync." Google's Zanzibar paper names the permission version, the "new enemy" problem, where a person already removed from a document can see what was added after. The pgvector README is candid about filtering: with a filter that matches 10% of rows and default settings, a query that should return 40 results returns about 4. And SharePoint Online caps an app's search at 25 requests a second, shared with every other app on the tenant.

Each is documented behavior, not a bug, and the glue between parts has to handle it. None of it shows at pilot size.

Maintenance and the supply chain

The calendar is the other risk. LangChain shipped three releases with migrations between May 2024 and October 2025. OpenAI turns off the Assistants API in August 2026. Black Duck's 2026 audit of 947 commercial codebases found components with no development for two years in 93% of them.

Every dependency is also a way in. The xz backdoor took two years of patient trust. The Shai-Hulud worm compromised more than 500 npm packages and spread itself with the credentials it stole. Code generators add another way in: Spracklen et al. found that 19.7% of the packages they recommend do not exist, and 43% of those invented names come back every time, so an attacker can register them.

What a foundation changes

That is the case for building the foundation once and putting every app on it. In Curiosity Studio, the browser holds a session token, never a key to the store. Access is an edge in the graph, checked per record before any count or facet, and the built in tools search as the person asking. Sudo, the assistant that drafts endpoints and screens, works in a sandbox with no route out, and nothing it writes runs until an administrator approves the diff. The graph, the text index and the vectors live in one store, fed by one commit. And Studio's own dependencies are ours to track and patch, in one release.

That does not make an app on Studio immune to everything above. Your own endpoints are still your code, and a user's chat can still meet a poisoned document. What changes is that the hard layers are built and reviewed once, and each new app is a small piece of code on top of them.

The paper has a table of incidents for each chapter and a two page list of sources. Read it on its page or download the PDF.

Read the whitepaper (PDF)

Further reading

• Tea app breach, The Record. The storage misconfiguration and what it exposed. (HN discussion)
• Data exfiltration from Slack AI, PromptArmor. A step by step account of a prompt injection that crossed a permission boundary. (HN discussion)
• GitHub MCP exploited, Invariant Labs. Why an agent holding access to every repository is the flaw, not the server. (HN discussion)
• The case against pgvector, Alex Jacobs. What happens to index builds and filtered queries once a vector store grows. (HN discussion)
• Why dual writes are a bad idea, Martin Kleppmann. The clearest explanation of why copies of the same data drift. (HN discussion)
• We Have a Package for You!, Spracklen et al., USENIX Security 2025. The measurement behind package hallucination.