Where your data goes, step by step, inside Curiosity

Leon

• 2026-08-04

Sooner or later, someone in your IT or security team asks the plain question. You connect the ticket tool, the file share and the maintenance logs. What happens to them after that? Where are they copied, who can see them, what reads the text, and what does the language model get?

We wrote the answer down. How Curiosity processes your data is a 13 page paper for business and IT professionals. It follows a record through seven steps, from the connector that loads it to the interface your users see. Each step links to its page in the documentation.

Below is the paper's argument, plus outside reading on the two places where systems like ours go wrong: permissions and search.

One idea: connect once, read the same result everywhere

The paper's summary fits in a paragraph. Curiosity connects your data once and loads it into one graph, together with its permissions. It enriches the text, indexes it for keywords and for meaning, and serves the result to search, to AI agents and to your own applications. Each of them reads the same permission-aware data.

Compare that with what most companies already run. Search has its index. The chatbot has its own vector store. The reporting app has a nightly export. Each copy has its own idea of who may see what, and each one drifts from the source at its own pace.

Steps 1 and 2: the data arrives with its access rights

A connector copies what a source system holds into Curiosity, and it copies who may see it too. It runs on a schedule. Each item keeps the ID it has in the source, so a rerun updates items instead of duplicating them. On Monday the first run copies every ticket and document. On Tuesday the next run adds only the tickets that changed.

The paper is honest about one sharp edge. Data deleted in the source is not deleted in Curiosity automatically, so the connector has to remove it.

Permissions are written at the same moment as the data. The connector marks a record private, or limits it to a team or a person, as the source does. Users arrive with the groups your identity provider gives them, refreshed at each sign in. From there, every part of the system checks the same rules. In the graph, a record is visible only if the user has a route to it. Every search is filtered by the user's teams. Endpoints and AI run as the signed-in user.

Two lines in this chapter are for your security reviewer. First, a record with no restriction is visible to everyone, so mirror the source's rules rather than trusting the default. Second, the language model never decides who may read a record. It only receives what the user may already see.

The idea of access as a path through a graph is not ours alone. Google's Zanzibar paper (USENIX ATC 2019) describes one data model for access control lists across Calendar, Drive, YouTube and other services, at the scale of trillions of lists. Curiosity is not Zanzibar. The point is narrower: one model of who may see what, evaluated on every request, has held up in production at very large scale. We wrote about why the check has to happen before the model reads anything in Permission-aware retrieval.

Step 3: text becomes linked objects

Enrichment finds the things a text mentions, such as part numbers and process references, and links each mention to the matching object in the graph. The paper's example is short. A support ticket mentions part 2741-08. Enrichment links that mention to the part, so the ticket shows up whenever someone looks at the part. A new part number found by a pattern can create its own object.

Built-in readers extract the text from files first, and scans and photos go through text recognition (OCR). Then you choose how to capture mentions, and the paper lists four ways, cheapest first:

Approach	Best for	Speed	Cost
Word list	Fixed lists: products, customers	Instant	Negligible
Pattern	Codes with a fixed format: part numbers	Instant	Negligible
Trained model	People, organizations, places in prose	Fast	Computing power
Language model	Open-ended text, summaries	Seconds	Per use

Its advice is to start with the simple, predictable methods and add the others only where gaps remain. Cost is one reason. Predictability is the other. A pattern for part numbers either matches or it does not. A language model asked to read a scanned page makes guesses that look right. The team at Pulse, which sells document extraction, catalogs this in Why LLMs suck at OCR: shifted decimal points, "0.5mg" read as "5mg", and models that solve an equation instead of transcribing it. It is a vendor's post with a product to sell, so read it as an argument. The failure it describes is the one you care about with part records, though: a plausible wrong character is worse than an obvious miss.

Steps 4 and 5: findable by keyword, by meaning and by link

You decide which fields are findable, and how. A keyword index matches exact words, which is what you want for identifiers: searching 2741-08 finds records with that part number. A vector index matches meaning, so "hydraulic leak" also finds a report that says "fluid loss". Hybrid search switches on both and returns one list, with exact matches first and similar reports after. Facets let users narrow by status, priority, a linked customer or a time range.

At query time there is a third route that a search index alone does not have: following links in the graph. From part 2741-08, find the service notices that mention it and the tickets that cite those notices. That answers questions about how records relate, which neither keyword nor meaning can do. Every route checks the user's permissions.

The case for hybrid search is well made outside our own material. John Wang, CTO of Assembled, describes in Better RAG results with Reciprocal Rank Fusion and Hybrid Search how vector search alone kept missing support articles for short keyword queries. Adding keyword search, and merging the two lists by rank, fixed it. He also found that hand-tuned weights between the two did not carry across customers. In the thread under it, one commenter adds that putting the data into a graph with queryable relationships helped them further. The paper's adjustable balance answers the tuning problem in a simpler way: more keyword for identifiers, more meaning for narrative text. If you want to know what the resulting score means, the search score paper covers it.

Step 6: the model asks, Curiosity answers as the user

AI orchestration is a loop. A person asks a question. Curiosity sends it, with your instructions and a list of tools, to the language model you chose: OpenAI, Anthropic, Azure or a local model. The model asks for a search or an action. Curiosity runs it as the signed-in user and returns only what that user may see. The model writes a reply that cites its sources, and each turn is logged for audit.

A skill is a written guide to how your business works, and a tool is a function that searches or acts. An agent saves instructions, tools and a model as one reusable assistant.

The counterpoint I take most seriously comes from Simon Willison's lethal trifecta talk. An agent with three things can be tricked into leaking: access to private data, exposure to untrusted content and a way to send data out. He also argues that a filter which stops 99% of attacks is a failing grade in security. Running retrieval as the user does not remove that risk. A ticket can carry hostile text, and the user can read their own records. What it does is bound the damage: the model only ever holds what that user could already open, and every turn leaves an audit record. I would rather say that plainly than claim the problem is solved.

Step 7: your own logic on the same data

The last step is where teams build. Endpoints add your own logic, such as a supplier risk check. Interfaces put it in front of your users, built with Tesserae and Transpose. The paper ends the chapter with a short guide to which tool does which job:

Load data from a system? Use a connector.
Run something on a schedule? Use a scheduled task.
Let chat trigger your logic? Use an AI tool.
Add permission-aware logic or an action? Use an endpoint.

Sudo, the admin assistant in Curiosity Studio, works on this configuration: endpoints, AI tools, agents, indexes, the schema and more. Ask for "a supplier risk check for the A320 line" and Sudo drafts and builds what it needs. You review the complete change, then approve or discard it. An approved change goes live as one tracked update.

Read the paper

If you are the person who has to sign off on connecting a system, start with chapters 01 and 02. If you are the person who has to make search good, start with 03 to 05. Each step points to its documentation page, so you can check every claim.

Read the paper online or download the PDF. When you are ready, ask us to walk through the step that matters most to you on your own systems.

Read the whitepaper (PDF)