Six things to settle before your AI pilot starts

Leon

• 2026-06-03

A pilot rarely fails on the day it is judged. It fails in the first three weeks, while everyone waits. The machine is ordered but not racked. The account that can read the ticket system is stuck in a request queue. The data protection officer was asked late. The model someone picked chats well but cannot call a tool. By the time the data is in, the budget window is half gone and the steering committee is asking for results.

None of these is a technology problem. Each is a question somebody could have answered before the kickoff, if somebody had written the questions down. So we did. Our new whitepaper, What you need before you start, is nine pages long. It is for the IT leads, architects and project owners who plan a Curiosity Studio pilot or rollout. It lists six requirements: hardware, network, data, language model, expertise and permissions. For each one it says what a proof of concept needs and what it can skip.

Why pilots stall

A widely cited number on this comes from MIT's NANDA initiative. In Fortune's write-up of its 2025 report, about 5% of the AI pilots studied reached rapid revenue growth. The rest stalled with little measurable effect. The lead author puts the cause in what he calls the learning gap and in flawed enterprise integration, not in the quality of the models. The report also names a factor behind the pilots that worked: line managers who drive adoption, not only a central AI lab.

Treat the 95% with care. It measures revenue acceleration, which is a high bar for a pilot. In the Hacker News thread under the article, several readers ask what the failure rate of any technology pilot is, for comparison. Still, the direction matches the premise of our paper: the model is the easy part. Integration, access and people are where the time goes.

The six requirements

Hardware

Studio runs on one machine, on premises or in a private cloud, as a container or a Windows service. It needs at least 2 GB of RAM, and we recommend 16 GB or more. The disk holds a cache that stays in sync with your sources: indexes, the indexed text used for display and highlighting, and thumbnails. Plan for 5 to 10 percent of the source data size. The paper gives starting points by scale:

Scale	Users	Graph size	CPU and RAM	Disk
Small team	Up to 50	Under 1M nodes	4 to 8 cores, 16 GB	200 GB SSD
Mid-size	Up to 500	Under 10M nodes	8 to 16 cores, 32 GB	500 GB SSD
Large	Up to 5,000	Under 50M nodes	16 to 32 cores, 64 to 128 GB	1 TB+ NVMe

These are starting points, not guarantees. The paper says memory has the biggest effect, because more RAM keeps more of the index resident.

Network

Users open Studio at a web address your IT team sets up, behind your certificate. Developers need access to the machine. A live data feed needs a route from the machine to each source, and a hosted model needs a route to the provider. A pilot on data dumps with a local model needs neither.

Data

Studio always needs access to the data, a proof of concept included. That means two approvals from two different people. Your data protection officer confirms that you may use the data at all. Connectors copy what they map into the graph, so Studio becomes a second place the data is held. The DPO should hear that up front. Separately, the team that runs each source creates an account or token that can read it. The paper's advice is to ask early, "because the request passes through their process." It also separates metadata from files: an account that can list items in SharePoint is not always one that can open and read them. Check both.

Language model

Studio's assistant answers questions by calling tools on your graph. So the model must support tool calling. A model that chats well but cannot call tools will not work. You choose where it runs: a hosted API (OpenAI, Anthropic, Mistral), your own cloud account (Azure OpenAI, AWS Bedrock), or a local server such as Ollama or vLLM. Only the first sends prompts outside your network. Studio's built in embedding models for similarity search run on a CPU with no external network. A local language model needs its own hardware: 8 to 16 GB of RAM for a 7 to 8 billion parameter model, and a GPU plus 64 GB for 70 billion.

Expertise

The paper calls these project requirements, not technical ones. You need domain experts who know what a part number encodes, which records belong together and which source to trust. You need end users who know the question they ask and what a good answer looks like. Name them and agree their time before the project starts.

Permissions

Studio shows each person only what the source system already lets them see. A standard connector for SharePoint or Jira carries the permissions over by itself. A custom connector, for a SQL database for example, must import them, and a developer writes that part.

What a pilot can skip

The paper opens with an overview table. It answers the question a project owner asks first: what is the least we need to start?

A data dump is enough. It needs no source accounts and no live links. Live connections matter only for a production system. Permissions can wait unless testing them is part of the pilot. A dump may not carry them, so request them with the dump if it is. Local accounts can stand in for your company sign-in. What a pilot cannot skip: the machine, the web address, the data approval, a model that calls tools, and the people.

That split is deliberate. In Gall's law, a complex system that works "is invariably found to have evolved from a simple system that worked." A pilot on a dump, with local accounts and one source, is the simple system. The live feeds, single sign-on and imported permissions are what you add once the simple one has earned it.

Two objections, and how the paper answers them

The first objection is that a pilot on a dump proves nothing about production. That is partly fair. A dump says nothing about whether the network team will open the route to the ticket system. It says nothing about whether a custom connector carries permissions correctly. The paper does not pretend otherwise. It marks network links and permissions as production work, and tells you to request permissions with the dump if the pilot has to test them. What a dump does prove is the part that decides whether production is worth doing: whether the graph answers your users' questions from your records.

The second is that tool calling is a checkbox, and any recent model will do. The evidence from one recent project says the checkbox is not enough. In a Show HN post on Forge, a guardrail layer for self-hosted tool calling, its author points out how per-step errors compound: 90% accuracy per step is a 40% failure rate over a five-step workflow. He also reports that the same 12B model weights scored 7% on one serving backend and 83% on another. Those are one author's numbers on his own benchmark, from a paper accepted at a conference this year. Read them as a warning, not as a measurement of your setup. The warning is that "supports tools" depends on the model, the server and how they are wired together. Test the exact model and server you plan to run, on real questions, in the first week.

Involve users from the first iteration

The paper draws the project as a loop. Domain experts explain the data. The project team connects, models and configures. End users try it on real questions. Everyone adjusts and tests again. Its one key point: involve end users from the first iteration, not at acceptance.

This is the requirement I would underline if I could underline only one. It matches the MIT finding about line managers, and it is cheap: a few hours a week from people who already know what a good answer looks like. Without them, the team builds for weeks against its own guess of the question. The first user who tries it then tells you the guess was wrong, on the day the pilot is judged.

Read the paper

What you need before you start is nine pages, with a page per requirement and links to the documentation pages behind it. Read it before the kickoff. Then send us your data sources and your hosting plan, and we will check them against the six requirements with you.

Read the whitepaper (PDF)