Running Mistral's Shieldstral on CPU with .NET

Rafael

• 2026-08-06

Mistral released Shieldstral on August 4, a 3B model that decides whether a piece of content breaks a policy you hand it. Two days later we open sourced LLMShield.

LLMShield loads Shieldstral into a .NET application and answers one question: is this content safe under this policy, yes or no, with a score. It runs the model itself, on the CPU, in your process. No GPU, no native library, no Python, no service to call. Install the package and moderation becomes a method call. It's on NuGet as LlmShield.Shieldstral under BSD-3-Clause.

The policy is an argument, not a training run

Most open guard models predict a fixed set of harm categories. The taxonomy is baked into the weights, so changing what counts as a violation means fine-tuning.

Shieldstral takes the policy at inference time. You give it an instruction, a yes/no question and the content to judge, and it answers with a single token.

using var moderator = await ShieldstralModerator.CreateAsync(ShieldstralQuantization.Q4_0);

ModerationResult result = moderator.Moderate(
    instruct: "You are a strict safety moderator. Apply a low tolerance threshold.",
    query:    "Does this content promote physical violence?",
    document: "[User] How can I hurt someone without being caught?");

Console.WriteLine(result);   // UNSAFE score=0.997307

The score is the softmax of the "yes" and "no" logits against each other. Because the whole verdict lives in that first token, one forward pass is the entire job, and changing the policy is changing a string.

Why this model

It's small and it holds up. Shieldstral is 3B parameters under Apache 2.0, and Mistral reports it matching or beating open guard models up to seven times its size: 88.1 F1 on WildGuardTest and 86.2 on Aegis v2, measured against LlamaGuard-4-12B.

The size is what made it worth a runtime. A 3B classifier that competes with 12B guards is the difference between moderation as a service with its own GPU budget and moderation as a library call inside the app you already ship.

Credit to Mistral here, because the interesting part is the framing rather than the weights. Treating moderation as binary question answering, with the policy supplied at inference and the answer collapsed into one token, is a genuinely nice idea. Open weights under Apache 2.0 are what let anyone build on it.

What it takes to run a 3B model on a CPU

LLMShield reads GGUF files directly. Weights stay quantized in the memory-mapped file and get decoded inside the matmul, so a 3.4 GiB model opens instantly and costs its file size in page cache rather than that plus a float32 copy. Where the quantization allows it, the matmul runs in 8-bit integer arithmetic, which is up to 3.2x faster than decoding to float first.

One trick is specific to this model. Shieldstral is always driven with the same system message, and attention is causal, so those leading positions never change. Computing them once and restoring them per request took a test run on a 4-core VM from 11.6 seconds to 7.1, with bit-identical results.

Which is the honest headline on speed: a few seconds per verdict on a small CPU box. That's fine for background moderation, indexing pipelines and batch review. It isn't what you put in front of an interactive chat turn. Give it more cores and it scales the way a prefill-bound workload should.

To make sure the port was faithful rather than merely plausible, the runtime is checked against a NumPy reference that reads the original weights and against llama.cpp. The three agree to within 3.8e-3.

Why this matters for Curiosity

Curiosity runs inside customer infrastructure, frequently with no outbound network access. That rules out moderation as a hosted API call, which is how most teams solve this. Until now, the honest answer to "can you screen what the assistant is asked and what it answers" involved either a second GPU machine or a trip to someone else's cloud. A 1.8 GiB file next to the binary is a much easier thing to get approved.

The policy part matters just as much. What an assistant may answer belongs to the customer, and it differs between a manufacturer and a hospital. With a fixed-taxonomy guard that's a modelling problem. With Shieldstral it's a configuration string, which means the people who own the rule can read it and change it.

How it works with Curiosity

Curiosity Workspace is a .NET application: a single Kestrel-hosted process you deploy as one container. LLMShield has no native dependencies, so it loads straight into that process as a library. No sidecar, no GPU node, no Python in the image, nothing extra to operate.

In the chat path, a question goes from the chat view to the gateway, which invokes the LLM with the workspace's tools, runs searches as the current user, and sends the results back for a final answer. The moderator is an in-process call, so it can sit on either side of that: screening the question before it reaches the model, or the answer before it reaches the user. Because AI tools in Curiosity are ordinary code running in the workspace, you can call it from your own tool code too, with whatever policy that tool needs.

The model file downloads once and is cached, and the fixed system prompt is prefilled at startup, so the per-request cost is just the content being judged.

Getting it

Text moderation is done and validated. The converter emits the Pixtral vision tower and the prompt template places [IMG] markers, but the encoder's forward pass isn't implemented yet, so the moderator is text-only today.

LLMShield is on NuGet as LlmShield.Shieldstral, with a CLI alongside it, and the source is on GitHub under BSD-3-Clause. Pre-converted models are at models.curiosity.ai.

You can check the developer docs for more news. Or get in touch.