NeMo Guardrails: The Production Guardrail Framework
SDD-B08 · Course 2B — Securing & Attacking Harnesses and LLMs
45 minutes · The programmable guardrail layer behind NemoClaw (Course 1 DD-09).
Five rail types enforce policy between user and model. Strengths: programmable, multi-rail, production-deployed, externally evaluated. The catch: the rails are themselves evaluated by models — so they are injectable. The guardrail is a model, and models are injectable.
Deep-Dive · SDD-B08
The five rail types: conversation lifecycle
| Rail | Stage governed | What it catches |
|---|
| Input | User message before the model | Toxicity, jailbreak patterns, PII, off-topic — 1st line vs injection |
| Dialog | Conversation flow | On-topic check, canonical-form match, "this agent only discusses X" |
| Retrieval | RAG chunks before the model | Indirect-injection filtering (SDD-B03); quarantine untrusted content |
| Execution | Tool calls + arguments + result | Disallowed tool invocation / argument manipulation (SDD-B04) |
| Output | Response before the user | Leaked data, harmful content, hallucinations — catches a model that complied |
External evaluation: governance beneath the agent
The load-bearing architectural property. The guardrail layer is a SEPARATE PROCESS the agent calls through. The agent does not host, run, or reach the rails.
Course 1 Module 0.2 principle: if the agent can reach the enforcement layer, a compromised agent can disable it. NeMo realizes the fix — enforcement sits outside the agent's reach.
NemoClaw (Course 1 DD-09) is the production reference: +4 score over its parent, almost entirely from external guardrails evaluating every model call. "Governance done right."
B08.1 — What NeMo is
Five rails, Colang, external evaluation
Programmable policy: code, not a fixed filter
Fixed filter (provider-side safety classifier): black box, stops what the designer anticipated, cannot express deployment-specific constraints.
Programmable rail (NeMo): Colang + Python, policy is code, version-controlled, reviewable. Stops what the DEPLOYER specifies — deployment-specific compliance and brand policies.
The trade-off: programmable policy is only as good as the POLICY AUTHOR. A missing rail = a missing control. The framework does not infer the policy the deployer should have written. Audit the rail configuration (B11).
B08.2 — Strengths & governance architecture
Programmable, multi-rail, production-deployed
Three strengths
| Strength | What it means |
|---|
| Programmable | Policy is code (Colang/Python), auditable, deployment-specific. Not a black box. |
| Multi-rail | Defense-in-depth across the conversation lifecycle. Input + output + retrieval + execution + dialog — each a layer. |
| Production-deployed | NVIDIA maintains and ships it. NemoClaw (DD-09) uses it at production scale. Not a demo. |
What external enforcement buys: the guardrails run EVEN WHEN THE AGENT IS COMPROMISED. A prompt injection that takes over the model cannot disable the input rails — they are not the model's code.
B08.3 — The attack surface
The guardrail is a model, and models are injectable
The load-bearing vulnerability
The rails are evaluated by MODELS — embedding models, LLM-based classifiers. External enforcement protects against DISABLE. It does NOT protect against EVADE.
Threat: DISABLE — a compromised agent reaches the rail and turns it off. STOPPED by external enforcement (the agent cannot reach the rail process).
Threat: EVADE — an adversarial input crafted against the RAIL'S OWN CLASSIFIER. The rail runs, classifies it as benign. NOT STOPPED.
Same structural property as SDD-B06: a pattern-calibrated classifier has a novel-pattern bypass. The context-specific attack technique, applied to the rail's classifier, evades it.
Bypass vectors per rail type
| Rail | Bypass vector |
|---|
| Input | Adversarial-input evasion — craft against the classifier's known patterns (SDD-B06 technique on the rail's model) |
| Output | Classifier-targeting injection — response contains a prompt injection targeting the output rail's own classifier model |
| Dialog | Canonical-form confusion — a disallowed intent matching an allowed canonical form (indirection, SDD-B03) |
| Retrieval | Indirect-injection passthrough — untrusted retrieved chunk not quarantined (SDD-B03) |
| Execution | Tool-argument manipulation — allowed tool, disallowed arguments (SDD-B04) |
Each is a measurable residual. Measure the bypass RATE over N attempts (SDD-B03/SDD-B06), per rail AND end-to-end. No rate, no finding.
Defense-in-depth: every layer has a residual
| Layer | The residual |
|---|
| Model refusal (SDD-B06) | Falls in 5 queries (context-specific attack) |
| NeMo guardrails (this DD) | Classifier-evasion bypass — the guardrail is a model |
| Harness governance (SDD-B04) | Judge is injectable |
| Deterministic boundary (SDD-B05) | Compilation residuals — but load-bearing |
The prescription: NeMo composed with the layered architecture HOLDS. NeMo alone has the residuals this deep-dive measures. The deterministic boundary (IronCurtain) is the limit the probabilistic rails cannot provide.
Lab & what's next
Lab (07): build a simulated NeMo-style guardrail layer (five rail types with model-evaluated classifiers), craft adversarial inputs targeting each rail's classifier surface, measure the per-rail and end-to-end bypass rates over N attempts, and demonstrate the defense-in-depth composition. Python 3.10+, no GPU.
Next — SDD-B09: Prompt Injection Detection Models. The dedicated detection layer — classifiers trained to detect injection, distinct from the guardrail classifiers they complement.