Course: Course 2B — Securing & Attacking Harnesses and LLMs
Module: SDD-B08 — NeMo Guardrails: The Production Guardrail Framework
Duration: ~30 minutes (spoken at ~140 wpm)
Format: Verbatim transcript with [SLIDE N] cues. Read aloud or use as speaker notes.
[SLIDE 1 — Title]
Welcome to SDD-B08. NeMo Guardrails — the production guardrail framework behind NemoClaw. If you took Course 1, you met NemoClaw in DD-09 — the hardened agent whose entire contribution over its parent was a governance layer the agent cannot reach. NeMo Guardrails is that layer. This deep-dive covers what it is, the governance architecture it enables, and — critically for a red-team — its attack surface. Because here is the headline: the guardrails are themselves evaluated by models, so they are subject to the same injection risks they are built to stop. The guardrail is a model, and models are injectable.
[SLIDE 2 — The five rail types]
NeMo enforces policy through five rail types, each governing a different stage of the conversation lifecycle. Input rails run on the user's message before it reaches the model — they check for toxicity, jailbreak patterns, PII, off-topic queries. This is the first line of defense against prompt injection. Dialog rails govern the conversation flow — whether the request is on-topic, whether it matches a canonical form the agent is allowed to handle. This is how a NeMo-guardrailed system enforces "this agent only discusses X; everything else is deflected."
Retrieval rails run on retrieved content in a RAG pipeline — they check the chunks coming back from the retrieval store before they feed the model. This is the rail that governs indirect prompt injection, SDD-B03 — the retrieved content is untrusted input, and the retrieval rail can filter or quarantine it. Execution rails govern the tools and actions the model calls — checking the tool, the arguments, and the result. This is the rail that prevents a manipulated model from invoking a disallowed tool, SDD-B04's function-call surface. And output rails run on the model's response before it reaches the user — checking for leaked data, harmful content, hallucinations. This catches a model that complied with an injection despite the input rails.
The five rails cover the full lifecycle: what comes in, how the conversation flows, what is retrieved, what is executed, what goes out. A well-configured deployment uses multiple rails in sequence — the defense-in-depth thesis from B2, realized at the guardrail layer.
[SLIDE 3 — External evaluation]
The load-bearing architectural property — the one that made NemoClaw the governance reference. The guardrail layer is a separate process the agent calls through. The agent does not host the rails, does not run them, and cannot reach them to disable or modify them. When the agent makes a model call, the call passes through the guardrail layer, which runs the input rails, forwards to the model if they pass, runs the output rails on the response, and returns the guarded result.
This is the realization of the Course 1 Module zero-point-two principle: if the agent can reach the enforcement layer, a compromised agent can disable it. By placing the rails outside the agent's process boundary, NeMo ensures that a model that has been manipulated — by a prompt injection, a jailbreak — cannot turn off its own guardrails. The agent can be fully compromised and the guardrails still run, because the guardrails are not the agent's code to control. This is why NemoClaw scored plus-four over its parent: external enforcement is the architecture that holds when the agent itself is the threat.
[SLIDE 4 — B08.1 section title]
Sub-section one. What NeMo Guardrails is, in detail.
[SLIDE 5 — Programmable policy]
NeMo's rails are configured in Colang — a domain-specific language NeMo introduced for conversational flow definition — and in Python for the action implementations. The policy is code, not a fixed filter. A fixed filter, like a provider-side safety classifier, stops what its designer anticipated; it is a black box. A programmable rail stops what the deployer's policy specifies, and the policy can be as specific as the deployment requires.
Example: "this banking agent must never disclose account numbers in the output." A fixed filter cannot express that. A Colang output rail can — a regex plus a model-based classifier, written by the deployer, version-controlled, reviewable. The programmability is the core architectural decision, and it is what makes the guardrail layer auditable. The rails are code in version control, reviewable like any other security control. That is the governance advantage: the policy is legible to the security and compliance processes that must assess it, B11.
The trade-off — and this matters: programmable policy is only as good as the policy author. A missing rail is a missing control, and the framework does not infer the policy the deployer should have written. You audit the rail configuration the way you audit any security control.
[SLIDE 6 — B08.2 section title]
Sub-section two. Strengths and the governance architecture.
[SLIDE 7 — Three strengths]
Three strengths. First, programmable — policy is code, auditable, deployment-specific. Second, multi-rail — defense-in-depth across the conversation lifecycle. Each rail is a classifier with its own false-negative rate; the composition's overall bypass rate is the product of the individual rates, under independence assumptions, which is why layered defenses reach single-digit bypass rates where single defenses sit near the baseline. That is the SDD-B03 measurement. Third, production-deployed. NeMo is not a research prototype; NVIDIA maintains and ships it, and NemoClaw uses it at production scale. A governance pattern that works in a demo but not under real load is not a governance pattern.
What external enforcement buys: the guardrails run even when the agent is compromised. A prompt injection that takes over the model cannot disable the input rails, because the input rails are not the model's code. A jailbreak that makes the model produce disallowed content cannot turn off the output rails. What it does not buy is invulnerability — and that is sub-section three.
[SLIDE 8 — B08.3 section title]
Sub-section three. The attack surface and red-teaming.
[SLIDE 9 — The load-bearing vulnerability]
Here is the central insight of the deep-dive, and it is the one that determines how a red-team approaches a NeMo-guardrailed system. The guardrails are evaluated by models. An input rail that detects toxicity uses an embedding model or a classifier. An output rail that checks for leaked data uses an LLM-based check. A dialog rail that determines whether a query is on-topic uses a model-based canonical-form matching. These models are themselves subject to prompt injection, adversarial inputs, and the full range of model-level attacks from SDD-B06.
Two threats. Threat one: disable. A compromised agent reaches the rail process and turns it off. This is STOPPED by external enforcement — the agent cannot reach the rail process. Threat two: evade. An adversarial input is crafted against the rail's own classifier model surface. The rail runs, but it classifies the adversarial input as benign, because its classifier is a model and the input was crafted against its blind spots. This is NOT stopped. The external architecture protected the rails from being disabled; it did not protect the rail's classifier from being evaded.
This is the same structural property as the SDD-B06 model-layer finding. A pattern-calibrated classifier has a novel-pattern bypass. The input rail's toxicity classifier was trained on known toxic patterns; an adversarial input crafted to avoid those patterns evades it. The output rail's LLM-based check can be manipulated if the response it is checking contains a prompt injection targeting the rail's own classifier model. The guardrail is a model, and models are injectable.
[SLIDE 10 — Bypass vectors per rail type]
The specific attack surfaces, by rail type. Input rails — adversarial-input evasion. You craft an input against the rail's classifier directly, using the context-specific technique from SDD-B06, applied to the rail's model rather than the underlying model. Output rails — classifier-targeting injection. You submit inputs designed to produce responses that contain rail-targeting injections, so the output rail's classifier model is fooled. Dialog rails — canonical-form confusion. You engineer a request that matches an allowed canonical form while carrying a disallowed intent — the same indirection as indirect injection, SDD-B03. Retrieval rails — indirect-injection passthrough. You poison the retrieval store with injection payloads and measure the retrieval rail's filtering rate. Execution rails — tool-argument manipulation. You call an allowed tool with disallowed arguments, per SDD-B04.
Each bypass vector is a measurable residual. You do not assert "the guardrail can be bypassed." You measure the bypass rate over N attempts, under fixed parameters, the way InjecAgent measures injection rates. The rail's residual is the false-negative rate of its classifier against the attack class.
[SLIDE 11 — Measuring residuals and the composition]
The measurement has layers. The input-rail bypass rate — what fraction of adversarial inputs pass the input rail. The output-rail bypass rate — what fraction of disallowed responses pass the output rail. And the end-to-end rate — what fraction of attacks produce a disallowed output to the user, having passed every rail in the chain. The end-to-end rate is the finding that matters.
And here is the subtlety. A five-percent input-rail bypass rate and a five-percent output-rail bypass rate compose, under independence, to roughly zero-point-two-five percent end-to-end. But the rails are not independent when they share an underlying model surface. An adversarial technique that evades the input rail's classifier may also evade the output rail's classifier if both use the same model class. The composition is weaker than the independence calculation suggests. You must test the composition directly, because the real attack passes the whole chain.
[SLIDE 12 — Defense-in-depth synthesis]
The synthesis. NeMo Guardrails is the strongest production realization of the governance-beneath-the-agent principle, and it is a layer — not a boundary. Every defense layer the course analyzed has a residual. CrabTrap's judge is injectable, SDD-B04. IronCurtain's deterministic policy has compilation residuals, SDD-B05. The model's refusal training falls in five queries, SDD-B06. And now NeMo's guardrails have classifier-evasion residuals.
The prescription is the one the course has been building: layered defense, where each layer's bypass is bounded by the others. NeMo's input rails raise the bar and provide signal. The output rails catch what the input rails miss. The retrieval and execution rails govern the indirect-injection and tool-call surfaces. And beneath all of it, the deterministic boundary — IronCurtain-style — provides the limit the probabilistic layers cannot. A NeMo deployment that relies on the guardrails alone, without the deterministic boundary, without the harness controls from B4, has the residuals this deep-dive measures. A NeMo deployment composed with the layered architecture has each residual bounded by the others, and that is the architecture that holds.
[SLIDE 13 — Lab and what's next]
The lab has you build a simulated NeMo-style guardrail layer — five rail types with model-evaluated classifiers — craft adversarial inputs targeting each rail's classifier surface, measure the per-rail and end-to-end bypass rates over N attempts, and demonstrate the defense-in-depth composition. Python three-point-ten plus, no GPU. Next: SDD-B09, Prompt Injection Detection Models. The dedicated detection layer — classifiers trained to detect injection, distinct from the guardrail classifiers they complement. Let's build.
# Teaching Script — SDD-B08: NeMo Guardrails: The Production Guardrail Framework **Course**: Course 2B — Securing & Attacking Harnesses and LLMs **Module**: SDD-B08 — NeMo Guardrails: The Production Guardrail Framework **Duration**: ~30 minutes (spoken at ~140 wpm) **Format**: Verbatim transcript with `[SLIDE N]` cues. Read aloud or use as speaker notes. --- [SLIDE 1 — Title] Welcome to SDD-B08. NeMo Guardrails — the production guardrail framework behind NemoClaw. If you took Course 1, you met NemoClaw in DD-09 — the hardened agent whose entire contribution over its parent was a governance layer the agent cannot reach. NeMo Guardrails is that layer. This deep-dive covers what it is, the governance architecture it enables, and — critically for a red-team — its attack surface. Because here is the headline: the guardrails are themselves evaluated by models, so they are subject to the same injection risks they are built to stop. The guardrail is a model, and models are injectable. [SLIDE 2 — The five rail types] NeMo enforces policy through five rail types, each governing a different stage of the conversation lifecycle. Input rails run on the user's message before it reaches the model — they check for toxicity, jailbreak patterns, PII, off-topic queries. This is the first line of defense against prompt injection. Dialog rails govern the conversation flow — whether the request is on-topic, whether it matches a canonical form the agent is allowed to handle. This is how a NeMo-guardrailed system enforces "this agent only discusses X; everything else is deflected." Retrieval rails run on retrieved content in a RAG pipeline — they check the chunks coming back from the retrieval store before they feed the model. This is the rail that governs indirect prompt injection, SDD-B03 — the retrieved content is untrusted input, and the retrieval rail can filter or quarantine it. Execution rails govern the tools and actions the model calls — checking the tool, the arguments, and the result. This is the rail that prevents a manipulated model from invoking a disallowed tool, SDD-B04's function-call surface. And output rails run on the model's response before it reaches the user — checking for leaked data, harmful content, hallucinations. This catches a model that complied with an injection despite the input rails. The five rails cover the full lifecycle: what comes in, how the conversation flows, what is retrieved, what is executed, what goes out. A well-configured deployment uses multiple rails in sequence — the defense-in-depth thesis from B2, realized at the guardrail layer. [SLIDE 3 — External evaluation] The load-bearing architectural property — the one that made NemoClaw the governance reference. The guardrail layer is a separate process the agent calls through. The agent does not host the rails, does not run them, and cannot reach them to disable or modify them. When the agent makes a model call, the call passes through the guardrail layer, which runs the input rails, forwards to the model if they pass, runs the output rails on the response, and returns the guarded result. This is the realization of the Course 1 Module zero-point-two principle: if the agent can reach the enforcement layer, a compromised agent can disable it. By placing the rails outside the agent's process boundary, NeMo ensures that a model that has been manipulated — by a prompt injection, a jailbreak — cannot turn off its own guardrails. The agent can be fully compromised and the guardrails still run, because the guardrails are not the agent's code to control. This is why NemoClaw scored plus-four over its parent: external enforcement is the architecture that holds when the agent itself is the threat. [SLIDE 4 — B08.1 section title] Sub-section one. What NeMo Guardrails is, in detail. [SLIDE 5 — Programmable policy] NeMo's rails are configured in Colang — a domain-specific language NeMo introduced for conversational flow definition — and in Python for the action implementations. The policy is code, not a fixed filter. A fixed filter, like a provider-side safety classifier, stops what its designer anticipated; it is a black box. A programmable rail stops what the deployer's policy specifies, and the policy can be as specific as the deployment requires. Example: "this banking agent must never disclose account numbers in the output." A fixed filter cannot express that. A Colang output rail can — a regex plus a model-based classifier, written by the deployer, version-controlled, reviewable. The programmability is the core architectural decision, and it is what makes the guardrail layer auditable. The rails are code in version control, reviewable like any other security control. That is the governance advantage: the policy is legible to the security and compliance processes that must assess it, B11. The trade-off — and this matters: programmable policy is only as good as the policy author. A missing rail is a missing control, and the framework does not infer the policy the deployer should have written. You audit the rail configuration the way you audit any security control. [SLIDE 6 — B08.2 section title] Sub-section two. Strengths and the governance architecture. [SLIDE 7 — Three strengths] Three strengths. First, programmable — policy is code, auditable, deployment-specific. Second, multi-rail — defense-in-depth across the conversation lifecycle. Each rail is a classifier with its own false-negative rate; the composition's overall bypass rate is the product of the individual rates, under independence assumptions, which is why layered defenses reach single-digit bypass rates where single defenses sit near the baseline. That is the SDD-B03 measurement. Third, production-deployed. NeMo is not a research prototype; NVIDIA maintains and ships it, and NemoClaw uses it at production scale. A governance pattern that works in a demo but not under real load is not a governance pattern. What external enforcement buys: the guardrails run even when the agent is compromised. A prompt injection that takes over the model cannot disable the input rails, because the input rails are not the model's code. A jailbreak that makes the model produce disallowed content cannot turn off the output rails. What it does not buy is invulnerability — and that is sub-section three. [SLIDE 8 — B08.3 section title] Sub-section three. The attack surface and red-teaming. [SLIDE 9 — The load-bearing vulnerability] Here is the central insight of the deep-dive, and it is the one that determines how a red-team approaches a NeMo-guardrailed system. The guardrails are evaluated by models. An input rail that detects toxicity uses an embedding model or a classifier. An output rail that checks for leaked data uses an LLM-based check. A dialog rail that determines whether a query is on-topic uses a model-based canonical-form matching. These models are themselves subject to prompt injection, adversarial inputs, and the full range of model-level attacks from SDD-B06. Two threats. Threat one: disable. A compromised agent reaches the rail process and turns it off. This is STOPPED by external enforcement — the agent cannot reach the rail process. Threat two: evade. An adversarial input is crafted against the rail's own classifier model surface. The rail runs, but it classifies the adversarial input as benign, because its classifier is a model and the input was crafted against its blind spots. This is NOT stopped. The external architecture protected the rails from being disabled; it did not protect the rail's classifier from being evaded. This is the same structural property as the SDD-B06 model-layer finding. A pattern-calibrated classifier has a novel-pattern bypass. The input rail's toxicity classifier was trained on known toxic patterns; an adversarial input crafted to avoid those patterns evades it. The output rail's LLM-based check can be manipulated if the response it is checking contains a prompt injection targeting the rail's own classifier model. The guardrail is a model, and models are injectable. [SLIDE 10 — Bypass vectors per rail type] The specific attack surfaces, by rail type. Input rails — adversarial-input evasion. You craft an input against the rail's classifier directly, using the context-specific technique from SDD-B06, applied to the rail's model rather than the underlying model. Output rails — classifier-targeting injection. You submit inputs designed to produce responses that contain rail-targeting injections, so the output rail's classifier model is fooled. Dialog rails — canonical-form confusion. You engineer a request that matches an allowed canonical form while carrying a disallowed intent — the same indirection as indirect injection, SDD-B03. Retrieval rails — indirect-injection passthrough. You poison the retrieval store with injection payloads and measure the retrieval rail's filtering rate. Execution rails — tool-argument manipulation. You call an allowed tool with disallowed arguments, per SDD-B04. Each bypass vector is a measurable residual. You do not assert "the guardrail can be bypassed." You measure the bypass rate over N attempts, under fixed parameters, the way InjecAgent measures injection rates. The rail's residual is the false-negative rate of its classifier against the attack class. [SLIDE 11 — Measuring residuals and the composition] The measurement has layers. The input-rail bypass rate — what fraction of adversarial inputs pass the input rail. The output-rail bypass rate — what fraction of disallowed responses pass the output rail. And the end-to-end rate — what fraction of attacks produce a disallowed output to the user, having passed every rail in the chain. The end-to-end rate is the finding that matters. And here is the subtlety. A five-percent input-rail bypass rate and a five-percent output-rail bypass rate compose, under independence, to roughly zero-point-two-five percent end-to-end. But the rails are not independent when they share an underlying model surface. An adversarial technique that evades the input rail's classifier may also evade the output rail's classifier if both use the same model class. The composition is weaker than the independence calculation suggests. You must test the composition directly, because the real attack passes the whole chain. [SLIDE 12 — Defense-in-depth synthesis] The synthesis. NeMo Guardrails is the strongest production realization of the governance-beneath-the-agent principle, and it is a layer — not a boundary. Every defense layer the course analyzed has a residual. CrabTrap's judge is injectable, SDD-B04. IronCurtain's deterministic policy has compilation residuals, SDD-B05. The model's refusal training falls in five queries, SDD-B06. And now NeMo's guardrails have classifier-evasion residuals. The prescription is the one the course has been building: layered defense, where each layer's bypass is bounded by the others. NeMo's input rails raise the bar and provide signal. The output rails catch what the input rails miss. The retrieval and execution rails govern the indirect-injection and tool-call surfaces. And beneath all of it, the deterministic boundary — IronCurtain-style — provides the limit the probabilistic layers cannot. A NeMo deployment that relies on the guardrails alone, without the deterministic boundary, without the harness controls from B4, has the residuals this deep-dive measures. A NeMo deployment composed with the layered architecture has each residual bounded by the others, and that is the architecture that holds. [SLIDE 13 — Lab and what's next] The lab has you build a simulated NeMo-style guardrail layer — five rail types with model-evaluated classifiers — craft adversarial inputs targeting each rail's classifier surface, measure the per-rail and end-to-end bypass rates over N attempts, and demonstrate the defense-in-depth composition. Python three-point-ten plus, no GPU. Next: SDD-B09, Prompt Injection Detection Models. The dedicated detection layer — classifiers trained to detect injection, distinct from the guardrail classifiers they complement. Let's build.