AI guardrails explained: what input and output filtering can and cannot do

A language model will answer almost anything it is asked and will say almost anything it can generate. Guardrails are the checks that sit around it so that a business application only accepts appropriate requests and only returns appropriate answers. They are worth understanding before you buy or build an assistant, because they are often described as a complete safety solution and are not one.

The five checkpoints

NVIDIA's open-source NeMo Guardrails documentation is a useful reference because it names where checks run, using the term "rails" ^[1][2].

The same documentation notes that input and output rails are the most common ^[2].

A concrete example

Imagine an assistant that answers customer questions about orders for a distributor (an illustration, not a case study). An input check stops requests to reveal internal pricing rules or to ignore its instructions. A retrieval check makes sure it only reads current, approved product documents. An execution check confirms that a refund request is for the logged-in customer and under a set limit before the system calls the payments tool. An output check strips any account number that slipped into the reply. Each layer catches something the others might miss.

What the evidence says

Filtering works, with a price. Anthropic tested a classifier system on 10,000 synthetic jailbreak prompts against Claude 3.5 Sonnet. Without classifiers, 86% of jailbreak attempts succeeded; with them, 4.4% did. The extra refusals on ordinary traffic rose by 0.38%, which was not statistically significant, but compute cost rose 23.7% ^[3].

Human testing was less clean. In a public demo in February 2025, 339 jailbreakers made over 300,000 chats; one participant found a universal jailbreak and another a borderline one ^[3]. Good filters raise the cost of an attack, not remove it.

Where guardrails fall short

The UK National Cyber Security Centre makes the sharpest point. Unlike SQL injection, which can be fixed at the root, large language models have no built-in separation between instructions and data, and the NCSC suggests prompt injection may never be fully mitigated. Its guidance is to reduce likelihood and impact, use deterministic non-LLM safeguards to constrain what the system can do, be wary of deny-lists that attackers can rephrase around, be sceptical of vendors claiming to "stop" prompt injection, and log inputs, outputs and tool calls ^[4].

False positives are the other cost: a filter that blocks legitimate requests, such as routine code analysis or debugging queries, frustrates users and creates support tickets. A guardrail that is too strict is a support ticket generator.

What to ask a vendor

• Which of the five checkpoints do you cover, and which are left to us?

• What can the system do if a filter is bypassed? Limits on tools and data access matter more than the filter.

• How do you measure false positives, and can we see logs of blocked requests?

Practical implication

Use guardrails as one layer, and design the assistant so that its worst-case behavior is acceptable even when a filter fails: narrow tool permissions, spending limits and human approval for irreversible actions.