LLM Guardrails

What are guardrails?

Guardrails are deterministic or model-based checks that run outside the LLM: validating inputs before they reach the model and outputs before they reach users or downstream systems. Prompting says “please behave”; guardrails verify. Production LLM apps treat model output like user input — untrusted until validated.

The main types

• Format guards — JSON schema validation, type checking, allowed-value enforcement; retry or repair on failure. The cheapest and highest-ROI guard.
• Safety guards — classifiers for toxicity, PII leakage, and prompt injection attempts on both input and output.
• Grounding guards — for RAG: check that claims in the answer are supported by retrieved sources (reduces hallucination reaching users).
• Action guards — for agents: allowlists of callable tools, spend/step budgets, human approval for irreversible operations.

Guardrails vs evals

Complementary, often confused: evals measure quality offline before you ship; guardrails enforce rules online on every request. Evals tell you the failure rate; guardrails catch the failures that still happen.

Cost reality

Format guards are free (code). Model-based guards add a classifier call per request — small models keep this at fractions of a cent, but at high volume it’s a real line item, and each guard adds latency (run them in parallel with generation where possible). The trade is worth it anywhere output feeds automation: one malformed JSON response crashing a pipeline costs more than a year of schema checks.

What people get wrong

• Relying on the system prompt as the only control. Anything critical must be enforced outside the model.
• Blocking without a fallback. A guard that just errors ruins UX; design retry-with-feedback, safe-default, or human-handoff paths.
• Guarding output but not input. Injection and abuse arrive on the way in.

Try it in code

The highest-ROI guardrail: schema validation with retry-with-feedback:

import json

def extract_invoice(text, max_retries=2):
    prompt = f"Extract as JSON with keys amount (number), currency (string), due_date (YYYY-MM-DD):\n{text}"
    for attempt in range(max_retries + 1):
        raw = llm(prompt)
        try:
            data = json.loads(raw)
            assert isinstance(data.get("amount"), (int, float))
            assert data.get("currency") in {"USD", "EUR", "GBP"}
            return data                                    # validated - safe for automation
        except (json.JSONDecodeError, AssertionError) as e:
            prompt += f"\n\nYour previous output was invalid ({e}). Return ONLY valid JSON."
    raise ValueError("model failed validation after retries")  # fail loudly, never silently

Treat model output like user input: parse, validate, and have a failure path. This one function prevents the “malformed JSON took down the pipeline” incident.