Building Safe AI Features: Guardrails, Prompt Injection and Data Privacy

LLM features introduce a new kind of attack surface. The model reads untrusted text (user messages, web pages, emails, documents) and that text can contain instructions. If your model can also call tools or see sensitive data, a cleverly worded document can turn your assistant against you.

This guide covers the main risks and the defenses that work in practice. The goal isn't to make AI features impossible to misuse; no system is. The goal is to make misuse hard, contained and visible.

#Key takeaways

• Prompt injection is the defining LLM security risk. Treat all model inputs from outside your system as untrusted.
• There's no perfect prompt-level fix. Defend in layers: least-privilege tools, confirmation for risky actions, output filtering, monitoring.
• Never put secrets in prompts and never rely on the model to enforce authorization. Enforce it in code.
• Minimize and redact personal data before it reaches the model; know your provider's data retention terms.
• Log and audit model inputs, outputs and tool calls.

#The threat model

#Prompt injection: why it's hard

LLMs process instructions and data as the same thing: text. There's no reliable equivalent of parameterized SQL queries that cleanly separates "command" from "content". Any text the model reads can influence what it does.

That's why indirect injection is the dangerous variant. Your user may be completely honest, but the web page, PDF, email or support ticket your agent reads may not be.

#Defenses that help

1. Delimit untrusted content with clear tags and tell the model that content inside them is data, not instructions. This reduces naive attacks; it does not stop determined ones.
2. Least privilege for tools. If a summarization feature doesn't need to send email, it must not have an email tool. Scope API tokens to the minimum.
3. Human confirmation for consequential actions: sending messages, making payments, deleting data, changing permissions. Show the user exactly what will happen.
4. Separate privileges by context. An agent that has read untrusted content in the current session shouldn't be able to call high-risk tools without confirmation.
5. Output validation. Check tool arguments against allowlists: domains, recipients, amount limits, record ownership.
6. Classifiers and detectors that flag likely injection attempts in inputs, as one layer among several.
7. Monitoring: alert on unusual tool-call patterns, such as an agent suddenly emailing external addresses.

#Authorization belongs in code, not prompts

A prompt saying "only show the user their own orders" is a suggestion, not a control. Enforce access at the data layer:

pythondef search_orders(user: User, query: str):
    # The model can choose the query. It can't choose whose data it sees.
    return db.orders.search(query=query, customer_id=user.customer_id)

The same applies to RAG: filter retrieved chunks by the user's permissions before they enter the prompt. See RAG explained.

#Handle model output as untrusted

Treat LLM output like user input:

• Escape before rendering. Don't inject raw model output as HTML. Render Markdown with a sanitizer.
• Never eval() generated code outside a proper sandbox with no network access and strict resource limits.
• Parameterize database queries. If the model generates SQL, run it with a read-only role, statement timeouts and table allowlists.
• Validate structured output against schemas before acting on it.

#Protecting sensitive data

• Minimize. Send the model only the fields it needs. It rarely needs full names, emails or account numbers to classify a ticket.
• Redact or pseudonymize PII before sending (replace with tokens like <PERSON_1>), then re-insert values after, if the output needs them.
• Know your provider's terms: data retention, whether data is used for training, regional processing and enterprise agreements. Configure zero-retention options where available and required.
• Keep secrets out of prompts. System prompts can be extracted. API keys, internal URLs and business logic that must stay secret belong in code.
• Log responsibly. Prompt logs are valuable for debugging and evals, and they're also a sensitive dataset. Apply retention limits and access controls.

#Content safety

For user-facing generation:

• Use provider moderation endpoints or classifiers on inputs and outputs where appropriate.
• Write clear scope rules ("You only answer questions about Acme's products") and a polite refusal format.
• Test with red-team prompts, both manual and automated, and add failures to your eval set (LLM evaluation guide).
• Evaluate for bias on relevant dimensions for your use case, especially in hiring, lending or healthcare contexts.

#Abuse and cost controls

• Rate limit per user and per IP; cap tokens per request and per day. See rate limiting algorithms.
• Require authentication for expensive features.
• Alert on spend anomalies.

#A practical security checklist

• Every tool the model can call is necessary for the feature
• Tool credentials are scoped to the minimum permissions
• Consequential actions require explicit user confirmation
• Authorization is enforced in code and at retrieval time
• Untrusted content is delimited and treated as data
• Model output is escaped, validated and never executed unsandboxed
• PII is minimized or redacted; provider retention terms are reviewed
• No secrets in prompts
• Inputs, outputs and tool calls are logged with access controls
• Rate limits and spend alerts are configured
• Red-team cases are part of the eval suite

#FAQ

#Can prompt injection be fully prevented?

Not with today's models. You can make attacks harder with good prompting and detectors, but the reliable controls are architectural: least privilege, human confirmation for risky actions, and enforcing permissions outside the model.

#Is it safe to send customer data to an LLM API?

It can be, with the right contractual terms, data-retention settings, minimization and redaction. Review your provider's data policies, your own privacy commitments and regulations such as GDPR or HIPAA, where applicable, before sending personal data.

#Should I hide my system prompt?

Assume it can be extracted. Don't put secrets or security logic in it. It's fine to keep it private as a matter of product polish, but it's not a security boundary.

#What is excessive agency?

Giving an AI system more permissions, tools or autonomy than its task needs. It turns a minor manipulation into a serious incident. The fix is least privilege and approval steps. See what are AI agents.