Add semantic guardrail layer for prompt-injection pre-check
Extends the regex-only pre-check into three stages: regex scan, then embedding-similarity search against known injection examples, and only if that score clears a threshold, an LLM judge confirms intent before blocking. Catches paraphrased injection attempts the fixed regex patterns miss without paying for an LLM call on every query. Validated end-to-end against a real Anthropic API key and a live Qdrant instance: the new path correctly blocked a paraphrased injection (audit log confirms rule_id: injection.semantic_llm_judge) while a benign near-miss passed through without ever reaching the LLM judge. Also documents two reliability issues found while testing with claude-haiku-4-5 (a non-deterministic policy verdict and a forced tool-call loop that can end without calling any tool) in docs/SAFETY_WRITEUP.md section 6 -- unrelated to this change, but worth recording since they were found in the same pass. Co-Authored-By: Claude Sonnet 5 <[email protected]>
This commit is contained in:
@@ -18,8 +18,13 @@ Three narrow agents behind a deterministic supervisor:
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Q["Customer Query"] --> PRE{{"Guardrail: Pre-check<br/>regex injection scan"}}
|
||||
PRE -- allow --> RET["Retrieval Agent<br/>vector search + LLM relevance filter"]
|
||||
Q["Customer Query"] --> REGEX{{"Regex injection scan"}}
|
||||
REGEX -- clean --> SEM{{"Embed query -> cosine similarity<br/>vs. known injection examples"}}
|
||||
SEM -- score < threshold --> RET["Retrieval Agent<br/>vector search + LLM relevance filter"]
|
||||
SEM -- score >= threshold --> JUDGE{{"LLM Injection Judge"}}
|
||||
JUDGE -- not injection --> RET
|
||||
JUDGE -- injection --> FINAL
|
||||
REGEX -- match --> FINAL["Final Answer + Citations -> Customer"]
|
||||
RET <--> QD[("Qdrant Vector Store<br/>chunked KB docs")]
|
||||
RET --> DRAFT["Drafting Agent<br/>tool-calling loop"]
|
||||
DRAFT -- lookup_shipment --> SHIP[("Shipment Repository")]
|
||||
@@ -27,9 +32,9 @@ flowchart TD
|
||||
DRAFT --> POL["Policy-Compliance Agent"]
|
||||
POL --> DOCS[("acceptable_use.md +<br/>tracking_policy.md")]
|
||||
POL --> POST{{"Guardrail: Post-check<br/>PII/injection scan + policy verdict"}}
|
||||
POST -- allow / escalate --> FINAL["Final Answer + Citations -> Customer"]
|
||||
PRE -- block --> FINAL
|
||||
PRE -.-> AUDIT[("Audit Log<br/>JSONL, append-only")]
|
||||
POST -- allow / escalate --> FINAL
|
||||
REGEX -.-> AUDIT[("Audit Log<br/>JSONL, append-only")]
|
||||
JUDGE -.-> AUDIT
|
||||
POST -.-> AUDIT
|
||||
|
||||
classDef agent fill:#eef2ff,stroke:#4f46e5,color:#312e81
|
||||
@@ -39,12 +44,20 @@ flowchart TD
|
||||
classDef io fill:#f8fafc,stroke:#334155,color:#0f172a
|
||||
|
||||
class RET,DRAFT,POL agent
|
||||
class PRE,POST guardrail
|
||||
class REGEX,SEM,JUDGE,POST guardrail
|
||||
class QD,SHIP,ESC,DOCS store
|
||||
class AUDIT audit
|
||||
class Q,FINAL io
|
||||
```
|
||||
|
||||
The pre-check is three layers, cheapest first: a regex scan catches fixed
|
||||
wordings for free; if that's clean, the query is embedded and compared
|
||||
against a small corpus of known injection examples so paraphrases that
|
||||
dodge the regexes still get caught; only if that similarity score clears a
|
||||
threshold does the pipeline pay for an LLM call to judge intent, since
|
||||
embedding similarity alone is too noisy to block a customer on. See
|
||||
`src/guardrails/semantic_guardrail.rs` and `src/guardrails/injection_judge.rs`.
|
||||
|
||||
The supervisor (`src/agents/supervisor.rs`) is what runs this graph -- plain
|
||||
Rust control flow, not another LLM call, so *what's allowed through* is
|
||||
never itself something the model can be talked out of. The dashed edges are
|
||||
@@ -114,8 +127,11 @@ deliberately does not, so CI-style checks don't need API keys.
|
||||
failure originated instead of just "test 17 failed."
|
||||
- **Guardrail pipeline** (`src/guardrails/`): a fast rule-based scan runs
|
||||
before and after the LLM agents, independent of what the policy agent
|
||||
concludes, and every decision -- allow or block -- is written to an
|
||||
append-only audit log.
|
||||
concludes; the pre-check also embeds the query and escalates
|
||||
high-similarity matches against known injection phrasings to an LLM
|
||||
judge before blocking, catching paraphrases the regexes miss without
|
||||
paying for an LLM call on every query. Every decision -- allow or
|
||||
block -- is written to an append-only audit log.
|
||||
|
||||
See [`docs/SAFETY_WRITEUP.md`](docs/SAFETY_WRITEUP.md) for the failure
|
||||
modes this was designed for, the trade-offs made, and what would need to
|
||||
|
||||
Reference in New Issue
Block a user