Add semantic guardrail layer for prompt-injection pre-check
Extends the regex-only pre-check into three stages: regex scan, then embedding-similarity search against known injection examples, and only if that score clears a threshold, an LLM judge confirms intent before blocking. Catches paraphrased injection attempts the fixed regex patterns miss without paying for an LLM call on every query. Validated end-to-end against a real Anthropic API key and a live Qdrant instance: the new path correctly blocked a paraphrased injection (audit log confirms rule_id: injection.semantic_llm_judge) while a benign near-miss passed through without ever reaching the LLM judge. Also documents two reliability issues found while testing with claude-haiku-4-5 (a non-deterministic policy verdict and a forced tool-call loop that can end without calling any tool) in docs/SAFETY_WRITEUP.md section 6 -- unrelated to this change, but worth recording since they were found in the same pass. Co-Authored-By: Claude Sonnet 5 <[email protected]>
This commit is contained in:
@@ -55,7 +55,7 @@ impl SupportSupervisor {
|
||||
}
|
||||
|
||||
pub async fn handle_query_traced(&self, query: &str) -> Result<QueryTrace> {
|
||||
let pre = self.guardrail.pre_check(query)?;
|
||||
let pre = self.guardrail.pre_check(query).await?;
|
||||
if pre.action == GuardrailAction::Block {
|
||||
return Ok(QueryTrace {
|
||||
retrieved_sources: Vec::new(),
|
||||
|
||||
Reference in New Issue
Block a user