# Maritime/Logistics Support Assistant A RAG + multi-agent support assistant for a fictional shipping company (Meridian Sea Logistics), built with a safety/eval layer as the headline feature rather than an afterthought. Written in Rust. ## Why this exists This is a portfolio piece modeled on real maritime-logistics support work, scoped to be publishable: its own fictional company, own documents, own IP. The interesting part isn't the RAG pipeline (that's table stakes) -- it's the guardrail and regression-eval layer wrapped around it, and the trade-offs documented in [`docs/SAFETY_WRITEUP.md`](docs/SAFETY_WRITEUP.md). ## Architecture Three narrow agents behind a deterministic supervisor: ```mermaid flowchart TD Q["Customer Query"] --> REGEX{{"Regex injection scan"}} REGEX -- clean --> SEM{{"Embed query -> cosine similarity
vs. known injection examples"}} SEM -- score < threshold --> RET["Retrieval Agent
vector search + LLM relevance filter"] SEM -- score >= threshold --> JUDGE{{"LLM Injection Judge"}} JUDGE -- not injection --> RET JUDGE -- injection --> FINAL REGEX -- match --> FINAL["Final Answer + Citations -> Customer"] RET <--> QD[("Qdrant Vector Store
chunked KB docs")] RET --> DRAFT["Drafting Agent
tool-calling loop"] DRAFT -- lookup_shipment --> SHIP[("Shipment Repository")] DRAFT -- escalate_to_human --> ESC[("Escalation Desk")] DRAFT --> POL["Policy-Compliance Agent"] POL --> DOCS[("acceptable_use.md +
tracking_policy.md")] POL --> POST{{"Guardrail: Post-check
PII/injection scan + policy verdict"}} POST -- allow / escalate --> FINAL REGEX -.-> AUDIT[("Audit Log
JSONL, append-only")] JUDGE -.-> AUDIT POST -.-> AUDIT classDef agent fill:#eef2ff,stroke:#4f46e5,color:#312e81 classDef guardrail fill:#fff7ed,stroke:#ea580c,color:#7c2d12 classDef store fill:#f1f5f9,stroke:#64748b,color:#334155 classDef audit fill:#f0fdf4,stroke:#16a34a,color:#14532d classDef io fill:#f8fafc,stroke:#334155,color:#0f172a class RET,DRAFT,POL agent class REGEX,SEM,JUDGE,POST guardrail class QD,SHIP,ESC,DOCS store class AUDIT audit class Q,FINAL io ``` The pre-check is three layers, cheapest first: a regex scan catches fixed wordings for free; if that's clean, the query is embedded and compared against a small corpus of known injection examples so paraphrases that dodge the regexes still get caught; only if that similarity score clears a threshold does the pipeline pay for an LLM call to judge intent, since embedding similarity alone is too noisy to block a customer on. See `src/guardrails/semantic_guardrail.rs` and `src/guardrails/injection_judge.rs`. The supervisor (`src/agents/supervisor.rs`) is what runs this graph -- plain Rust control flow, not another LLM call, so *what's allowed through* is never itself something the model can be talked out of. The dashed edges are the audit trail: every guardrail decision is logged, not just the blocked ones. Every LLM call asks for a schema-validated structured output (via a forced tool-call, see `src/llm/client.rs`) rather than free text -- there's no Rust equivalent of pydantic-ai, so this is a small hand-rolled agent loop against the raw Anthropic Messages API. The crate is organized as a dependency-injected module tree (see `src/cli.rs`, the composition root) -- no global/static singletons anywhere; every component is constructed explicitly and passed to its collaborators through the constructor. ``` src/ config.rs Settings, loaded once in cli.rs models/ domain types + structured agent I/O + eval types knowledge/ the fictional company's markdown knowledge base data/ seed shipment records embeddings/ EmbeddingProvider trait + local fastembed impl retrieval/ chunking, Qdrant wrapper, reference extraction tools/ shipment lookup + human escalation (as LLM tools) llm/ Anthropic client, tool trait, agentic loop agents/ retrieval / policy / drafting agents + supervisor guardrails/ rule-based scanner, audit log, decision pipeline eval/ golden dataset loader, scoring, regression harness cli.rs composition root + subcommands ``` ## Running it Requires a Qdrant instance and an Anthropic API key. Embeddings run locally via `fastembed` (ONNX Runtime) -- the model downloads once on first use, then runs fully offline. ```bash docker run -p 6334:6334 qdrant/qdrant cp .env.example .env # fill in ANTHROPIC_API_KEY cargo run -- ingest # chunk + embed + index the knowledge base cargo run -- ask "Where is my container MSCU7654321?" cargo run -- eval # run the golden dataset, print the report ``` ## Testing ```bash cargo test # pure-logic unit + integration tests, no network required cargo clippy --all-targets ``` The `eval` command exercises the live model and Qdrant; `cargo test` deliberately does not, so CI-style checks don't need API keys. ## The safety/eval layer - **Golden dataset** (`data/golden_dataset.jsonl`, 44 examples): normal queries, prompt-injection attempts, policy-violation requests, hallucination bait (asking about shipments that don't exist), and out-of-scope requests. - **Regression harness** (`src/eval/harness.rs`): scores retrieval accuracy, generation accuracy, and guardrail correctness *separately* per example and per category, so a failing run points at where the failure originated instead of just "test 17 failed." - **Guardrail pipeline** (`src/guardrails/`): a fast rule-based scan runs before and after the LLM agents, independent of what the policy agent concludes; the pre-check also embeds the query and escalates high-similarity matches against known injection phrasings to an LLM judge before blocking, catching paraphrases the regexes miss without paying for an LLM call on every query. Every decision -- allow or block -- is written to an append-only audit log. See [`docs/SAFETY_WRITEUP.md`](docs/SAFETY_WRITEUP.md) for the failure modes this was designed for, the trade-offs made, and what would need to change for real production scale.