Maritime/Logistics Support Assistant

A RAG + multi-agent support assistant for a fictional shipping company (Meridian Sea Logistics), built with a safety/eval layer as the headline feature rather than an afterthought. Written in Rust.

Why this exists

This is a portfolio piece modeled on real maritime-logistics support work, scoped to be publishable: its own fictional company, own documents, own IP. The interesting part isn't the RAG pipeline (that's table stakes) -- it's the guardrail and regression-eval layer wrapped around it, and the trade-offs documented in docs/SAFETY_WRITEUP.md.

Architecture

Three narrow agents behind a deterministic supervisor:

flowchart TD
    Q["Customer Query"] --> REGEX{{"Regex injection scan"}}
    REGEX -- clean --> SEM{{"Embed query -> cosine similarity<br/>vs. known injection examples"}}
    SEM -- score < threshold --> RET["Retrieval Agent<br/>vector search + LLM relevance filter"]
    SEM -- score >= threshold --> JUDGE{{"LLM Injection Judge"}}
    JUDGE -- not injection --> RET
    JUDGE -- injection --> FINAL
    REGEX -- match --> FINAL["Final Answer + Citations -> Customer"]
    RET <--> QD[("Qdrant Vector Store<br/>chunked KB docs")]
    RET --> DRAFT["Drafting Agent<br/>tool-calling loop"]
    DRAFT -- lookup_shipment --> SHIP[("Shipment Repository")]
    DRAFT -- escalate_to_human --> ESC[("Escalation Desk")]
    DRAFT --> POL["Policy-Compliance Agent"]
    POL --> DOCS[("acceptable_use.md +<br/>tracking_policy.md")]
    POL --> POST{{"Guardrail: Post-check<br/>PII/injection scan + policy verdict"}}
    POST -- allow / escalate --> FINAL
    REGEX -.-> AUDIT[("Audit Log<br/>JSONL, append-only")]
    JUDGE -.-> AUDIT
    POST -.-> AUDIT

    classDef agent fill:#eef2ff,stroke:#4f46e5,color:#312e81
    classDef guardrail fill:#fff7ed,stroke:#ea580c,color:#7c2d12
    classDef store fill:#f1f5f9,stroke:#64748b,color:#334155
    classDef audit fill:#f0fdf4,stroke:#16a34a,color:#14532d
    classDef io fill:#f8fafc,stroke:#334155,color:#0f172a

    class RET,DRAFT,POL agent
    class REGEX,SEM,JUDGE,POST guardrail
    class QD,SHIP,ESC,DOCS store
    class AUDIT audit
    class Q,FINAL io

The pre-check is three layers, cheapest first: a regex scan catches fixed wordings for free; if that's clean, the query is embedded and compared against a small corpus of known injection examples so paraphrases that dodge the regexes still get caught; only if that similarity score clears a threshold does the pipeline pay for an LLM call to judge intent, since embedding similarity alone is too noisy to block a customer on. See src/guardrails/semantic_guardrail.rs and src/guardrails/injection_judge.rs.

The supervisor (src/agents/supervisor.rs) is what runs this graph -- plain Rust control flow, not another LLM call, so what's allowed through is never itself something the model can be talked out of. The dashed edges are the audit trail: every guardrail decision is logged, not just the blocked ones.

Every LLM call asks for a schema-validated structured output (via a forced tool-call, see src/llm/client.rs) rather than free text -- there's no Rust equivalent of pydantic-ai, so this is a small hand-rolled agent loop against the raw Anthropic Messages API.

The crate is organized as a dependency-injected module tree (see src/cli.rs, the composition root) -- no global/static singletons anywhere; every component is constructed explicitly and passed to its collaborators through the constructor.

src/
  config.rs        Settings, loaded once in cli.rs
  models/          domain types + structured agent I/O + eval types
  knowledge/       the fictional company's markdown knowledge base
  data/            seed shipment records
  embeddings/      EmbeddingProvider trait + local fastembed impl
  retrieval/       chunking, Qdrant wrapper, reference extraction
  tools/           shipment lookup + human escalation (as LLM tools)
  llm/             Anthropic client, tool trait, agentic loop
  agents/          retrieval / policy / drafting agents + supervisor
  guardrails/      rule-based scanner, audit log, decision pipeline
  eval/            golden dataset loader, scoring, regression harness
  cli.rs           composition root + subcommands

Running it

Requires a Qdrant instance and an Anthropic API key. Embeddings run locally via fastembed (ONNX Runtime) -- the model downloads once on first use, then runs fully offline.

docker run -p 6334:6334 qdrant/qdrant
cp .env.example .env   # fill in ANTHROPIC_API_KEY

cargo run -- ingest              # chunk + embed + index the knowledge base
cargo run -- ask "Where is my container MSCU7654321?"
cargo run -- eval                # run the golden dataset, print the report

Testing

cargo test      # pure-logic unit + integration tests, no network required
cargo clippy --all-targets

The eval command exercises the live model and Qdrant; cargo test deliberately does not, so CI-style checks don't need API keys.

The safety/eval layer

• Golden dataset (data/golden_dataset.jsonl, 44 examples): normal queries, prompt-injection attempts, policy-violation requests, hallucination bait (asking about shipments that don't exist), and out-of-scope requests.
• Regression harness (src/eval/harness.rs): scores retrieval accuracy, generation accuracy, and guardrail correctness separately per example and per category, so a failing run points at where the failure originated instead of just "test 17 failed."
• Guardrail pipeline (src/guardrails/): a fast rule-based scan runs before and after the LLM agents, independent of what the policy agent concludes; the pre-check also embeds the query and escalates high-similarity matches against known injection phrasings to an LLM judge before blocking, catching paraphrases the regexes miss without paying for an LLM call on every query. Every decision -- allow or block -- is written to an append-only audit log.

See docs/SAFETY_WRITEUP.md for the failure modes this was designed for, the trade-offs made, and what would need to change for real production scale.