Production AI Systems

Most AI projects fail at the boundary, not the model

The model is rarely the hard part. The hard part is everything around it: what gets retrieved, what the model is allowed to do with a tool, what happens on a malformed response, and how anyone proves the system is still correct three releases later.

We treat an LLM as one non-deterministic component inside an otherwise deterministic system. Every input is bounded, every tool call is typed, every output that touches money, health, or legal state is validated before it leaves the process.

• Retrieval that cites its source documents, chunk by chunk, so answers are auditable
• Typed tool contracts - the model chooses, the runtime validates and executes
• Evaluation suites run in CI against golden datasets before any prompt change ships
• Explicit fallback paths for low-confidence, timeout, and refusal cases

How we build retrieval

Most RAG failures are ingestion failures. Documents are chunked without regard to structure, embeddings are generated once and never re-scored, and nobody measures recall. We start at the corpus.

We build structure-aware chunking, hybrid search (dense vectors plus lexical BM25 on Postgres), reranking on the top candidates, and a recall benchmark that tells you what percentage of answerable questions actually retrieve their source. If retrieval recall is 62%, no prompt engineering will save the answer quality - you fix retrieval first.

• pgvector on PostgreSQL - one database, transactional consistency, no separate vector store to keep in sync
• Hybrid dense + lexical retrieval with a reranking pass
• Incremental re-indexing on document change, not nightly full rebuilds
• Per-query retrieval telemetry: what was fetched, what was used, what was cited

Agents, where they earn their cost

Multi-agent architectures are oversold. Most workloads are better served by one model with good tools and a state machine around it. We use a supervisor pattern only where the task genuinely branches - research, triage, and multi-system reconciliation.

When we do build agents, the orchestration layer is ordinary software: a durable state machine with retries, idempotency keys, and full replay. If a run fails at step seven, it resumes at step seven.

Evaluation is the deliverable

We hand over an evaluation harness with every AI system, because it is the only thing that lets your team change prompts without fear. Golden datasets, regression thresholds, and a CI gate - the same discipline any other production system gets.