← All projects

Project 01

Enterprise Knowledge Base

Hybrid RAG over a messy corpus, with citations back to source

RAGEmbeddingsVector searchHybrid retrievalRe-rankingEnterprise search

Ingests a deliberately messy corpus — PDFs, wiki pages, email threads — then chunks, embeds and indexes it. Queries run vector search and BM25 keyword search in parallel, fuse the results, and pass the top candidates through a cross-encoder re-ranker. Every answer cites the source chunk it came from, because an enterprise answer without provenance is unusable.

Try it

How it works

  1. 01
    Ingest

    Documents land in S3. An event triggers the ingestion Lambda.

  2. 02
    Chunk & enrich

    Structure-aware chunking, then metadata enrichment (source, section, version).

  3. 03
    Embed

    Batch embedding calls, cached by content hash so re-ingestion is free.

  4. 04
    Index

    Vectors and BM25 postings in Aurora Serverless v2 (pgvector + tsvector), one database for both.

  5. 05
    Retrieve

    Parallel dense + sparse retrieval, reciprocal rank fusion, cross-encoder re-rank of the top 30.

  6. 06
    Answer

    Grounded generation with inline citations and a refusal path when retrieval returns nothing relevant.

Infrastructure & running cost

ServiceRoleCost
Lambda (container)FastAPI app, shared across all 5 projectsFree tier
Aurora Serverless v2pgvector + full-text index, scales to 0 ACU~$3/mo
S3Raw corpus + embedding cache~$0.10/mo
DynamoDBQuery cache, keyed by normalised questionFree tier

Anchor project — everything else reads from this index.