← David Liu

RAG Microservice

FastAPI · pgvector

Ask Tethos's internal documents a question and get a cited answer. The interesting problem isn't retrieval, it's not re-embedding the 95% of documents that didn't change.

Context

Problem

Institutional knowledge lives across a Drive folder tree and a Notion workspace, and nobody reads either.

New executives ask the same questions every September.

Shape

A standalone service, not a feature of the platform. It's containerized and deployed on its own so the portal can call it and never own its dependencies.

Build

Ingest

Walks a Drive folder tree recursively, holding a visited set so a cyclic shortcut can't trap the crawler. Notion is crawled breadth-first from a root page, handling eleven block types.

Both land in one documents table, with Notion IDs namespaced so the two sources coexist.

The Hard Part

Embedding is the expensive step. Re-embedding an unchanged 50-chunk document is 50 API calls for no new information.

So re-indexing gates on two signals, not one: Drive's modifiedTime and a SHA-256 hash of the exported text. A doc whose timestamp moved because someone left a comment, but whose text is byte-identical, is skipped by the hash. A doc whose text changed under a stale timestamp is still caught. Either signal alone leaks.

The endpoint reports documents_processed against documents_skipped, so it tells you its own cache hit rate.

Writes

Updating a document is upsert-then-replace: delete its chunks, re-insert them, all in one transaction that rolls back on failure. A doc is never left half-chunked.

Chunk identity is deliberately not preserved. Boundaries shift when the text shifts, so a chunk-level diff would be lying to itself.

Failures are per-document, so one bad export can't abort a 200-document crawl.

Retrieval

Sentence-aware chunking at ~1000 characters with 200 characters of overlap, carrying the previous chunk's trailing sentences so a fact split across a boundary survives.

Query embeds, pulls the top 6 chunks by vector distance, and answers with titles and URLs attached. If retrieval comes back empty the service refuses to guess.

Four routing modes trade cost against reasoning, so a cheap question doesn't buy an expensive model.

Stack

Service

Python, FastAPI, Pydantic, Docker, Cloud Run.

Retrieval

PostgreSQL with pgvector. OpenAI text-embedding-3-small, 1536 dimensions.

Sources

Google Drive API. Notion API.