Freorit

The problem: AI coding assistants forget everything

If you use an AI coding assistant like Claude Code day to day, you know the ritual: open a new session, paste in three files for context, explain the project structure again, then ask your actual question. Every session starts from zero. The assistant is smart, but it doesn't know your codebase — it only knows what you remembered to paste.

That's fine for a single script. It falls apart across a real project with dozens of repos, months of decisions, and docs explaining why something was built a certain way — context that lives in your head, not in the current file you happen to have open.

So I built a small system that fixes this: every prompt automatically gets the handful of code chunks and doc snippets most relevant to what I'm asking, pulled from an index of my own repos — no copy-pasting, no manual context management, and no dependency on a paid embeddings API.

The idea: retrieval, not memorization

This is a pattern called RAG — Retrieval-Augmented Generation. Instead of trying to cram an entire codebase into a model's context window (or relying on the model to "just remember"), a separate, lightweight system searches an index for the most relevant pieces of text and hands only those to the assistant. Small, targeted, current.

The guiding principle I stuck to throughout: deterministic before clever. Git history, file paths, and vector similarity search do almost all the work. There's no extra LLM in the loop trying to guess what I "really meant" by rewriting my query — that's a layer I deliberately left out, because it adds latency and an entire new failure mode for a benefit I hadn't actually measured yet. Simple, observable pieces first; add cleverness only once you can prove it's needed.

The architecture: three roles, each doing one job

Git host (small, low-power)
   -- push webhook -->
Cloud worker (embeddings + vector search)
   -- retrieval query -->
Dev machine (AI coding assistant)
   -- prompt --> back to the git host

• A small, always-on Git server (self-hosted Forgejo on Debian) stays the single source of truth. It chunks changed files locally (by function, by heading, by code block — not blind character splits) and ships only the diff on every push. No heavy computation happens here; the hardware behind it is deliberately modest.
• A free-tier cloud worker does the actual embedding computation via Ollama (a tool for running open LLMs and embedding models locally, not a paid API), and stores the resulting vectors in Oracle Autonomous Database 23ai, which has native vector search built in — similarity search runs directly where the data already lives, no separate vector-database service needed.
• My dev machine runs the AI coding assistant plus a small hook that fires automatically on every prompt: check if I'm in an indexed repo, query the retrieval service, inject whatever comes back.

A quick word on what "embedding" actually means, for anyone new to this: an embedding model turns a piece of text into a list of numbers that represents its meaning, not the text itself. Two pieces of text with similar meaning end up with similar number-lists. That's what makes vector search possible — instead of matching exact words, you're searching for "closeness" in meaning-space. Ollama does that conversion locally on the cloud worker; Oracle 23ai's native vector type is what lets the database search that meaning-space directly.

Nothing here needs a paid API key beyond the coding assistant itself. Free-tier cloud compute, an open embedding model, a database that already does vector search natively, and infrastructure that was already running for other reasons.

Measuring retrieval quality instead of guessing

It's easy to eyeball a RAG system, ask it a few questions, decide "looks good," and move on. I didn't want to trust that. So before calling retrieval quality good enough to rely on, I built a small set of realistic test queries, each paired with the document that should come back as the top result — a "golden query" set — and a script that runs all of them automatically and reports Recall@1, Recall@5, Recall@10.

That measurement caught a real bug immediately: the embedding model needs task-specific prompt formatting (documents and queries have to be wrapped differently before embedding) — skip that, and Recall@1 sits around 30%. Fix it, and a focused test set jumped to over 80% at rank 1, 100% within the top 5. As more projects got indexed and the query set grew harder and broader, recall settled around 60–70% — which is a real, known number now, not a feeling, and something any future change can be checked against instead of "it seems worse today."

Fail loud, not silent

The most valuable lesson from actually running this in production had nothing to do with search relevance — it was about honesty under failure.

The first version of the automatic-injection hook was built and tested to fail silently on any problem — unreachable service, timeout, nothing relevant found, all produced the same outcome: no context injected, prompt proceeds untouched. At the time, that was the right call: retrieval was explicitly best-effort, a bonus when it worked, never something to depend on.

That stopped being true once retrieval became mandatory for every prompt in an indexed project, not just a nice-to-have. And that's when the old design's real weakness showed up: silence is ambiguous. Did retrieval run and find nothing, or did it never run at all? Once you're actually depending on a system, "maybe it worked" is worse than a visible failure — you can't reason about a failure mode that's indistinguishable from the system doing nothing at all.

The fix was small: the hook now always reports what happened — success with results, a genuine "nothing relevant found," a timeout, or an error — instead of ever staying silent (short of the one case where it correctly shouldn't run at all: a project outside the index). Four honest outcomes instead of one ambiguous one.

The CPU-contention problem nobody warns you about

Running the embedding worker and the interactive retrieval query on the same small machine created a real, measurable problem: whichever one was already using the CPU slowed the other down badly. A background indexing job and a human waiting on an answer have completely different priorities, but by default they competed for the same resource equally.

Measured before any fix: an interactive query during active background indexing took ~16 seconds on average — and a third of attempts timed out completely.

The fix didn't require a new service or a message queue. A simple file-based advisory lock, a pattern any Linux-based system already supports natively:

@contextlib.contextmanager
def interactive_priority(lock_path: str):
    with open(lock_path, "w") as lock_file:
        fcntl.flock(lock_file, fcntl.LOCK_EX)
        try:
            yield
        finally:
            fcntl.flock(lock_file, fcntl.LOCK_UN)

The interactive path holds this lock for the duration of its request. The background worker checks — with a short, bounded wait — whether the lock is held before starting its next batch of work. It can't interrupt something already running (no cancellation mechanism exists for that), but it stops piling new background work onto an already-busy system exactly when someone is waiting on an answer.

Measured after the fix, same test, same load: zero timeouts, most queries back in under six seconds. Not a complete fix — one sample in five still hit the old slow path, landing on an in-flight background call the lock couldn't preempt — but a large, honest, measured improvement. I wrote that limitation down rather than rounding it up to "solved."

What this took, and what it didn't

No paid embeddings API. No managed vector database service — Oracle 23ai's native vector support did that job. No new message broker. Just infrastructure already sitting idle, one open embedding model, roughly 15 minutes of setup per new project to bring it into the index, and a handful of small, well-tested services doing one job each.

The build-vs-buy tradeoff is real — a managed RAG platform would have been faster to stand up. What I got instead: full visibility into every failure mode, zero recurring API cost, a measured (not assumed) retrieval quality number, and — more valuably for me — a concrete, working answer to "can you actually build a retrieval pipeline, not just call one."

Open source and technologies used

This pipeline is built entirely on freely available tools:

• Ollama – runs the open embedding model locally, no paid API
• Oracle Autonomous Database 23ai – free-tier database with native vector search
• Forgejo – self-hosted Git server acting as the source of truth
• Debian – the Linux distribution running the cloud worker

Try the pattern yourself

You don't need three machines to start. The core idea scales down fine: one process that chunks and embeds your docs, a vector-capable database (several now support this natively — check what you're already running before adding a new service), a small golden-query test set so you can measure instead of guess, and a small hook or script that queries it before handing a prompt to your assistant of choice. Start there, measure before adding anything clever, and let the numbers tell you what to build next.