A non-tech problem deserves a non-tech solution
Last time I wrote here, the problem was mine and the solution was deep in a system: a Postgres proxy sitting in the data path. This one is the opposite shape. The problem belonged to someone who doesn't write code, the whole point was that the solution had to stay that way, and the interesting part isn't a wire protocol, it's what embeddings quietly let you get away with.
I've always been a little fascinated by lawyers. The job is, at its core, retrieval under pressure: find the right authority, the exact clause, the case on point, and be ready to defend why it applies. It's a profession that runs on a vast body of text and on knowing where to look in it. Which is exactly the kind of work where a good tool stops being a nice-to-have and starts changing the day. I think law is one of the fields where AI buys the most productivity the fastest, not by being clever, but by collapsing the distance between a question and the paragraph that answers it. You don't need a model that reasons like a jurist. You need one that can find the right three paragraphs out of a few million and hand them over with a page number.
A friend of mine put that belief to the test without meaning to. He's a lawyer who uses Claude the way a lot of professionals now do, as a smart colleague he talks to in plain French, and he has about 9 GB of French legal PDFs, a working library built up over years, most of them scanned. He'd been trying to use them the obvious way, convert a document to markdown, paste it into the conversation, ask his question. It works for a memo. It falls apart on a 3,600-page volume, because you cannot put a 3,600-page book inside a context window, and even if you could, you'd pay for all of it on every single question to use a paragraph of it. He could feel the wall without knowing its name.
A note on tooling, because I said I'd always be straight about it. I did this with Claude Code over an evening, /goal and a swarm of subagents as usual. What's different from last time isn't the technique, it's who the work was for. Sluss was me learning a system I wanted to understand. This was me solving a real person's real problem, and the constraint that made it interesting was non-technical: every decision had to survive the question "can someone who has never opened a terminal actually use this?" That constraint is more demanding than any benchmark. As before, I won't tell you which model. The point is the same: an evening means more than it used to.
The wall is called context, and you don't climb it, you go around
The instinct when documents don't fit is to ask for a bigger window. It's the wrong instinct, and seeing why is most of the insight.
9 GB of extracted legal text is on the order of tens of millions of tokens. That is not "a bit over the limit," it's two or three orders of magnitude past the largest context window that exists, and a single one of his books can blow the budget on its own. There is no near-future window size that makes "just paste it all in" the answer. Worse, context you fill is context you pay for and wait on, every turn, whether or not the model needed it. Stuffing a window is the most expensive possible way to ignore 99% of what you stuffed in.
So you go around. The shape that works is the one search engines have used forever and that the LLM era renamed RAG: keep the documents outside the model, and at question time pull in only the handful of passages that actually bear on the question. The model never sees the library; it sees the few excerpts that matter, each tagged with where it came from. His instinct to convert the PDFs to text wasn't wasted, by the way; clean text is exactly what you need. He was just putting it in the wrong place, the prompt instead of an index.
Embeddings are the whole trick
Here's the part I find quietly beautiful, and it's worth saying plainly because it's easy to lose in the acronyms.
An embedding model reads a chunk of text and returns a vector, a list of a thousand-odd numbers that is a coordinate in a space where meaning is geometry. Passages that say similar things land near each other, even when they share no words. "Délai de recours contentieux" and a paragraph about the two-month window to challenge an administrative decision are close in that space, though a keyword search would miss the link entirely. You embed every chunk of the corpus once, ahead of time. Then at question time you embed the question the same way and ask: which chunks are nearest? That nearest-neighbour lookup over vectors is semantic search. No model is "reading" the library to answer; it's a distance calculation.
I used BGE-M3 for the embeddings. It earns its place here for two unglamorous reasons. It's genuinely multilingual, which matters when the corpus is French legal prose. And it produces a dense vector and a sparse, lexical one from the same model, which let me get both kinds of matching from one pass. That last point is not academic. Law lives on exact tokens: an article number, a "§ 12.3", a party name, a case number. Pure semantic search is a little too relaxed about those; it will hand you something about liability when you asked for the precise indemnification clause. Keyword search nails the exact token but misses the paraphrase. You want both, and you fuse the two ranked lists. The dense vector finds "this is about the same idea," the sparse one insists "and it contains literally L. 242-2," and the combination is what makes a legal answer trustworthy instead of merely plausible.
Local is a feature, not a compromise
Every vector and every document sits in a single SQLite file on his Mac. sqlite-vec holds the dense vectors and does the nearest-neighbour search; SQLite's built-in FTS5 does the keyword side; a small table holds the catalog. One file, no server, no database to run, trivial to back up. When he asks a question, the search happens entirely on his laptop; the only thing that leaves the machine is the three or four short passages that get sent to Claude to write the answer.
It would have been easy to reach for a hosted vector database here. For one lawyer's library it would also have been worse: a monthly bill, an account to manage, documents leaving his control, and a moving part that can break on a Tuesday. The honest reason local wins isn't ideology, it's that for this size it's simply less. Less to install, less to pay, less to explain, less to go wrong. The numbers are small enough that "small enough to keep on the laptop" is the right engineering answer and the right human answer at the same time.
The one thing that genuinely isn't free is the scanned text. Most of these PDFs are images with an OCR layer of wildly varying quality, and a chunk of bad OCR (1:1x::eur dene la choix du pr-Sfat where the page says l'erreur dans le choix du préfet) poisons both the search and the citation. So the pipeline checks each file's text quality with two cheap heuristics, one for the words-fused-together failure and one for the character-soup failure, and only re-runs OCR on the files that fail. That re-OCR is the single heavy step, embarrassingly parallel, and the kind of one-time batch you do once and never again.
Build it where the compute is, run it where the person is
Indexing 9 GB once is real work: re-OCR a few thousand pages, embed about 122,000 chunks. That is a bad fit for a laptop and a perfect fit for a machine you rent for an hour. So the build ran on a 96-core box, and the asymmetry is the nice part: you ship 9 GB up, you do the heavy work, and what comes back is an 800 MB SQLite file. The expensive, parallel, throwaway part happens on hardware sized for it; the thing that lives on his Mac is small. Build where the compute is, run where the person is. The instance was torn down the same evening.
One honest note from that build, in the spirit of "RDS edged Aurora by 6%, expected, because." Embedding 122,000 chunks on CPU is just slow, slower than I'd have liked, because a single embedding process can't saturate a big box. The fix was the same as every other "go around the bottleneck" in these posts: shard the work across many processes and let them run in parallel. It's not a clever trick. It's the obvious trick, applied without ego. (A GPU box would have been the genuinely fast answer; the corpus didn't justify one.)
The part that had to stay boring
This is where the project differs from a system experiment. Sluss could end at "here's the benchmark." Here, a working index that a non-technical person can't install is worth nothing. The deliverable wasn't the index, it was the installing.
Claude Desktop turns out to support exactly the right thing: a bundle format (.mcpb) where you double-click a file, click Install, and it wires a local tool into the app for you. No config files, no JSON with absolute paths, no Python to install by hand. The dependencies are declared, not bundled, and the runtime fetches the right platform-specific wheels on his machine at install time, which is the detail that makes "double-click and it works" actually true across Macs instead of true on mine.
My first instinct was to put everything in that one file, code and 800 MB index together, and ship a single artifact. The bundle rejected it: an .mcpb caps any single file inside it at 512 MB, and the index is 800. That limit turned out to be a gift. The right shape was obvious once the wrong one was blocked: ship a tiny bundle (the code is a few kilobytes) and have it pull the heavy pieces, the index and the embedding model, on first run into a cache next to the app. Smaller artifact, nothing awkward in version control, and the same path scales the day his library outgrows whatever an arbitrary cap happens to be. The bundle is a stub; the big files live at a URL and are fetched once, then never again. After that he opens Claude and asks questions in French, and gets answers with a citation he can verify, page and paragraph number, like a memo from a junior who actually read the books. I wrote him a one-page guide with a FAQ. The longest item on it is "the first question is a little slow, that's normal," which is now literally true: the first question is what triggers the one-time download.
I'll be straight about the seam I haven't closed. Adding his own new PDFs later is harder than querying, because OCR needs native tools that don't ride along in the bundle, and re-embedding on a laptop is slow. The clean answer is the same trick again, send new documents to a cheap one-hour cloud build and sync the small index back, but I haven't built it yet. Querying the library he has is done and it works; growing the library is v2.
What the build got wrong
Two failures worth writing down, both in the family of "the model tells a clean story; the artifact is the only thing that's true," which is becoming the throughline of these posts.
The first re-OCR pass produced zero text and didn't error. Tesseract was being handed a PNG it couldn't read on this machine, failing, and the failure was getting swallowed. The agent's narration was happily green. The bytes on disk said zero characters. You only catch that by looking at the output, not the status. The fix was boring (render to TIFF, capture the real error), but the lesson is the recurring one: verify the artifact.
The second was mine to catch and I almost didn't. The first parallel build farmed one whole document per worker, which is fine until the last document is a 3,600-page book and ninety-five cores sit idle while one grinds through it alone. The tell was the load average quietly dropping to 1 while the job claimed to be busy. The fix was to split the work by page-range instead of by file, so the big book spreads across all the cores like everything else. It's a lesson that keeps re-teaching itself: the unit of parallelism has to match where the work actually is, not where it's convenient to draw the boundary.
Closing the file
You don't beat a context limit by asking for more context. You notice that the question only ever needs a few paragraphs, and you build the thing that finds those paragraphs. Embeddings make that finding semantic instead of literal; a sparse signal alongside keeps it honest about the exact tokens law depends on; and at this scale the whole apparatus fits in one file on one laptop, which is the right answer for a single user in every way that matters.
It's worth saying the bigger version of this out loud, carefully, because it's easy to get carried away. Much of what makes legal help expensive isn't the judgment, it's the lookup: the hours spent finding the right authority and assembling what's relevant before anyone can advise on it. That layer is genuinely commodifiable, and when it gets cheaper two things follow. Lawyers spend more of their time on the part that is actually theirs, the judgment and the strategy and the responsibility for the advice; and legal help gets a little more reachable for the people who can't currently afford the lookup. None of it replaces the lawyer. A passage with a page number is not advice; someone still has to decide what it means and whether it applies. So I don't think the promise is AI making judicial calls, which it shouldn't. It's AI making the raw material of justice cheaper to reach, and that strikes me as one of the more quietly useful things this technology can do for a profession, and for the people who depend on it.
The arc is what I keep turning over, same as last time but from the other end. Before, a Sunday experiment became a benchmark I ran against rented hardware to see how far it would push. This time it became something smaller and, to me, more satisfying: a non-technical person's non-technical problem, met with a tool he installs by double-clicking, that keeps his documents on his own machine and answers him with a citation. The technology underneath is real (embeddings, hybrid retrieval, a vector index, a rented machine for an hour). The experience on top is a lawyer asking a question and getting an answer with a page number. Keeping those two facts true at the same time was the whole job.