embedding

One pruned line in the morning build settled it: an unpruned vector index answers with documents that no longer exist, and free-form data is a contract.

TL;DR: Renaming a research file left a stale embedding in a JSON vector index that kept returning a ghost document. The fix added tolerant reading for summary field aliases, automatic pruning of missing files during builds, and a self-testing suite. It is a sixty-line reminder that small local tools need honest indexes and defensive boundaries.

A cosine-similarity duplicate gate is only as good as its vector coverage. Lessons from a 32-article blind spot and a self-measured 0.50 cutoff.

TL;DR: I nearly trusted a duplicate check running on a half-populated index, since 164 of 196 articles had vectors and the missing 16% never showed up. The fix was a backfill plus a self-check confirming each article's own title returns its slug. The 0.50 cutoff isn't universal either; measure your corpus's own score distribution first.

Queries returned just a path and a score. So I stole OpenViking's L0 summary pattern: one line per dossier, no new infrastructure.

TL;DR: Search results in my knowledge base showed only scores, so I kept opening dossiers just to judge relevance. Rather than install OpenViking, I stole its L0 tier pattern and added a mandatory one-line summary field to each dossier in about 40 lines of Python. Making summaries a hard requirement keeps them consistent, no new dependency needed.

Pure-Python cosine, vectors in MariaDB postmeta, embeddings computed locally. Semantic features with no new service on a small server.

TL;DR: Running a blog on a 1GB VPS rules out a vector database, so the author computes bge-m3 embeddings locally via Ollama and stores them as JSON in MariaDB postmeta. Related-article matching and dup-checks use pure-Python cosine similarity, fast enough for around 100 articles. Lesson: for small blogs, skip Qdrant or pgvector until scale demands it.

The semantic related layer for 80 articles turned out to be a Python dict and a cosine loop, no numpy and no vector database.