Samuel Herman

Writing

Writing

Retrieval at scale

Six pieces, written in order, that form one argument. It starts at the hardware and works up through storage and index construction, then turns to the embedding model, the economics of running it, and the geometry that decides which index will work at all. The paper on the research page is where the argument currently ends.

  1. Part 1

    Before adding an approximate index, check whether brute-force KNN is affordable on the hardware you already have. Measured CPU and GPU throughput, not vendor claims.

    March 2025 · 12 min

    KNN Part I: Are GPUs Good For Vector Search?

  2. Part 2

    At a billion vectors the bottleneck is disk I/O, not compute. Product quantization is measured against what it promises.

    June 2025 · 9 min

    The Reality of Vector Quantization: Does Product Quantization Deliver on Its Promises?

  3. Part 3

    Graph index construction can be lock-free and near-linearly scalable without losing search quality. Shipped in jVector for OpenSearch.

    August 2025 · 7 min · also on the OpenSearch blog

    Breaking the Single-Thread Bottleneck: Concurrent Vector Graph Construction in OpenSearch

  4. Part 4

    "Just fine-tune it" is bad advice for domain vocabulary. The failure is visible at the MLM logit level, and a staged adaptation pipeline fixes it.

    December 2025 · 25 min

    Teaching Embedding Models New Words: A Deep Dive into Domain Adaptation

  5. Part 5

    Past roughly a hundred million vectors, storage and retrieval cost dwarfs inference. The database, not the model, decides whether a retrieval system is viable.

    February 2026 · 30 min

    When the Database Becomes the Bottleneck: The Economics of Vector Retrieval and the Technologies That Change the Math

  6. Part 6

    PLAID and MUVERA are products of a specific dataset geometry and a specific similarity function. When either changes, the math stops working in your favor.

    February 2026 · 25 min

    When Geometry Is Everything: Multi-Vector Retrieval Beyond the MaxSim Monoculture