Retrieval methods & R&D

Each method below is a named step under python/pydocs_mcp/retrieval/steps/, addressable from YAML. The default chunk_search_graph.yaml composes single-vector dense retrieval with reference-graph expansion (graph_expand) — on the RepoQA benchmark this lifts recall@10 from 0.40 (keyword-only) to 0.77 on standard queries and to 1.00 on structurally-reachable answers. Everything else is opt-in via a preset swap (--config).

Keyword — BM25 over SQLite FTS5

Full-text search with porter stemming and the unicode61 tokenizer. Free, instant, and the baseline that every other method composes with through the fusion steps below.

Single-vector dense — FastEmbed + TurboQuant

  • Embedder. FastEmbed with BAAI/bge-small-en-v1.5 by default — runs on CPU via ONNX, no PyTorch, no torch download. OpenAI text-embedding-3-small is the optional alternative for users with an API key. Pass --gpu to run the on-device embedders (FastEmbed / sentence_transformers) on CUDA instead — same vectors, lower latency.

  • Bigger on-device model — the sentence_transformers provider. For stronger dense recall without an API key, switch to Qwen/Qwen3-Embedding-0.6B served via sentence-transformers (torch). It is GPU-reliable — torch frees CUDA memory between sequential index-builds — and the weights download at runtime on first use. Install the extra (pip install 'pydocs-mcp[sentence-transformers]', ~1-5 GB with torch; torchvision is not included — if a model load demands it, the construction-time error message walks through the remedies), then set it in your YAML:

    embedding:
  provider: sentence_transformers
  model_name: Qwen/Qwen3-Embedding-0.6B
  dim: 1024
  # Optional. Token cap (attention is O(seq^2) — the OOM guard). Omit to
  # use the embedder's own default (2048).
  max_seq_length: 2048
  # Optional. L2-normalize output (default true).
  normalize: true
  # Optional. Named asymmetric query prompt; omit to use the model's own.
  query_prompt_name: query

    The provider also runs ONNX / OpenVINO exports for fast CPU inference — typically 2–4× with a qint8-quantized file — via two optional keys (pip install 'pydocs-mcp[openvino]' for the OpenVINO runtime):

    embedding:
  provider: sentence_transformers
  model_name: BAAI/bge-small-en-v1.5
  dim: 384
  backend: openvino          # torch (default) | onnx | openvino
  model_file_name: openvino/openvino_model_qint8_quantized.xml

    Setting either key re-embeds on the next index (quantized vectors differ from full-precision ones); defaults leave existing indexes untouched.

    The provider supports several on-device models — set model_name and the matching dim:

    The default remains bge-small; the sentence_transformers provider is opt-in.

  • Air-gapped / offline deployments. Point embedding.model_name at a local directory of side-loaded weights (e.g. a git clone of the HF repo made on a connected machine) and nothing is downloaded — HF offline mode is forced, so a missing file fails locally instead of reaching for the network. Works for every provider: fastembed additionally needs the model’s recipe in YAML (pooling, normalize, model_file_name) since an arbitrary ONNX folder doesn’t carry it — and note fastembed pools only mean/cls, so last-token models like Qwen3-Embedding must use provider: sentence_transformers (which reads the recipe from the model directory itself). openai rejects a local path. Package mirrors: the [sentence-transformers] extra never pulls torchvision, so offline package mirrors need no torchvision wheel; if a model load does demand it, the mirror must add a torchvision wheel whose version exactly matches the mirrored torch wheel (e.g. torchvision 0.26.0 ↔ torch 2.11.0, 0.28.0 ↔ torch 2.13.0) — a skewed pair is unresolvable offline. See python/pydocs_mcp/defaults/default_config.yaml for full examples.

  • Vector store. TurboQuant (turbovec) — Online Vector Quantization with near-optimal distortion. ~16× smaller than float32 (a 1536-dim vector drops from 6,144 to 384 bytes; a 10 M-doc corpus fits in ~4 GB instead of ~61 GB) and faster than FAISS FastScan at the same recall. Persists as a .tq sidecar next to the SQLite DB.

  • Query-embedding cache. At serve time, repeated and concurrent identical queries are embedded once — an in-process LRU plus in-flight request coalescing, on by default and tunable (or disabled) via embedding.query_cache.* in your pydocs-mcp.yaml; a multi-repo workspace also shares a single embedder model load across all projects. Late-interaction query encodes get the same treatment via a separate, smaller-by-default late_interaction.query_cache.* block (per-token matrices are bigger entries).

Late-interaction (multi-vector / MaxSim) — opt-in

The flagship R&D backend. One vector per token instead of one pooled vector per chunk; queries score via ColBERT’s MaxSim — for each query token, take the maximum cosine to any document token, then sum. Higher recall on long, structurally distant queries (often the hard cases for single-vector retrievers).

  • Method. ColBERT late interaction (Khattab & Zaharia, SIGIR 2020).

  • Engine. PLAID (Santhanam et al., CIKM 2022) via fast-plaid — a Rust-backed IVF + residual-decompression engine. Persists as a per-project directory sidecar at ~/.pydocs-mcp/{slug}.plaid/.

  • Embedder. PyLate (arXiv:2508.03555) with the default model lightonai/LateOn-Code — late-interaction trained on code.

  • Lighter-weight model — lightonai/LateOn-Code-edge. For a smaller per-token footprint, point the same PyLate path at lightonai/LateOn-Code-edge (48-dim token vectors instead of LateOn-Code’s 128) in your YAML:

    late_interaction:
  enabled: true
  provider: pylate
  model_name: lightonai/LateOn-Code-edge
  embedding_dim: 48
  document_length: 2048
  query_length: 256

    The default stays LateOn-Code; LateOn-Code-edge is opt-in.

  • SQLite + fast-plaid coupling. A chunk_multi_vector_ids mapping table bridges SQLite’s chunk_id to fast-plaid’s plaid_doc_id. The shipped FilterAdapter Protocol pushes metadata filters down to SQLite, then the result chunk-id list is passed as subset= to fast-plaid’s MaxSim search — so MaxSim is always bounded to the SQLite-eligible candidates and the two engines stay in their own id spaces.

  • Enable. pip install 'pydocs-mcp[late-interaction]', set late_interaction.enabled: true in your YAML, then point --config at the shipped chunk_search_late_interaction.yaml preset.

Hybrid fusion

  • Reciprocal Rank Fusion (RRF) — Cormack, Clarke & Buettcher, SIGIR 2009. Rank-only 1 / (k + rank) with k=60 default; the workhorse for combining BM25 + dense, or BM25 + late-interaction.

  • Weighted Score Interpolation (WSI) — score-space α · score_a + (1 − α) · score_b with min-max normalization, for cases where the score distributions are well-calibrated and rank isn’t enough. α is tunable from YAML.

  • Post-fusion dense re-rank (dense_scorer) — an optional final step that takes the fused candidate list and re-scores just that subset against the TurboQuant vectors (an allowlist search, no fresh ANN scan), sorting the vector-scored hits to the top. Candidates with no dense vector — BM25-only, or skipped by the selective-embed policy — keep their fused order and trail behind, so recall is preserved while the embedded results get the sharper ordering. Mirrors the late-interaction scorer on the single-vector side.

LLM tree reasoning — opt-in

A vectorless mode for broad, structural questions (“walk me through the request lifecycle”). Instead of embedding text, an LLM walks the code map — module / class titles plus short summaries — and picks the best spots itself. Inspired by PageIndex (VectifyAI)’s reasoning-over-tree-of-contents approach.

Three shipped presets under python/pydocs_mcp/pipelines/: tree_only.yaml, chunk_search_with_tree_reasoning_parallel.yaml (run alongside chunk search, fuse via WSI), and chunk_search_with_tree_reasoning_after.yaml (use chunk search as the candidate pool, let the LLM re-rank). Provider / model / temperature / max_tokens are tuned under the llm: section of YAML; any OpenAI-compatible endpoint works.

Code reference graph

Beyond embeddings, pydocs-mcp captures a graph of how code references code during indexing: CALLS, IMPORTS, INHERITS, and optional MENTIONS (backtick-quoted dotted names in markdown). The same surface answers an AI’s “what calls this?” / “what does this extend?” questions through the get_references(direction=…) MCP tool — callers, callees, inherits, impact (everything that transitively depends on a symbol), and governed_by (which recorded decisions govern it):

pydocs-mcp refs requests.auth.HTTPBasicAuth --direction inherits
pydocs-mcp refs my_module.Parser.parse --direction callers

Capture is on by default and tunable under reference_graph: in YAML (toggle, kinds-to-emit, output bounds).

The graph is also a search signal, not just a navigation surface: the chunk_search_graph.yaml preset seeds graph expansion from the top dense hits to recover structurally-adjacent answers a dense embedder misses (callers / callees / overrides) — on a structural-recall split this lifts recall@10 from 0.30 to 1.00 (see benchmarks). The graph_expand step’s kind_weights YAML knob assigns a per-edge-kind trust so a weak signal (say MENTIONS) can be traversed but discounted — its weight compounds along each expansion path. Two opt-in index-time analytics (reference_graph.node_scores / reference_graph.similar_edges, [graph] extra) add PageRank/community rerankers and synthetic embedding-kNN edges — see DOCUMENTATION.md.

Architectural decisions — the why behind the code

Reading code tells your agent what it does; it rarely tells it why. During indexing (your project only), pydocs-mcp mines architectural decisions from the artifacts that already record them — ADR files, inline decision markers, commit messages, the changelog, and prose docs — deduplicates near-identical findings, and stores each as a first-class, searchable record. An optional LLM pass structures a chosen record into fields (context / decision / consequences) when you turn it on.

Two surfaces expose them:

  • get_why — ask “why is this the way it is?” by free-text query or by target symbol/file, and get the governing decisions back:

    pydocs-mcp why "why do we cache embeddings per chunk"
pydocs-mcp why --target pydocs_mcp.storage.sqlite.chunk_repository

  • search_codebase(kind="decision") — search the mined decisions directly, alongside the usual docs / api kinds (pydocs-mcp search "vector store choice" --kind decision).

Each decision also becomes a graph node linked to the symbols it affects, so get_references(direction="governed_by") traces from a symbol back to the decisions that govern it. Capture is on by default and tunable under decision_capture: (which sources run, dedup threshold, the optional LLM structuring); read-side output bounds live under decisions.output.