Live re-indexing

The file-system watcher re-triggers indexing on edits so subsequent queries see fresh data. Two modes are available, each tuned for a different workflow:

pydocs-mcp serve . --watch   # MCP server + watcher (for AI clients connected over stdio)
pydocs-mcp watch .            # watcher only (no MCP server; index stays fresh for CLI `search` / `symbol` / `refs`)

Use serve --watch when an AI client (Claude Code, Cursor, Continue.dev) is connected over stdio and you want the index to refresh as you edit. Use watch when you don’t need an MCP server running — for example, you prefer the CLI search / symbol / refs commands, or you want to keep the index fresh from an IDE-driven workflow without leaving an idle FastMCP stdio process. Both modes share the same YAML knobs under serve.watch.*.

Without either mode, the server (or pydocs-mcp index) indexes once and exits — today’s behavior, unchanged.

Install

The watcher uses watchdog, which is part of the default install:

pip install pydocs-mcp
pydocs-mcp serve . --watch    # or:
pydocs-mcp watch .

The [watch] extra from older install instructions remains as a deprecated no-op alias, so existing commands and scripts that request it keep resolving — it installs nothing beyond the default set.

How it works

  1. The watcher monitors the project root (NOT site-packages/, which is under the ignored .venv). It fires on edits to the file types the project scope indexes (extensions, which by default follows extraction.discovery.project.include_extensions) and to dependency manifests (pyproject.toml / requirements*.txt) — so adding a package (e.g. uv add X, which edits pyproject.toml) reindexes and picks up the new dependency once it’s installed.

  2. A file-system event is queued when the path clears three filters, in order:

    • Directory exclusions. Anything under a directory project discovery never indexes is dropped — its fixed exclusion floor (build output such as target/, dist/, build/, htmlcov/, tool caches such as .tox/ and .mypy_cache/, vendored trees such as extern/ and third_party/) and your own exclude_dirs from either surface, matched with the same predicate the discovery walk uses. So a compiler or bundler writing its output never triggers a reindex, and neither does a manifest in one of those trees — dependency discovery is handed the same exclusions and would read no package from it. Both apply below the project root only and need no ignore_globs entry.

    • File type. The path matches extensions, or it is a dependency manifest (pyproject.toml / requirements*.txt) — manifests are exempt from extensions, and from that alone.

    • ignore_globs. Operator-authored fnmatch patterns, applied to the absolute path.

  3. Events are debounced by debounce_ms — N edits within the window collapse into a single reindex. Editor atomic-save sequences (temp create → delete → rename) naturally fall under the same trigger.

  4. Edits arriving during an in-flight reindex schedule exactly one follow-up reindex (no thundering herd from git checkout / git rebase rewrites).

  5. The two-level cache (packages.content_hash + chunks.content_hash) makes the no-change case <100 ms; only modified packages are re-extracted, only added/changed chunks are re-embedded.

YAML knobs (serve.watch.*)

All tunables live in YAML — no MCP tool params change (the MCP surface stays fixed at the nine task-shaped tools). Either switch enables watching: the CLI --watch flag or serve.watch.enabled: true in YAML (the flag cannot force watching off when the key is true).

# pydocs-mcp.yaml
serve:
  watch:
    enabled: false              # either this key or the CLI --watch flag enables watching
    debounce_ms: 500            # 1 .. 59_999 ms (must be < 60_000); 500ms is editor-safe
    extensions: null            # null = follow extraction.discovery.project.include_extensions
    ignore_globs:
      - "**/__pycache__/**"
      - "**/.git/**"
      - "**/.venv/**"
      - "**/node_modules/**"
      - "**/.pytest_cache/**"
      - "**/*.pyc"

extensions: null (the default) makes the watcher follow the project discovery scope: it reacts to every file type a project index pass reads, so narrowing extraction.discovery.project.include_extensions narrows the watch set too. List extensions under serve.watch.extensions (for example [".py", ".md"]) to override the project scope.

Trade-offs

  • Memory + one OS event handle for the watcher process — small, but headless / CI deployments that never edit code should leave --watch off.

  • Brief query-latency hit during reindex — SQLite WAL mode allows concurrent readers, so MCP queries continue serving stale-but-correct data while the reindex transaction commits.

  • Reindex failures are logged but do not crash the MCP server — the server keeps serving the previous index. Check the logs if results look stale.