Gnosis

<a href="#quick-start"><img src="https://raw.githubusercontent.com/nicholasglazer/gnosis-mcp/main/demo/demo-hero.gif" alt="Gnosis MCP — ingest docs, search, view stats, serve" width="700"></a> <br> _{Ingest docs → Search with highlights → Stats overview → Serve to AI agents}

Without a docs server

• LLMs hallucinate API signatures that don't exist
• Entire files dumped into context — 3,000–15,000 tokens per doc
• Architecture decisions buried across dozens of files
• Every repeated lookup pays full context cost

With Gnosis MCP

• search_docs returns ranked, highlighted excerpts — typically 300–800 tokens
• Real answers grounded in your actual docs, not guesses from training data
• One local index across hundreds of files — instant multi-doc search
• 5–10× token savings per lookup when your corpus covers the question

What makes gnosis-mcp different

• Your data stays on your machine. SQLite by default, PostgreSQL at scale — nothing leaves the host.
• Index anything that's docs-shaped. Markdown, git commit history, crawled websites — one index, one search API.
• Measured, not marketed. Ships BEIR SciFact numbers (0.671 nDCG@10 — within 1 % of the Lucene BM25 baseline), a reproducible eval harness (gnosis-mcp eval), and a chunk-size sweep showing where the quality plateau actually sits.

Full side-by-side vs Context7 / docs-mcp-server / mcp-local-rag: gnosismcp.com#compare.

Features

• Zero config — SQLite by default, pip install and go
• Hybrid search — keyword (BM25) + semantic (local ONNX embeddings, no API key). Tune RRF fusion with GNOSIS_MCP_RRF_K.
• Cross-encoder reranking — optional [reranking] extra with a 22M-param ONNX model. Off by default. Test on your own corpus before enabling — the bundled MS-MARCO reranker hurts dev-doc retrieval in our measurements.
• Git history — ingest commit messages as searchable context (ingest-git)
• Web crawl — ingest documentation from any website via sitemap or link crawl
• Multi-format — .md .txt .ipynb .toml .csv .json + optional .rst .pdf
• Auto-linking — relates_to frontmatter creates a navigable document graph
• Watch mode — auto-re-ingest on file changes
• Prune stale docs — gnosis-mcp ingest --prune removes chunks whose source file was deleted. --wipe for a full reset before re-ingest.
• Built-in eval harness — gnosis-mcp eval prints Hit@K / MRR / Precision@K in one command
• PostgreSQL ready — pgvector + tsvector when you need scale

Performance

Fast. 8.7 ms mean MCP round-trip. Hybrid search p50 < 30 ms on a 700-doc corpus. Keyword QPS scales from 9,463 @ 100 docs to 471 @ 10,000 docs (full numbers).

Finds the right answer. On 558 real dev docs with 25 hand-written golden queries: Hit@5 = 0.92, nDCG@10 = 0.87, MRR = 0.79. On BEIR SciFact (5,183 docs, public retrieval benchmark): nDCG@10 = 0.671 — within 1 % of the Lucene BM25 baseline.

Tokens saved. Each search_docs call returns 200–500 tokens of on-point snippets instead of the 3,000–15,000 tokens a full-file Read would have cost. Track your own with gnosis-mcp savings (v0.12.0+) — the ledger writes to search_access_log on every call and aggregates per tool per --days N:

$ gnosis-mcp savings --days 7
  Tool calls:               142
  Tokens returned:        7,104
  Tokens baseline:      231,580
  Tokens saved:         224,476
  Ratio:                   32.6×

Typical compression runs 10–60× depending on corpus coverage and query specificity — verify on yours. access_log is on by default; GNOSIS_MCP_ACCESS_LOG=false opts out.

Reproducible. gnosis-mcp eval runs a RAG eval harness locally in one second. tests/bench/*.py reproduce every number. Methodology: docs/benchmarks.md.

Rerankers stay off by default. The bundled MS-MARCO cross-encoder drops nDCG@10 by 27 points on dev-docs and adds 400× latency; BGE-reranker-v2-m3 drops it 31 points at 2400×. Test on your corpus before enabling — full write-up: bench-experiments-2026-04-18.

Quick Start

pip install gnosis-mcp           # or: uv tool install gnosis-mcp
gnosis-mcp ingest ./docs/        # loads docs into SQLite (auto-created)
gnosis-mcp serve                 # starts MCP server

That's it. Your AI agent can now search your docs.

Connect your editor — see llms-install.md for copy-paste JSON snippets for Claude Code, Claude Desktop, Cursor, Windsurf, VS Code, JetBrains, and Cline.

Re-organized your docs? gnosis-mcp ingest ./docs --prune re-ingests and removes any DB chunk whose source file no longer exists. --wipe resets the entire index first. Or run gnosis-mcp prune ./docs --dry-run to preview what would be deleted.

Want semantic search? Add local embeddings — no API key needed:

pip install gnosis-mcp[embeddings]
gnosis-mcp ingest ./docs/ --embed   # ingest + embed in one step
gnosis-mcp serve                    # hybrid search auto-activated

Test it before connecting to an editor:

gnosis-mcp search "getting started"           # keyword search
gnosis-mcp search "how does auth work" --embed # hybrid semantic+keyword
gnosis-mcp stats                               # see what was indexed

Multi-arch image, ~140 MB, ships with local ONNX embeddings + REST:

# Serve your ./docs on http://localhost:8000 — MCP at /mcp, REST at /api/*
docker run -p 8000:8000 \
  -v "$PWD/docs:/docs:ro" -v gnosis-data:/data \
  ghcr.io/nicholasglazer/gnosis-mcp:latest

# First-run: ingest into the persistent volume
docker run --rm \
  -v "$PWD/docs:/docs:ro" -v gnosis-data:/data \
  ghcr.io/nicholasglazer/gnosis-mcp:latest \
  ingest /docs --embed

Or use the committed docker-compose.yaml:

docker compose up -d
docker compose exec gnosis gnosis-mcp ingest /docs --embed

Images tagged :latest, :<version>, :<version-minor>, :main, :sha-<sha>.

uvx gnosis-mcp ingest ./docs/
uvx gnosis-mcp serve

Web Crawl

Ingest docs from any website — no local files needed:

pip install gnosis-mcp[web]

# Crawl via sitemap (best for large doc sites)
gnosis-mcp crawl https://docs.stripe.com/ --sitemap

# Depth-limited link crawl with URL filter
gnosis-mcp crawl https://fastapi.tiangolo.com/ --depth 2 --include "/tutorial/*"

# Preview what would be crawled
gnosis-mcp crawl https://docs.python.org/ --dry-run

# Force re-crawl + embed for semantic search
gnosis-mcp crawl https://docs.sveltekit.dev/ --sitemap --force --embed

Respects robots.txt, caches with ETag/Last-Modified for incremental re-crawl, and rate-limits requests (5 concurrent, 0.2s delay). Crawled pages use the URL as the document path and hostname as the category — searchable like any other doc.

Git History

Turn commit messages into searchable context — your agent learns why things were built, not just what exists:

gnosis-mcp ingest-git .                                  # current repo, all files
gnosis-mcp ingest-git /path/to/repo --since 6m           # last 6 months only
gnosis-mcp ingest-git . --include "src/*" --max-commits 5 # filtered + limited
gnosis-mcp ingest-git . --dry-run                         # preview without ingesting
gnosis-mcp ingest-git . --embed                           # embed for semantic search

Each file's commit history becomes a searchable markdown document stored as git-history/<file-path>. The agent finds it via search_docs like any other doc — no new tools needed. Incremental re-ingest skips files with unchanged history.

Editor Integrations

Add the server config to your editor — your AI agent gets search_docs, get_doc, and get_related tools automatically:

{
  "mcpServers": {
    "docs": {
      "command": "gnosis-mcp",
      "args": ["serve"]
    }
  }
}

Add to .vscode/mcp.json (note: "servers" not "mcpServers"):

{
  "servers": {
    "docs": {
      "command": "gnosis-mcp",
      "args": ["serve"]
    }
  }
}

Also discoverable via the VS Code MCP gallery — search @mcp gnosis in the Extensions view.

Transport

Stdio (default) spawns one server per editor session — simplest. HTTP shares one process across every client so the DB, embedding cache, and file watcher stay in sync across sessions:

gnosis-mcp serve --transport streamable-http --host 0.0.0.0 --port 8000

{ "mcpServers": { "docs": { "type": "url", "url": "http://127.0.0.1:8000/mcp" } } }

Pick HTTP for multi-session agent setups (Claude Code with agent teams, parallel terminals, CI). Full write-up: gnosismcp.com/doc/docs/deployment.

REST API

v0.10.0+ — HTTP endpoints alongside MCP on the same port.

gnosis-mcp serve --transport streamable-http --rest

CORS, Bearer auth, custom public-path allowl

…