kb — comprehensive knowledge boss

kb is a knowledge base over code, docs, tasks, and chats.

Results at a glance

Feature ladder on a 60-question EnterpriseRAG-Bench slice (conflicting_info / constrained / basic), each stage adds one capability. DCS (Dynamic Currency Score) is the share of answers that name the current value rather than a superseded one, on 11 conflicting_info questions.

#	Stage	Adds	DCS	stale_leak	miss	facts	retrieval_hit
0	Native	dense-only, one LLM call	0.18	0.09	0.73	0.23	54/60
1	+Hybrid	dense + BM25 + RRF	0.27	0.09	0.64	0.20	57/60
2	+Graph	graph-aware fusion	0.27	0.18	0.55	0.23	57/60
3	+Rerank	LLM rerank	0.27	0.27	0.45	0.21	57/60
4	+Logic	Graph-of-Thoughts	0.58 (0.45-0.64)	0.06	0.24	0.24	58-60/60
5	+Temporal	strict supersede + contradiction detection	0.61 (0.55-0.64)	0.15	0.15	0.21	59-60/60
6	+Qualifiers	qualifier filter	0.55 (0.36-0.73)	0.12	0.24	0.25	59-60/60

Stages 0-3 are a single run; stages 4-6 are the mean of three identical runs (range in brackets).

Read it as a pattern, not a ranking, on 11 questions (one question moves DCS by 0.09). The single run that first ranked stages 4-6 as 0.45 / 0.64 / 0.55 was noise: three repeats of each overlap almost entirely (qualifiers alone ranges 0.36-0.73). What the data supports: the Graph-of-Thoughts stages (means 0.55-0.61, lowest single run 0.36) sit above the naive stages (0.18-0.27, one run each, so their own variance is unmeasured); temporal and qualifiers show no gain over plain GoT; retrieval is not the bottleneck (54-60/60 at every stage). Method, per-question results and caveats: docs/bench/erb-evolution-report.md.

Requirements

• Go 1.26+ to build from source (or use a prebuilt release binary).
• An OpenAI-compatible LLM endpoint, exposing chat completions at /v1/chat/completions and embeddings at /v1/embeddings.

Quickstart

Point kb at your OpenAI-compatible endpoint, build, and serve:

git clone https://github.com/alterfo/kb.git
cd kb
make build
export KB_LLM_BASE_URL=http://<your-llm-host>:11434   # chat + completions
export KB_EMBED_BASE_URL=http://<your-llm-host>:11434  # embeddings (same host is fine)
export KB_LLM_MODEL=qwen3.8:latest
export KB_EMBED_MODEL=qwen3-embedding
./bin/kb serve

Or install the latest release directly:

go install github.com/alterfo/kb/cmd/kb@latest

Open http://127.0.0.1:8080. By default serve binds to loopback and has no authentication; binding a non-loopback -addr requires KB_WEB_AUTH_TOKEN (refused otherwise) and enables bearer/X-KB-Token/cookie auth plus an optional KB_WEB_RATE_LIMIT. See Configuration below for the full environment surface and Usage for each CLI command.

Architecture

flowchart TD
    src["External sources<br/>GitHub / GitLab / Wiki / MCP / chats / trackers / files<br/>PDF / XLSX / JSON / SQL DDL"] --> fetch["Connector.Fetch → Document (chan)"]
    fetch --> render["render:<br/>Document → markdown + YAML frontmatter"]
    render --> sink["sink: FileSink / APISink / TeeSink<br/>state: .sync-state.json, tombstones.json"]
    sink --> idx["engine (indexer):<br/>AddOrUpdateDocument / RemoveDocument / Reindex"]

    idx --> chunk["chunk:<br/>sentences + ChatChunker"]
    idx --> graphn["graph:<br/>LLM extraction of entities / relations<br/>→ merge / dedup → communities (Louvain) → summaries"]

    chunk --> vstore[("VectorStore<br/>embeddings BLOB, brute-force cosine")]
    chunk --> bm25[("Lexical index<br/>SQLite FTS5 (default) or in-memory BM25")]
    graphn --> gstore[("GraphStore<br/>entities / relations / communities in SQLite")]

    vstore --> retr["retriever.Retriever:<br/>hybrid + graph-aware fusion<br/>dense multi-query + BM25 + RRF + authority + per-doc cap<br/>→ entity-linking → neighbor expansion → community context"]
    bm25 --> retr
    gstore --> retr

    retr --> rerank["rerank.Reranker:<br/>noop / llm / onnx (optional, fail-open)"]
    rerank --> got["got.Orchestrator:<br/>Graph-of-Thoughts<br/>decompose → DAG → waves → aggregate → gaps → finalize"]

    got --> mcp["internal/mcp:<br/>MCP server, stdio + HTTP"]
    got --> web["internal/web:<br/>dashboard, html/template + htmx + SSE"]

Loading

Persistence is a single file $PERSIST_DIR/kb.db: vector tables (chunks), graph tables (entities/relations/communities), kb_meta/corpus_version, and search_history/ask_runs (dashboard search and ask history) in one SQLite database. The lexical index defaults to a SQLite FTS5 table over chunks (KB_FTS5=false reverts to the legacy in-memory BM25 index, which is rebuilt from chunks whenever corpus_version changes).

See docs/architecture.md for the pipeline in detail.

Configuration

Copy .env.example to .env and adjust as needed (or export the variables directly). Most settings have defaults and live in internal/config/env.go; connector-level options such as KB_SOCKS_PROXY are read directly by the connector that needs them (Discord).

Variable	Default	Meaning
KB_ROOT	./kb_root	Root directory for ingested markdown documents
PERSIST_DIR	./kb_root/.persist	Directory for kb.db, .sync-state.json, tombstones
KB_LLM_BASE_URL	http://127.0.0.1:11434	OpenAI-compatible endpoint base URL for the chat/completions model (entity/relation extraction, answer synthesis, rerank). Point at your model host.
KB_EMBED_BASE_URL	http://127.0.0.1:11434	Query-time embeddings (retrieval) — same host as KB_LLM_BASE_URL is fine.
KB_EMBED_INDEX_BASE_URL	KB_LLM_BASE_URL	Bulk indexing embeddings — defaults to the chat endpoint.
KB_LLM_MODEL	qwen3.8:latest	Chat model
KB_EMBED_MODEL	qwen3-embedding	Embeddings model (must support embeddings)
KB_HYBRID	true	Hybrid retrieval (dense + BM25 + RRF); false = dense-only
KB_RERANK	off	Reranker: off | llm | onnx
KB_AUTHORITY_BONUS	notes/=0.15,notes/approved/=0.30	Authority prior bonuses, prefix=bonus,...
KB_NO_PROXY	127.0.0.1	Comma-separated hosts that bypass HTTP(S)_PROXY (direct connection)
KB_TOP_K	10	Default top-K retrieval results
KB_CHUNK_SIZE	4096	Chunk size (tokens)
KB_CHUNK_OVERLAP	512	Chunk overlap (tokens)
KB_RRF_K	60	Reciprocal Rank Fusion constant
KB_CANDIDATE_K	20	Per-leg candidate window before fusion (bench tuning)
KB_PER_DOC_CAP	2	Max chunks per document in fused results
KB_SET_MAX_ROUNDS	3	Max query-variant rounds for set/count retrieval (ModeSet)
KB_QUALIFIER_FILTER	false	Extract structured metadata qualifiers from the question via one LLM call and filter every retrieval leg
KB_ABSTAIN_THRESHOLD	(off)	Float in (0,1]: answer "not found" when every subgoal is uncovered and average coverage is below the threshold
KB_SUPERSEDE_MODE	soft	soft = rank superseded docs lower; strict = drop a superseded doc from synthesis when its replacement is retrieved
KB_SUPERSEDE_MIN_SHARED	1	Minimum number of graph entities a new document must share with an older chunk to mark that chunk superseded at index time. 1 means any single shared entity (e.g. the project name) supersedes; raise to 3+ for corpora of related but complementary documents
KB_INTRA_DOC_BUDGET	(off)	Approx token budget for pulling sibling sections of winning documents into results (intra-document questions)
KB_STALE_AFTER	24h	Sync staleness threshold for doctor / /integrations
KB_COMMUNITY_ALGO	louvain	Community detection: louvain | leiden (Leiden is hierarchical)
KB_LLM_TIMEOUT	60s	Request timeout for embed + chat calls to the endpoint
KB_LLM_NO_THINK	false	Skip chain-of-thought generation on hybrid-reasoning models (e.g. Qwen3) for non-streaming, non-tool Chat() calls (graph extraction, GoT synthesis) by routing them to Ollama's native /api/chat with think:false instead of /v1/chat/completions — the OpenAI-compat endpoint ignores think:false on Ollama and still pays the full reasoning cost. Only affects the LLM pointed at by KB_LLM_BASE_URL; requires that endpoint to be Ollama, serving a hybrid-reasoning model. Some Ollama versions reject the think field outright for a plain (non-hybrid) model — only enable this when KB_LLM_MODEL actually supports thinking mode. Streaming (kb serve's Ask) and tool-calling requests are unaffected.
KB_MAX_SUBGOALS	5	Max GoT subgoals per question; lower cuts per-question LLM calls at some recall cost
KB_MAX_GAP_QUERIES	3	Max GoT gap-refine queries per question; lower cuts per-question LLM calls
KB_GOT_MAX_REFINE_LATENCY_MS	0 (unlimited)	Wall-clock budget in ms for a GoT run before its optional refine pass; if already exceeded when the refine decision is made, refine is skipped (fail-open) and degraded records refine_skipped_budget_exceeded
KB_DESCRIBE_MODEL	qwen3.8:latest	Chat model used by kb describe (independent of KB_LLM_MODEL)
KB_DESCRIBE_BATCH	10	Batch size for kb describe summary generation
KB_SOCKS_PROXY	(unset)	SOCKS5 proxy (socks5://host:port) used by connectors that need it (e.g. Discord)
KB_FTS5	true	Lexical index backend: SQLite FTS5 (default) vs. the legacy in-memory BM25 index when false
KB_WEB_AUTH_TOKEN	(unset)	Bearer/X-KB-Token/cookie token required by kb serve; mandatory when -addr is non-loopback
KB_WEB_RATE_LIMIT	0 (off)	Requests per minute per client IP for kb serve; only enforced on non-loopback binds
KB_WEBSEARCH_URL	(unset)	SearXNG base URL enabling optional web search in Ask follow-up chat; empty (default) disables the feature
KB_WEBSEARCH_MAX_RESULTS	5	Max web results returned per confirmed web search
KB_WEBSEARCH_MAX_PER_THREAD	0 (unlimited)	Max confirmed web searches per ask thread; 0 = no per-thread limit

Connector instances are declared in $KB_ROOT/sources.yaml. The file stores only the names of environment variables that hold secrets; values are read from the process environment at resolve time and never written to disk. Format and per-connector options: docs/sources.md.

Usage

go build -o bin/kb ./cmd/kb

doctor

Health and sync-health report: LLM endpoint reachability (embed dimension + chat round-trip), index version/dimension, a PRAGMA integrity_check on kb.db, and a presence-only per-source report (which secret env vars are set, last sync time, staleness vs KB_STALE_AFTER, last sync error).

./bin/kb doctor

config

Dumps the effective runtime configuration as NAME=VALUE lines, including connector/subsystem variables that are read directly instead of going through internal/config. Secret values are redacted to <set> / (unset).

./bin/kb config show                 # current effective configuration
./bin/kb config show --preset fast   # low-latency DRAGON-tuned preset
./bin/kb config show --preset quality # higher-recall DRAGON-tuned preset

Numeric and enum configuration values are validated at startup, including direct connector/subsystem variables such as KB_SOCKS_PROXY and the benchmark/verification thresholds.

backup

Creates a consistent SQLite backup of $PERSIST_DIR/kb.db using VACUUM INTO. If no destination is given, the backup is written to $PERSIST_DIR/backups/kb-<UTC-timestamp>.db; pass an explicit path to control the location. The command refuses to overwrite an existing destination.

./bin/kb backup
./bin/kb backup /backups/kb-$(date -u +%Y%m%dT%H%M%SZ).db

To recover, stop the running server, replace $PERSIST_DIR/kb.db with the backup file, then run ./bin/kb doctor to verify the restored database. Keep backup files off the same disk as $PERSIST_DIR when possible so a single drive failure cannot take both the live index and the recovery copy.

sync

Runs the configured connectors and writes documents to KB_ROOT via the sink. Cursor is advance-on-success with rollback; tombstones prevent re-import of deleted items; prune happens only on full reconcile.

./bin/kb sync --all            # all sources from sources.yaml
./bin/kb sync --source=NAME    # only the named source
./bin/kb sync --all --api=http://127.0.0.1:8321   # push to a running server (POST /documents) instead of writing files

--api mode indexes documents in the server without writing files; API-fed documents are kept across kb reindex (full reindex garbage-collects only filesystem-backed docs that no longer exist). Chat messages are indexed one message at a time in this mode, so reply chains are chunked per message rather than glued into a single thread chunk — run a file-based sync or a full reindex for thread glueing.

reindex

Rebuilds the vector + graph index from the documents under KB_ROOT (chunking → embeddings → LLM graph extraction → communities → summaries). Optional positional argument restricts reindexing to a subpath. A document whose content is unchanged since the last successful index is skipped (content-hash check) — a repeat reindex only pays the embed/LLM cost for files that actually changed. Output reports indexed/skipped/removed counts.

./bin/kb reindex [subpath]
./bin/kb reindex --reembed   # clear stored embeddings + dimension, re-embed from scratch

--reembed clears stored embeddings and the stored dimension before reindexing — use it when switching KB_EMBED_MODEL or recovering from an ErrDimMismatch.

describe

Walks the corpus for documents without a summary frontmatter key and generates a short description via the LLM (fail-open, in batches), writing it back through the sink + indexer and refreshing BM25. Documents that already have a summary are skipped. If the LLM is unreachable, generation falls back to the first ~200 characters of the first meaningful sentence of the body.

./bin/kb describe [--source NAME]   # only describe documents from one source

Settings: KB_DESCRIBE_MODEL (default qwen3.8:latest) and KB_DESCRIBE_BATCH (default 10).

verify

Runs Q&A evals over a golden set built from closed Leon issues (question = issue title, expected = issue body): retrieval + synthesis answers are judged by an LLM with an offline overlap fallback (fail-open), and a report with pass rate and per-source hit rate is written to PERSIST_DIR/last-qa-report.json.

./bin/kb verify [--pairs testdata/leon-qa/qa_pairs.json] [--limit N]
./bin/kb verify --build-golden [--source leon-ai] [--golden-out testdata/leon-qa/qa_pairs.json]
./bin/kb verify --top-k 8 --report $PERSIST_DIR/last-qa-report.json

Flags: --pairs (golden input path, default testdata/leon-qa/qa_pairs.json), --build-golden (rebuild the golden set from KB_ROOT), --golden-out (golden output path), --report (eval report path, default $PERSIST_DIR/last-qa-report.json), --source (source filter for --build-golden, default leon-ai), --limit (evaluate the first N pairs), --top-k (chunks retrieved per question, default KB_TOP_K).

EnterpriseRAG-Bench (kb bench)

Run the EnterpriseRAG-Bench question set (500 questions, 10 categories) against the kb pipeline and emit a leaderboard-ready submission:

unzip all_documents.zip -d /data/erb/corpus
./bin/kb bench \
  --corpus /data/erb/corpus \
  --questions questions.jsonl \
  --out answers.jsonl \
  --limit 50 --types constrained,conflicting_info,completeness,info_not_found

# one-minute sanity run on the checked-in bilingual subset
./bin/kb bench --smoke

# reuse a persisted index and accumulate metrics history
./bin/kb bench --corpus /data/erb/corpus --questions questions.jsonl \
  --persist-dir /data/erb/persist --history /data/erb/bench-history.json

Outputs: answers.jsonl in the official submission format ({"question_id","answer","document_ids"}) plus a per-type metrics report (*.report.json) with document recall vs gold docs, abstention share and citation coverage. Bench-specific knobs: KB_QUALIFIER_FILTER, KB_SUPERSEDE_MODE, KB_ABSTAIN_THRESHOLD, KB_SET_MAX_ROUNDS, KB_CANDIDATE_K, KB_PER_DOC_CAP, KB_INTRA_DOC_BUDGET, KB_DETECT_CONTRADICTIONS (see table above). --smoke uses the checked-in testdata/lang-bench subset (16 docs, 20 questions). --persist-dir reuses a corpus index and skips unchanged docs via doc_hashes; --history (or the default next to the report) appends each run's metrics report. kb bench itself needs a live LLM endpoint; the slice and score subcommands are local-only and do not call the LLM.

-answer-mode selects the answering path: got (default, the existing Graph-of-Thoughts orchestrator) or naive (single-shot retrieval plus one chat call). The default is unchanged, so existing kb bench invocations keep using GoT.

Building the ERB evolution slice

kb bench slice copies the gold documents for the selected questions into a smaller corpus tree so the feature ladder runs against a fixed, targeted subset:

./bin/kb bench slice \
  --corpus /data/erb/corpus \
  --questions questions.jsonl \
  --types basic,semantic,conflicting_info,completeness,project_related,constrained \
  --out-corpus docs/bench/erb-evolution/corpus \
  --out-questions docs/bench/erb-evolution/questions.jsonl

Flags: -corpus (source .txt corpus root), -questions (source questions JSONL), -types (comma-separated types to keep, empty = all), -limit-per-type (cap questions per type), -out-corpus and -out-questions (output locations). It writes the filtered questions and copies every referenced gold document, reporting any missing document IDs.

Scoring ERB submissions

kb bench score grades a submission JSONL against the source questions JSONL using English gold-answer and fact stemming:

./bin/kb bench score \
  --questions questions.jsonl \
  --out answers.score.json \
  answers.jsonl

Flags: one positional <submission.jsonl> argument, -questions (required), -out (score report JSON, default <submission>.score.json), and -history (score history JSON, default <out>.history.json). The report contains retrieval_hits, answer_contains_gold, avg_facts_coverage, and per-type breakdowns.

Comparing embedders (RU vs EN)

To check whether swapping the embedding model improves Russian retrieval without regressing English, use the checked-in bilingual dataset (testdata/lang-bench: 16 docs and 20 questions tagged language: "ru" or "en"):

KB_EMBED_MODEL=qwen3-embedding ./bin/kb bench \
  --corpus testdata/lang-bench/corpus \
  --questions testdata/lang-bench/questions.jsonl \
  --out /tmp/baseline.jsonl

# swap the embedder, then re-run
KB_EMBED_MODEL=<new-model> ./bin/kb bench \
  --corpus testdata/lang-bench/corpus \
  --questions testdata/lang-bench/questions.jsonl \
  --out /tmp/candidate.jsonl

./bin/kb bench compare \
  /tmp/baseline.jsonl.report.json \
  /tmp/candidate.jsonl.report.json

bench compare prints signed per-language and per-type deltas (candidate minus baseline) for recall, abstention, and citation coverage; add --out delta.json to also write the comparison as JSON. Each run reindexes the supplied --corpus from scratch into an isolated temporary database using KB_EMBED_MODEL, so no manual reindex is needed; embedding dimension mismatches still fail loudly by design.

DRAGON RU RAG-Bench (kb bench-dragon)

Run kb over the Russian DRAGON RAG benchmark (ai-forever/rag-bench-public-texts — 526 news articles, ai-forever/rag-bench-public-questions — 600 questions), fetched directly from HuggingFace, indexed into an isolated temporary database, and answered through the full GraphRAG + Graph-of-Thoughts pipeline:

./bin/kb bench-dragon --limit 5           # quick smoke run

./bin/kb bench-dragon --smoke --persist-dir /tmp/dragon-persist

# full 526-doc / 600-question run
./bin/kb bench-dragon --out answers.dragon.json --concurrency 3

Output: answers.dragon.json, a {question_id: {found_ids, model_answer}} map in the exact shape DRAGON's official evaluator expects for a leaderboard submission. This is a self-run submission file, not an official DRAGON score — the public question set ships without gold answers, so grading happens only when the file is submitted to the DRAGON maintainers. A committed sample run lives at docs/bench/dragon-answers.json (full 600-question set, indexed with graph extraction on). Flags: --limit (cap the question count), --concurrency (parallel questions), --top-k (chunks per subgoal, default KB_TOP_K), --hf-base-url (override the HuggingFace datasets-server endpoint, mainly for testing), --persist-dir/--force-reindex (reuse or rebuild a persisted index), --doc-limit (keep only the first N fetched texts), and --smoke (a fixed 12-doc/5-question sanity subset). bench-dragon score --history PATH appends each score report to a metrics history. The command needs a live LLM endpoint and network access to datasets-server.huggingface.co.

For the feature-attribution ladder (which capability contributes to the score), see docs/bench/dragon-evolution-report.md — a reduced-corpus diagnostic run (Native → Hybrid → Graph → Rerank → Logic → Temporal → Qualifiers) whose absolute numbers are not comparable to the full-corpus 75.2% figure, only the relative deltas between adjacent stages.

For the same ladder on the category-targeted ERB slice, see docs/bench/erb-evolution-report.md — a single-run diagnostic over all seven stages with the Dynamic Currency Score (kb bench dynamic), which scores whether answers name the current rather than a superseded fact. The sample is 11 questions, so only the broad pattern is supported, not stage-to-stage rankings; see the report's Reliability section.

The verify command needs a live LLM endpoint (retrieval + synthesis); integration tests for the QA harness are gated behind -tags integration + KB_LLM_IT=1.

RU dynamic bench (kb bench generate / kb bench evolve)

An own Russian-language RAG benchmark on an enterprise-like corpus, built on the DRAGON mechanics (hand-written seed questions for validation + questions generated from the corpus on the fly, scored separately for retrieval, context, and answer). It reuses the language-agnostic kb bench pipeline and adds a Russian-aware scorer, a question generator, a context metric, and the generic feature-evolution ladder. Full report: docs/bench/ru-dynamic/report.md; corpus provenance and license: docs/bench/ru-dynamic/SOURCE.md.

The checked-in RU corpus and seed live under testdata/ru-bench/ (298 doka.guide documents, 39 Russian questions in 4 types). Generate one single-doc question per document with the LLM, anchored to the document and gated so a hallucinated gold answer is dropped:

./bin/kb bench generate \
  --corpus testdata/ru-bench/corpus \
  --seed testdata/ru-bench/questions.jsonl \
  --out generated-ru-questions.jsonl \
  --count 20

Run the questions through the pipeline and grade them with the Russian-aware scorer (retrieval_hit, answer_contains_gold, avg_facts_coverage, context_precision/context_recall, optional LLM-judge faithfulness):

./bin/kb bench --ru-smoke

./bin/kb bench \
  --corpus testdata/ru-bench/corpus \
  --questions testdata/ru-bench/questions.jsonl \
  --out ru-answers.jsonl

./bin/kb bench score \
  --questions testdata/ru-bench/questions.jsonl \
  ru-answers.jsonl

-ru-smoke aliases the checked-in testdata/ru-bench/ subset. bench generate flags: -corpus, -seed (few-shot examples), -out, -count (one question per document, 0 = all), -model. bench score picks the snowball stemmer by each question's language field (ru → Russian, otherwise English) and adds -judge-faithfulness for the optional per-question LLM judge (off by default).

Run the seven-stage cumulative feature ladder (Native → Hybrid → Graph → Rerank → Logic → Temporal → Qualifiers), reusing persist-a (no graph) for stages 0–1 and persist-b (graph on) for stages 2–6:

./bin/kb bench evolve \
  --corpus testdata/ru-bench/corpus \
  --questions testdata/ru-bench/questions.jsonl \
  --out-dir docs/bench/ru-dynamic/evolution

bench evolve writes per-stage submissions, score reports, and score history to --out-dir; flags: -corpus, -questions, -persist-dir-a, -persist-dir-b, -out-dir, -concurrency, -top-k, -limit, -types. Both bench generate and bench evolve need a live LLM endpoint.

serve

Starts the web dashboard (internal/web):

• Search — synthesized answers, htmx loading indicator, persistent search history (survives restart) shown on /search; clicking a history entry opens its saved answer via /search?id=<id> without re-running retrieval, with a re-run button.
• Ask (Graph-of-Thoughts, SSE progress) — LogicRAG-style adaptive reasoning: decomposition into a dependency DAG, topological wave scheduling, a greedy forward pass that injects resolved dependency answers, bounded rolling memory (KB_ASK_ROLLING_WINDOW), and one dynamic gap-expansion round; progress renders as a structured list of steps (type/status/stage/answer), not raw JSON; runs persist to SQLite, so /ask/history lists past and in-flight runs and a run started before a restart still shows its last known state instead of an empty page. Completed ask responses are cached in SQLite keyed by hash(query + corpus_version + config fingerprint), so a repeated question against an unchanged corpus and configuration returns the previous answer (including fail-open placeholders) without paying LLM/retrieval cost again; stale entries are pruned on startup.
• Ask follow-up chat — after the final answer, the conversation can continue on the same thread; responses render as markdown with reasoning hidden behind <think> spoilers, and the thread persists to SQLite across restarts. With KB_WEBSEARCH_URL set, follow-ups can optionally search the web: the outgoing query is LLM-generalized, passed through a deterministic leak filter, then shown to the user for explicit per-query confirmation before any request leaves the process (fail-closed). Web results are untrusted external data, cited as [web:N], and never indexed into the corpus.
• Documents — summary list, edit form, htmx delete; /documents/view shows a document's graph relationships (entities/relations whose source chunks overlap the document), not just its raw content.
• Integrations — add/edit/delete source instances in sources.yaml.
• Graph — interactive canvas (Cytoscape.js, vendored, zoom/pan/drag) fed by a paginated/filtered /graph/data JSON endpoint (search, community, type, min-degree filters) instead of dumping the whole graph as static SVG; entity/ relation CRUD unchanged, click a node to open its edit panel.
• MCP — /mcp/info shows the live HTTP endpoint, the full tool list, and copy-paste client config for both the stdio and HTTP transports.
• Reports, cleanup, and trash routes.

./bin/kb serve                 # default 127.0.0.1:8080
./bin/kb serve -addr 127.0.0.1:9000   # custom listen address

serve binds to loopback by default: the dashboard has no authentication there and exposes destructive routes. Binding a non-loopback -addr requires KB_WEB_AUTH_TOKEN to be set (the process refuses to start otherwise), which enables bearer/X-KB-Token/cookie token auth on every non-/healthz route; pair it with KB_WEB_RATE_LIMIT (requests per minute per client IP, non-loopback binds only) if the endpoint is reachable beyond a trusted network. An SSH tunnel or authenticating reverse proxy remains the simpler option for remote access.

mcp

Exposes the knowledge base over the Model Context Protocol (internal/mcp): search, ask, get_document, list_sources, add_note, add_source, graph_query, generate_report, reindex, status.

Two transports:

• stdio — kb mcp, for local process integration (Claude Desktop, Claude Code).
• HTTP — mounted at /mcp inside kb serve (same tool set, streamable HTTP transport), for remote/networked MCP clients. The dashboard's /mcp/info page shows the live endpoint URL, the full tool list, and copy-paste client config for both transports.

./bin/kb mcp

memory compile

Compiles curated knowledge JSON into a .plepatch overlay sidecar for Qwen3.8-Flash-Next (qwen4exp). The compiler reads the PLE n-gram table layout from GGUF metadata, tokenizes each trigger with the model's GGUF-BPE tokenizer, and writes resolved vectors into the target rows. Supported operations are set, zero, random, and copy_from; output is always an overlay patch, never a materialized model rewrite.

{
  "entries": [
    {"trigger": "The capital of France is", "op": "copy_from", "copy_from": "Paris"},
    {"trigger": "Boiling point of water is", "op": "set", "vector": "vector.raw"},
    {"trigger": "obsolete fact", "op": "zero"}
  ]
}

./bin/kb memory compile -gguf model.gguf -knowledge knowledge.json
./bin/kb memory compile -gguf model.gguf -knowledge knowledge.json -out model.gguf.plepatch
./bin/kb memory compile -gguf model.gguf -knowledge knowledge.json -report report.json
./bin/kb memory compile -gguf model.gguf -knowledge knowledge.json -dry-run

Without -out, the patch is written next to the knowledge file with a .plepatch extension. -report writes a JSON plan/report, and -dry-run builds the plan without writing the patch. Hot-swap requires the patched ortegaalfredo/llama.cpp-NLTM runtime serving the Qwen3.8-Flash-Next model; standard Ollama does not support .plepatch overlays.

Connectors

Registered types (in internal/connectors/registry, wired in cmd/kb/connectors.go):

Type	Source	Notes
github	GitHub org or explicit repos	issues, PRs, contents, wiki
gitlab	GitLab group or projects	issues, MRs, wikis, files
wiki	MediaWiki or Confluence Cloud	config.variant: mediawiki|confluence
mcp	MCP server	stdio or HTTP transport
telegram	Telegram chats	bot token
slack	Slack channels	bot token
mattermost	Mattermost team/channels	base URL + token
yandex-tracker	Yandex Tracker queues	OAuth token + org id
youtrack	YouTrack projects	base URL + token
kaiten	Kaiten spaces	base URL + token
weeek	Weeek spaces	token
searchapi	Generic search API	configurable fields/pagination/auth
discord	Discord guild channels	bot token; optional SOCKS via KB_SOCKS_PROXY
trello	Trello board	public export, or API key/token for private boards
rss	RSS 2.0 feed	config.feed_url
web	Website via sitemap or explicit pages	config.sitemap_url or config.pages, content_selector
file	Local directory	applies file importers by extension

Connector secrets: GitHub/GitLab token, wiki token/email, chats token, Discord token, trackers token (Trello also key), searchapi per-config auth_* keys — all as env-var names. rss and web need no secrets.

File importers

internal/importer maps file extensions to importers, used by the file connector during kb sync (kb reindex indexes the markdown documents under KB_ROOT instead):

Extension	Importer
.pdf	PDF text extraction (pure-Go, pdftotext fallback)
.xlsx	Excel worksheets → documents
.json	JSON documents (gjson paths)
.sql	SQL DDL schema documents
.md	Legal-codex structural parser (legalru) — non-legal markdown yields no documents
.go	Go source → code-graph documents (code importer)

Memory upgrades: temporal graph, code graph, retrieval modes

Temporal knowledge graph

relations are bi-temporal (internal/store/sqlite): valid_from/valid_to carry when a fact is true in the real world (open-ended when NULL, e.g. a statute redaction date), created_at/expired_at carry when the system learned / stopped considering the fact. On an ingestion conflict (same src + predicate, different dst, still-open valid_to) the old edge is closed (valid_to/expired_at = time of the new fact), not overwritten — the audit trail is preserved.

• GraphStore.RelationsAsOf(ctx, ids, t) — point-in-time query: valid_from <= t AND (valid_to IS NULL OR valid_to > t).
• Neighbors/MatchEntities take an optional time parameter; the default is "now", so existing callers keep the current behavior.
• Legal articles carry their amendment history in frontmatter (redactions: YYYY-MM-DD:FZ,... plus redaction_date/fz_number for the latest revision); deterministic AMENDS edges get valid_from = amendment date (internal/graph/legal.go). Plenum clarifications use INTERPRETS edges and are intentionally non-temporal.

Legal corpus importer (legalru)

internal/importer/legalru is a deterministic structural parser for Russian legal codes: it splits a curated markdown codex (part → section → chapter → article, amendment history, Plenum resolutions) into one Document per article, no LLM involved. Documents carry kind: legal-article (Plenum points: legal-plenum), the ID scheme code/чN/рN/глN/стN, and frontmatter (code, code_title, article_number, article_title, redactions...). The curated gold corpus lives in internal/importer/legalru/testdata/gold/ (ГК РФ, часть первая + Постановление Пленума ВС РФ N 25) with expected_graph.json and qa_pairs.json — see docs/legal-gold-corpus.md.

Code knowledge graph

internal/graph/codegraph extracts a deterministic graph from Go sources with go/ast + go/types, no LLM call: nodes Package/Function/Type/Method, typed edges Calls/Imports/Implements/Declares, discriminated with kind=code. The indexer routes .go documents to this path and skips files with syntax errors (fail-open). Retrieval links symbol names from queries to entities and expands neighbors along Calls/Imports edges exactly like semantic edges.

Retrieval modes

retriever.Options.Mode selects the pipeline (internal/engine/retriever):

• local (default) — the hybrid graph-aware pipeline: dense multi-query + BM25 + RRF + authority prior + per-doc cap → entity-linking → neighbor expansion → community context.
• global — map-reduce over root-level community summaries: parallel partial answers (top 20 communities) + one reduce; degrades to local when no hierarchy exists (fail-open).
• drift — a vector search over community summary embeddings seeds a local refine (top 3 communities, up to 30 seed chunks).

The GoT decompose step picks a mode per sub-question ("main topics / how many" → global; "what exactly about X" → local; otherwise drift). Community detection: KB_COMMUNITY_ALGO=louvain|leiden (default louvain; Leiden produces the multi-level hierarchy, Louvain remains the fallback).

Chat two-phase extraction

Chat documents (kind: message, from the telegram/slack/mattermost connectors) get a thread-scope mini-graph instead of generic extraction: a deterministic phase attributes each message to its speaker (frontmatter user), filters small talk (heuristics, plus an optional LLM classifier that overrides them), and stamps DECIDED/PROPOSED/AGREED edges with the speaker's message timestamp; an LLM phase extracts topic entities and the decision edges themselves. Multi-message threads are glued into one chunk with per-speaker attribution (speakers chunk metadata), so edges are credited to the right participant, not the thread's first author. Edited messages are re-delivered (not skipped) and carry a normalized edit_at frontmatter key (RFC3339, matching updated_at): telegram edited_message/edited_channel_post, slack edited.ts, mattermost EditAt — so edits re-enter the indexing/lineage path.

Verification layer

• internal/verify — DiffGraph golden-graph diff (missing/extra/mismatched report), CheckCitations (every citation in an answer must exist in the retrieved context actually passed to the LLM), ContradictionDetector (LLM pass over retrieved chunks flags explicit contradictions; fail-open, off by default in GoT).
• internal/verify/legaleval — legal faithfulness harness (integration-only, KB_LLM_IT=1): Non-Hallucinated-Statute-Rate, Statute-Relevance-Rate, Legal-Claim-Truthfulness over the gold qa_pairs. Methodology and metric definitions: docs/legal-gold-corpus.md.
• internal/verify/qa — closed-issue Q&A evaluation (kb verify): LLM judge with a token-overlap fallback, writes last-qa-report.json. The fallback path is tested offline with no judge, so QA scoring runs without a live endpoint.
• The deterministic fake-LLM e2e (internal/integration/e2e_fake_test.go) wires DiffGraph and CheckCitations into the import→index→ask path and runs under make check with no network.

Incremental reindexing

• Chunk lineage: the update path soft-closes old chunk versions (valid_to, replaces) instead of physically deleting them (VectorStore.SoftCloseByDoc); retrieval and BM25 see only active chunks (valid_to IS NULL), while ChunksByDoc returns the full version history for lineage links. RemoveDocument still hard-deletes and clears superseded_by marks on other docs.
• Blast-radius supersession: after indexing, chunks of other docs sharing ≥ N entities (default 1) with the new doc's touched set are marked superseded_by = <ref_doc_id> (GraphStore.OverlappingChunks + VectorStore.SetSuperseded). Retrieval applies a soft ×0.9 rank penalty but never excludes them (fail-open — conflicts are still caught by verify.ContradictionDetector).
• Lazy communities: writes mark affected components stale=1 instead of running Leiden per write; GraphUpdater.RefreshStaleCommunities recomputes only stale components in batch at the end of a sync batch and lazily on a query throttle in the retriever. A failed refresh degrades to serving stale summaries as-is (fail-open).
• End-to-end demonstration of this actualization path, with a permanent CI-safe regression test: docs/bench/actualization-report.md. The regression test proves chunk soft-close (VectorStore) and bi-temporal relation-close (GraphStore) both work; live Slack input only reliably drives the chunk-level supersession, since ChatExtractor (the real extraction path for chat-sourced documents) only emits decision-tracking edges, not typed facts, so it doesn't produce a relation to close.

Feature evolution ladder (benchmarks)

kb is built as a cumulative ladder: each rung adds exactly one capability on top of the previous one, so the delta between adjacent rungs is attributable to that single feature. Every capability is switched on with one env var, which is what makes this measurable at all.

Run on the DRAGON RU RAG-Bench diagnostic slice (192 docs, N=150 for rungs 0–3 and N=100 for rungs 4–6; chat/graph qwen3.8:latest, embeddings qwen3-embedding:0.6b). Absolute numbers are a reduced diagnostic slice and are not comparable to the full-corpus run — only the deltas between adjacent rungs are meaningful.

The numbers below are the mean of two full runs of the same ladder on the same slice (run 1: 2026-09-03, run 2: 2026-09-14). The answering path is LLM-driven, so results are not bit-for-bit reproducible and the two runs differ by a few questions per rung; the per-rung counts below are averaged across both. Treat them as "what this slice showed across two runs", not as a stable benchmark score. Run-1/run-2 raw counts are in docs/bench/evolution/ and docs/bench/evolution/rerun-20260914/.

#	Rung	Adds	Path	N	retrieval_hit	answer_contains	Δ retrieval	Δ answer	Wall time
0	Native	dense-only retrieval, one LLM call	naive	150	3.0% (4.5/150)	33.7% (50.5/150)	—	—	16.2m
1	+Hybrid	KB_HYBRID=true (BM25 + RRF)	naive	150	2.0% (3.0/150)	34.0% (51.0/150)	−1.5	+0.5	16.1m
2	+Graph	KB_INDEX_GRAPH=true	naive	150	4.0% (6.0/150)	35.7% (53.5/150)	+3.0	+2.5	53.8m
3	+Rerank	KB_RERANK=llm	naive	150	3.7% (5.5/150)	36.7% (55.0/150)	−0.5	+1.5	46.4m
4	+Logic	Graph-of-Thoughts (-answer-mode=got)	got	100	5.0% (5.0/100)	39.0% (39.0/100)	+2.5*	0.0*	105.4m
5	+Temporal	KB_SUPERSEDE_MODE=strict, KB_DETECT_CONTRADICTIONS=true	got	100	4.5% (4.5/100)	37.0% (37.0/100)	−0.5	−2.0	128.5m
6	+Qualifiers	KB_QUALIFIER_FILTER=true	got	100	3.0% (3.0/100)	33.0% (33.0/100)	−1.5	−4.0	129.4m

* Rungs 0–3 are scored on N=150, rungs 4–6 on N=100, so the 3→4 delta is only comparable on the shared 100-question subset: there the averaged step goes retrieval_hit 2.5→5.0 and answer_contains 39.0→39.0 (reasoning improves retrieval, not the final answer count).

What the ladder shows across the two runs:

• +Graph is the rung with the clearest positive delta — it lifts retrieval on multi-hop questions and adds to the final answer.
• +Rerank is the cleanest gain on the final answer among the retrieval-side rungs, at no retrieval cost.
• +Hybrid is flat here because the slice is homogeneous and does not exercise lexical matches (typos, code identifiers); it needs a noisier corpus to show up.
• +Temporal is structurally inert on a one-shot static import — there is nothing to supersede. The mechanism is verified separately by a live chat-actualization run (10/10 affected answers update) plus a deterministic fake-LLM test asserting superseded_by/valid_to.
• +Qualifiers reads as a small regression here (answer 37%→33%), which the methodology attributes to the lack of constrained-type questions, not to a broken feature.

See docs/bench/dragon-evolution-report.md for the full methodology, per-type breakdown, and noise discussion, and docs/bench/dragon-report.md for the full-corpus (542-doc / 600-question) run: 75.2% (451/600) answer-contains-gold, with 0 hallucinated citations in the verified sample.

Development

go test ./...     # all offline tests: unit + fake-LLM e2e (no network, no live endpoint)
go vet ./...
gofmt -l .

Integration tests that talk to a real LLM endpoint are gated behind the integration build tag and KB_LLM_IT=1:

KB_LLM_IT=1 go test -tags integration ./...

Testing conventions: DI seams everywhere (EnvLookup, httptest.Server, injected HTTPDoer/clock, in-memory Sink, fake LLM/Embedder/Reranker/GraphStore), golden files in testdata/, fail-open behavior asserted on every stage.

Adding a connector

Step-by-step guide with a checklist of test axes: docs/new-connector.md.

Documentation

Start from the docs index: docs/README.md.

• docs/architecture.md — GraphRAG pipeline diagram and data flow
• docs/sources.md — sources.yaml format and per-connector options
• docs/new-connector.md — how to add a new connector
• docs/legal-gold-corpus.md — legal gold-corpus methodology and eval metrics
• docs/bench/dragon-report.md — DRAGON RU RAG-Bench full-corpus run (75.2%)
• docs/bench/dragon-evolution-report.md — feature evolution ladder (per-rung deltas)
• docs/bench/erb-evolution-report.md — EnterpriseRAG-Bench slice (methodology; run pending)
• docs/bench/actualization-report.md — chat-actualization demo (temporal updates change answers)
• CONTRIBUTING.md — development and testing conventions
• SECURITY.md — loopback/no-auth design and vulnerability reporting
• CHANGELOG.md — release notes

License

Apache License 2.0. See LICENSE and NOTICE.