Persistent source code graph for semantic tools

[screenfluent]

Hi @yogthos,

Dirge already has useful semantic tools based on tree-sitter. Would it make sense to extend them with a persistent source code graph?

The idea is similar to https://github.com/DeusData/codebase-memory-mcp

The graph could store symbols and relations directly from source code, for example:

• functions, classes and files
• calls between functions
• imports and module dependencies
• definitions and references

This could improve:

• callers and callees accuracy
• tracing a path between functions
• impact analysis before changing code
• architecture overview
• incremental indexing after file changes

I saw #393, but I think this is a different graph. #393 stores information from sessions and tool outputs. This proposal is about a graph built directly from the source code.

Later it could also support data flow, Git history relations and cross-repository analysis, but these do not need to be part of the first version.

Do you think this direction fits Dirge?

[allen-munsch]

So I've been running the Janet compressor, but dirge doesn't actually reach to it for usage very clearly when I've had it enabled, it might be worth moving it out of Janet into an actual rust module? Also, might require deeper architecture/thinking, the initial Janet stuff was meant as a quick plugin. Something to think about.

I don't have a solution in mind, but here's what I found.

What's missing:

The SymbolIndex is entirely ephemeral. Cache is per-process, mtime-invalidated, and lost on restart. There is no persistent store of {function X} CALLS {function Y}, no {file A} IMPORTS {file B}, and no persisted call graph. find_callers runs a grep every time. find_callees re-parses the source every time.

Your proposal is a third graph, distinct from #393: a source-code structure graph built from tree-sitter, persisted in SQLite, with incremental invalidation on file change. This is feasible, dirge already has the extraction layer.

Where the integration point could be:
• SymbolIndex::ensure_file() already walks a file's AST and extracts symbols. After extraction, each symbol's callees (from find_callees_in_range) and the file's imports could be upserted into a code_symbols + code_relations table.
• The existing entity_db.rs and graph.rs tool pattern for #393 could be extended or mirrored, same SQLite session DB, same recursive CTE for traversal, same FTS5 for search, but a separate set of tables for source-level entities.
• Incremental invalidation: on next ensure_file() call, if mtime changed, drop the old rows for that file and re-extract.

This would give persistent, queryable call graphs, import graphs, and symbol indices, and later, cross-session impact analysis and architecture overview.

I think it'd slot cleanly into the existing semantic + entity_db infrastructure.

[screenfluent]

Thanks, this matches what I had in mind.

I agree that the source code graph should be separate from the session/entity graph, but it could reuse the same SQLite and traversal patterns. Starting from SymbolIndex::ensure_file() also sounds like a natural integration point.

One concern is that persisting the current find_callees results would improve speed and persistence, but maybe not accuracy for same-name functions and cross-module calls. I think qualified symbol identity and import-aware call resolution would be important, even if added step by step.

Moving the core indexing into Rust also makes sense if this becomes a native semantic feature.

[allen-munsch]

@screenfluent would you be interested (have bandwidth) to give this a go?

[yogthos]

And all of this sounds reasonable to me. 👍