From acb6fc431a25815af60ec91afcdad8f6964a7381 Mon Sep 17 00:00:00 2001 From: lkdvos Date: Mon, 5 Oct 2026 14:00:24 -0400 Subject: [PATCH 1/9] docs: finalize usage guides and package comparison for v0.1.0 --- docs/make.jl | 3 +- docs/src/comparison.md | 41 +++++++++++ docs/src/configuration.md | 19 +++--- docs/src/dashboard.md | 57 +++++++--------- docs/src/disk.md | 103 +++++++++++----------------- docs/src/index.md | 73 ++++++++------------ docs/src/interface.md | 12 ++-- docs/src/keys.md | 98 +++++++++++++-------------- docs/src/timing.md | 8 +-- docs/src/usage.md | 139 ++++++++++++++++++++++++++++++++++++++ src/call.jl | 1 + src/macro.jl | 10 ++- 12 files changed, 347 insertions(+), 217 deletions(-) create mode 100644 docs/src/comparison.md create mode 100644 docs/src/usage.md diff --git a/docs/make.jl b/docs/make.jl index 8d1df13..fc5b9a5 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -13,7 +13,8 @@ makedocs(; assets = String[], ), pages = [ - "Home" => "index.md", "Custom cache keys" => "keys.md", "Configuration" => "configuration.md", "Dashboard" => "dashboard.md", + "Home" => "index.md", "Usage" => "usage.md", "Why Cached?" => "comparison.md", + "Custom cache keys" => "keys.md", "Configuration" => "configuration.md", "Dashboard" => "dashboard.md", "Timing" => "timing.md", "Disk caching" => "disk.md", "Implementing a cache" => "interface.md", "Reference" => "reference.md", ], diff --git a/docs/src/comparison.md b/docs/src/comparison.md new file mode 100644 index 0000000..3393a07 --- /dev/null +++ b/docs/src/comparison.md @@ -0,0 +1,41 @@ +# Why Cached? + +Cached is for memoization that needs ongoing management, especially in long-running or repeated computations. +Its niche is the combination of bounded RAM caches, optional persistence across runs, and tools to inspect and adjust caches while a program runs. +You can apply these controls per function or package without writing cache bookkeeping around each computation. + +For example, a workload can keep frequently used results in RAM, recover older results from disk after a restart, and show which functions benefit from caching in the dashboard. +Strategy dispatch lets the same function share expensive results across tasks, keep independent task-local caches, or recompute cheap argument types. + +## Choosing a package + +| Package | Choose it when | Tradeoff | +| :-- | :-- | :-- | +| [Memoize.jl](https://github.com/JuliaCollections/Memoize.jl#usage) | You want a small `@memoize` API with a dictionary of your choice. | Limits and statistics depend on the container; persistence, a dashboard, and package-level settings require additional integration. | +| [Memoization.jl](https://github.com/marius311/Memoization.jl#readme) | You need to memoize individual calls or closures, as well as method definitions. | Its default cache is unbounded and not thread-safe; a custom container can address those needs, but the management and persistence tools require additional integration. | +| [LRUCache.jl](https://github.com/JuliaCollections/LRUCache.jl#readme) | You want a thread-safe cache dictionary with size limits, statistics, resizing, and eviction callbacks. | You write the function wrapper and management code yourself, or combine it with a memoization macro. | +| Cached.jl | You want bounded caches, per-type strategies, disk persistence, and live monitoring under one API. | The macro requires a method definition; disk results need explicit versioning and cleanup, and shared caches use locks. | + +Memoize.jl and Memoization.jl both accept LRUCache.jl containers, so bounded memory and statistics are available without Cached. +Memoization.jl also preserves return-type inference. +Cached's benefit is that strategies, settings, persistence, and monitoring work together out of the box, with SQLite.jl and Tachikoma.jl loaded for their respective features. + +## Differences that affect behavior + +**Key matching:** Memoize.jl and Memoization.jl default to `IdDict`, matching arguments by identity; both allow a different dictionary. +By default, Cached's built-in caches match by value using `hash` and `isequal`, and distinguish argument types, so `f(3)` and `f(3.0)` occupy separate entries. +Custom cache keys can merge equivalent inputs across argument types; see [Custom cache keys](keys.md). +Value-based matching can reuse separately constructed equal keys, but hashing large keys can cost more than recomputing. + +**Concurrency:** Cached's shared RAM caches are thread-safe by default, but their locks can contend and simultaneous misses may duplicate computation. +Task-local caches avoid sharing storage, at the cost of repeated computations and memory in each task; they are absent from the dashboard and global management API. +Memoization.jl documents thread safety for top-level functions with a thread-safe container, but not for closures or callable objects; see its [limitations](https://github.com/marius311/Memoization.jl#limitations). + +**Invalidation:** Cached requires manual clearing after a method or external state changes. +Memoization.jl handles method redefinition for memoized definitions, though memoized individual calls require clearing; see its [redefinition behavior](https://github.com/marius311/Memoization.jl#limitations). +For Cached's persistent results, you must also update the disk version when results or stored types change. + +**Persistence and limits:** Disk caching adds serialization and file I/O, so it suits results that are expensive to recompute. +Disk databases have no automatic eviction; RAM limits measure entries or values, not total process memory. +The dashboard monitors global caches used in the current process. +See [disk caching](disk.md) and [usage](usage.md) for the details. diff --git a/docs/src/configuration.md b/docs/src/configuration.md index 0e06bc2..7cdafb8 100644 --- a/docs/src/configuration.md +++ b/docs/src/configuration.md @@ -4,8 +4,7 @@ CurrentModule = Cached ``` -Default cache settings are read with [Preferences.jl](https://github.com/JuliaPackaging/Preferences.jl) -from `LocalPreferences.toml`, next to the active project. +Default cache settings are read with [Preferences.jl](https://github.com/JuliaPackaging/Preferences.jl) from `LocalPreferences.toml`, next to the active project. ## Settings @@ -50,24 +49,26 @@ Unknown keys and invalid values are ignored with a warning. - `disk` and `disk_path` are read when a function first uses its disk cache in the session. - `maxsize` and `measure` are read when a function's first cache is created, so a change applies to functions that have not been called yet in the current session, and to every function after a restart. -- `container` is a compile-time preference, because it selects the default `CacheStyle`. Changing it recompiles Cached on the next start. `GlobalCache()` and `TaskLocalCache()` use this container, so `CacheStyle` methods that return them follow the preference. +- `container` is a compile-time preference, because it selects the default `CacheStyle`. + Changing it recompiles Cached on the next start. + `GlobalCache()` and `TaskLocalCache()` use this container, so `CacheStyle` methods that return them follow the preference. Use [`set_cache_preferences!`](@ref) to write the sections, which merges with what is already there: ```julia using Cached -set_cache_preferences!(; maxsize = 50_000) # [Cached] -set_cache_preferences!(MyPackage; measure = "bytes") # [MyPackage.Cached] +set_cache_preferences!(; maxsize = 50_000) # [Cached] +set_cache_preferences!(MyPackage; measure = "bytes") # [MyPackage.Cached] set_cache_preferences!(MyPackage.expensive; maxsize = 1000) # [MyPackage.Cached.expensive] set_cache_preferences!(MyPackage.expensive; maxsize = nothing) # remove a setting ``` ## Measuring sizes -With `measure = "bytes"`, every cached value is measured once, on insertion, by -[`Cached.cachesize`](@ref), which defaults to `Base.summarysize`. That traverses the whole -value, which can be slow for large nested values, and counts memory shared between values -once per value. Overload it for your own types: +With `measure = "bytes"`, every cached value is measured once, on insertion, by [`Cached.cachesize`](@ref), which defaults to `Base.summarysize`. +That traverses the whole value, which can be slow for large nested values, and counts memory shared between values once per value. +Keys and container overhead are excluded; this is a value budget, not a limit on total process memory. +Overload it for your own types: ```julia Cached.cachesize(t::MyTensor) = sizeof(t.data) diff --git a/docs/src/dashboard.md b/docs/src/dashboard.md index 46c62a5..4cee89a 100644 --- a/docs/src/dashboard.md +++ b/docs/src/dashboard.md @@ -4,9 +4,8 @@ CurrentModule = Cached ``` -[`cache_dashboard`](@ref) opens a terminal dashboard of the global caches, to browse them, watch -their hit rates live, and empty or resize them while your program runs. It is a package -extension on [Tachikoma.jl](https://github.com/kahliburke/Tachikoma.jl), so load Tachikoma first: +[`cache_dashboard`](@ref) opens a terminal dashboard of the global caches, to browse them, watch their hit rates live, and empty or resize them while your program runs. +It is a package extension on [Tachikoma.jl](https://github.com/kahliburke/Tachikoma.jl), so load Tachikoma first: ```julia using Cached, Tachikoma @@ -14,8 +13,8 @@ cache_dashboard() # refreshes every second cache_dashboard(; interval = 0.2, filter = "fs") # faster, only functions whose name contains "fs" ``` -It has two tabs, for the caches in RAM and on disk. With a few example caches, the RAM tab -looks like this (rendered while building these docs): +It has two tabs, for the caches in RAM and on disk. +With a few example caches, the RAM tab looks like this (rendered while building these docs): ```@example dashboard using Cached, Tachikoma, SQLite # hide @@ -60,17 +59,12 @@ end # hide preview(m, 16) # hide ``` -The header sums all caches: their entries, the bytes of those measured in bytes, and the recent -hit rate (over the last 10 refreshes). Each function is a row (or one per container type, in -the rare case that its [`CacheStyle`](@ref) selects several). The columns are a bar for the -recent hit rate (`-` without lookups), a bar for the size against the limit (in bytes for -caches measured in bytes), the container (`LRU` or `Clock`, with `+disk` if the function also -has a [disk cache](disk.md)), and a sparkline of hits per second. The panel at the bottom shows -the numbers for the selected row, including its lifetime hit rate. Task-local caches are not -shown. +The RAM tab shows one row per function and container type: recent hit rate, size against its limit, cache kind, and hits per second. +The header totals entries and byte-measured values; the selected row's details include lifetime statistics. +Recent hit rates cover the last 10 refreshes. +Task-local caches are not shown. -The Disk tab (`Tab`, or `2`) lists the functions whose [disk cache](disk.md) is open in this -process (with SQLite loaded), including those without a cache in RAM: +The Disk tab lists the disk caches opened in this process, including functions without a RAM cache: ```@example dashboard m.tab = 2 # hide @@ -81,18 +75,16 @@ filter!(!=(env), LOAD_PATH) # hide nothing # hide ``` -Its columns are the entries and size of each database on this node (the size includes the -write-ahead log), the hit rate of the lookups on disk, and their activity; the panel adds the -path of the database. Hits are results read from disk and misses results computed and written, -counted in this process (see [`Cached.disk_cache_stats`](@ref)); the header sums them over all -disk caches, with the total entries and size, and says when disk caches are off -([`disable_disk_caches!`](@ref)). The entries and sizes come from [`disk_cache_info`](@ref), -which is read in the background while the tab is shown, at most every 10 seconds (and on `g`), -as counting the entries of a large database on a network file system can take a while; `…` -shows until they arrive. +The Disk tab shows database entries and size (including the write-ahead log), hit rate, and activity. +The selected row adds the database path. +Hit/miss counters cover this process; entries and sizes describe the database shared on this machine. -The name column is as wide as the longest name; narrow terminals cut names in the middle and -drop columns from the right, and below 40×8 the dashboard asks for a larger terminal. +Database sizes and entry counts are read in the background, at most every 10 seconds while the tab is shown, or on `g`. +`…` appears until they arrive. +The header indicates when [`disable_disk_caches!`](@ref) has turned disk caching off. + +Narrow terminals shorten names and drop columns. +The minimum size is 40×8. | Key | Action | |:-|:-| @@ -105,12 +97,9 @@ drop columns from the right, and below 40×8 the dashboard asks for a larger ter | `g` | refresh now | | `q`, `Esc` | quit | -In resize mode, `←` and `→` halve and double the limit, `[` and `]` change it by 10%, `Enter` -applies and `Esc` cancels; the size bar previews the new limit. Applying calls -[`set_cache_size!`](@ref) with the function's current measure (entries or bytes), so the limit -persists for the function. Emptying and resizing act on the RAM tab only: a disk cache is -persistent and may be shared with other processes, so it is emptied with -[`empty_disk_caches!`](@ref) outside the dashboard. +In resize mode, `←` / `→` halve or double the limit, `[` / `]` adjust it by 10%, `Enter` applies, and `Esc` cancels. +The preview uses the cache's current measure (entries or bytes); applying calls [`set_cache_size!`](@ref). -The statistics are read with [`cache_info`](@ref) and [`Cached.cache_stats`](@ref), which take each -cache's lock only briefly, so the dashboard is safe to run while other tasks use the caches. +Emptying and resizing act on RAM only. +Use [`empty_disk_caches!`](@ref) to clear persistent results outside the dashboard. +Statistics are copied under brief locks, so the dashboard can run while other tasks use the caches. diff --git a/docs/src/disk.md b/docs/src/disk.md index 1de2f58..fb317da 100644 --- a/docs/src/disk.md +++ b/docs/src/disk.md @@ -4,19 +4,17 @@ CurrentModule = Cached ``` -Results that are expensive to compute can also be kept on disk, so that they survive the -process. The disk is a second level below the cache in memory: a call looks in RAM first (as -chosen by [`CacheStyle`](@ref)), then on disk, and only computes when both miss, after which the -result is written to both. +Results that are expensive to compute can also be kept on disk, so that they survive the process. +The disk is a second level below the cache in memory: a call looks in RAM first (as chosen by [`CacheStyle`](@ref)), then on disk, and only computes when both miss, after which the result is written to both. -Disk caching is a package extension on [SQLite.jl](https://github.com/JuliaDatabases/SQLite.jl): -load it with `using SQLite`, or, in a package, add SQLite to its dependencies and `import SQLite`. +Disk caching is a package extension on [SQLite.jl](https://github.com/JuliaDatabases/SQLite.jl): load it with `using SQLite`, or, in a package, add SQLite to its dependencies and `import SQLite`. ```julia using Cached, SQLite @cached function expensive(n::Int)::Matrix{Float64} - # takes long to compute + sleep(1) # stand-in for an expensive computation + [Float64(i == j) for i in 1:n, j in 1:n] end Cached.DiskCacheStyle(::typeof(expensive), args...) = DiskCache() @@ -24,10 +22,8 @@ Cached.DiskCacheStyle(::typeof(expensive), args...) = DiskCache() ## Choosing what goes to disk -[`DiskCacheStyle`](@ref)`(f, args...)` selects the disk strategy per function and argument -types, independently of `CacheStyle`: [`NoCache()`](@ref NoCache) (the default) or a -[`DiskCache`](@ref). Both resolve at compile time, so functions without a disk cache are not -affected, and a RAM hit costs the same with or without one. +[`DiskCacheStyle`](@ref)`(f, args...)` selects the disk strategy per function and argument types, independently of `CacheStyle`: [`NoCache()`](@ref NoCache) (the default) or a [`DiskCache`](@ref). +Both resolve at compile time, so functions without a disk cache are not affected, and a RAM hit costs the same with or without one. | `CacheStyle` | `DiskCacheStyle` | a call | |:-|:-|:-| @@ -37,61 +33,51 @@ affected, and a RAM hit costs the same with or without one. ## Where the data goes -Each function has one SQLite database per machine, `.-v-.sqlite`. The -directory is the `disk_path` preference of the function, its package, or Cached (see -[Configuration](configuration.md)), and by default a -[scratch space](https://github.com/JuliaPackaging/Scratch.jl) of the package that owns `f`. +Each function has one SQLite database per machine, `.-v-.sqlite`. +The directory is the `disk_path` preference of the function, its package, or Cached (see [Configuration](configuration.md)), and by default a [scratch space](https://github.com/JuliaPackaging/Scratch.jl) of the package that owns `f`. Functions outside packages (in scripts or the REPL) use a scratch space of Cached. ```julia set_cache_preferences!(MyPackage; disk_path = "/path/to/cache") ``` -- **One file per function and machine**, whatever the number of entries, which matters on file - systems that limit the number of files. -- **Processes on one machine share the database**, reading and writing at the same time. A - write is a transaction, so an interrupted process never leaves a partial entry. -- **Machines write separate files**, named after the host, since SQLite's locking does not work - across machines on a shared file system. Results are not shared between machines. +- **One file per function and machine**, whatever the number of entries, which matters on file systems that limit the number of files. +- **Processes on one machine share the database**, reading and writing at the same time. + A write is a transaction, so an interrupted process never leaves a partial entry. +- **Machines write separate files**, named after the host, since SQLite's locking does not work across machines on a shared file system. + Results are not shared between machines. -Every disk lookup takes SQLite's file locks. On a local disk a lookup takes microseconds; on a -network file system the locks go over the network, which can make it orders of magnitude -slower. That is negligible for results that take long to compute, which are the ones worth -keeping on disk; otherwise, point `disk_path` at a local disk. +Disk lookups take file locks, which can be slow on network storage. +Use a local `disk_path` when possible, and reserve disk caching for expensive computations. +Disk storage has no automatic size limit or eviction; use [`empty_disk_caches!`](@ref) to reclaim space in the current database. ## Versions -The file name holds a version, [`Cached.diskversion(f)`](@ref), `"1"` by default. Keeping it -current is up to you: bump it when the results of `f` change, and the old file is no longer -read. +The file name holds a version, [`Cached.diskversion(f)`](@ref), `"1"` by default. +Keeping it current is up to you: bump it when the results of `f` change, and the old file is no longer read. ```julia Cached.diskversion(::typeof(expensive)) = "2" ``` -The default format is that of `Serialization`, which is not guaranteed to be readable by other -Julia versions, nor after the definition of a stored type changes. Bump the version when that -happens, or write a stable format with a serializer of your own (below). +The default format is that of `Serialization`, which is not guaranteed to be readable by other Julia versions, nor after the definition of a stored type changes. +Bump the version when that happens, or write a stable format with a serializer of your own (below). -Entries that cannot be read, or are not of the value type of the call, count as misses and are -overwritten. Errors of the disk itself, such as a full disk, never fail a call: they are -reported once, and the call goes on without the disk. +Entries that cannot be read, or are not of the value type of the call, count as misses and are overwritten. +Errors of the disk itself, such as a full disk, never fail a call: they are reported once, and the call goes on without the disk. ## Formats -Keys and values are written with `Serialization`, indexed by the SHA-256 hash of the serialized -key; the key is stored too, to guard against hash collisions. +Keys and values are written with `Serialization`, indexed by the SHA-256 hash of the serialized key; the key is stored too, to guard against hash collisions. -[`Cached.cachekey`](@ref) selects the key for both RAM and disk caching. To share disk entries -between equivalent inputs, return a canonical representation that serializes to the same -bytes. [`Hashed`](@ref) changes RAM hashing and equality, but its wrapped value is still -serialized, so custom equality alone does not merge disk entries. See [Custom cache keys](keys.md) -for examples and the equality contract. +[`Cached.cachekey`](@ref) selects the key for both RAM and disk caching. +To share disk entries between equivalent inputs, return a canonical representation that serializes to the same bytes. +[`Hashed`](@ref) changes RAM hashing and equality, but its wrapped value is still serialized, so custom equality alone does not merge disk entries. +See [Custom cache keys](keys.md) for examples and the equality contract. -The serializer is a parameter of the style, `DiskCache(; serializer = Serializer)`. To choose -the format of some types, define a serializer type with its own `serialize` and `deserialize` -methods for them; everything else is written as by `Serialization`. A serializer is a mutable -`AbstractSerializer` with these fields and a constructor from an `IO`: +The serializer is a parameter of the style, `DiskCache(; serializer = Serializer)`. +To choose the format of some types, define a serializer type with its own `serialize` and `deserialize` methods for them; everything else is written as by `Serialization`. +A serializer is a mutable `AbstractSerializer` with these fields and a constructor from an `IO`: ```julia using Serialization @@ -118,22 +104,19 @@ end Cached.DiskCacheStyle(::typeof(expensive), args...) = DiskCache(; serializer = MySerializer) ``` -`writetag` and `OBJECT_TAG` are internals of `Serialization`, used the same way by -`Distributed`'s `ClusterSerializer`. +`writetag` and `OBJECT_TAG` are internals of `Serialization`, used the same way by `Distributed`'s `ClusterSerializer`. ## Turning it off -[`disable_disk_caches!()`](@ref disable_disk_caches!) turns all disk caches off in this process, -and [`enable_disk_caches!()`](@ref enable_disk_caches!) back on; while off, nothing is read from -or written to disk. The `disk = false` preference turns them off per function, package, or -globally: +[`disable_disk_caches!()`](@ref disable_disk_caches!) turns all disk caches off in this process, and [`enable_disk_caches!()`](@ref enable_disk_caches!) back on; while off, nothing is read from or written to disk. +The `disk = false` preference turns them off per function, package, or globally: ```julia set_cache_preferences!(MyPackage; disk = false) ``` -`disk` and `disk_path` are read when a function first uses its disk cache in a session. Disk -caches are never used during precompilation. +`disk` and `disk_path` are read when a function first uses its disk cache in a session. +Disk caches are never used during precompilation. ## Managing disk caches @@ -144,15 +127,12 @@ empty_disk_caches!(expensive) # remove the entries Cached.disk_cache_stats() # [expensive => (; hits, misses), ...] in this process ``` -[`disk_cache_info`](@ref) and [`empty_disk_caches!`](@ref) act on this machine's current -database; files of older versions or other machines are left alone. The Disk tab of the -[dashboard](dashboard.md) lists the open disk caches with these numbers. +[`disk_cache_info`](@ref) and [`empty_disk_caches!`](@ref) act on this machine's current database; files of older versions or other machines are left alone. +The Disk tab of the [dashboard](dashboard.md) lists the open disk caches with these numbers. ## Precomputed results as an artifact -A package can ship precomputed results, read-only, as a -[Pkg artifact](https://pkgdocs.julialang.org/v1/artifacts/): fill the disk cache, export it with -[`export_disk_cache`](@ref), and point [`Cached.disk_artifact`](@ref) at the artifact. +A package can ship precomputed results, read-only, as a [Pkg artifact](https://pkgdocs.julialang.org/v1/artifacts/): fill the disk cache, export it with [`export_disk_cache`](@ref), and point [`Cached.disk_artifact`](@ref) at the artifact. ```julia using Pkg.Artifacts @@ -164,6 +144,5 @@ end Cached.disk_artifact(::typeof(expensive)) = artifact"expensive" ``` -A call then looks in RAM, the artifact, and this machine's database, in that order, and only -then computes. New results go to the database; the artifact is never written, and holds the -results of one version of `f`. +A call then looks in RAM, the artifact, and this machine's database, in that order, and only then computes. +New results go to the database; the artifact is never written, and holds the results of one version of `f`. diff --git a/docs/src/index.md b/docs/src/index.md index b9cc4db..f4c2af3 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -4,59 +4,42 @@ CurrentModule = Cached ``` -Transparent, strategy-aware memoization for Julia functions. +Cached reuses the results of Julia functions called with the same arguments. +It is useful when repeated computations are expensive and their caches need limits, monitoring, or persistence. -!!! warning - This package is under active development and its API may still change. +- [`@cached`](@ref) keeps ordinary Julia method signatures, including defaults, keywords, varargs, and `where` parameters. +- [`CacheStyle`](@ref) selects shared, task-local, or uncached execution per function and argument type. +- Built-in [`ClockCache`](@ref) and [`LRU`](@ref) containers bound storage in entries or bytes. + With concrete keys and an inferred concrete return type, RAM hits do not allocate. +- [`Cached.cachekey`](@ref) maps equivalent inputs to a shared result; see [Custom cache keys](keys.md). +- [Preferences](configuration.md) set defaults per package or function; runtime tools inspect, clear, and resize global caches. +- Optional extensions add [disk persistence](disk.md), a [terminal dashboard](dashboard.md), and [timing](timing.md). -```julia -using Cached +## Quick start -@cached function combine(a, b; normalize = true) - # expensive computation -end +Requires Julia 1.10 or later. +Add Cached to your environment with Julia's package manager: -combine(1, 2) # computed -combine(1, 2) # looked up in combine's cache; the result is still inferred -uncached(combine, 1, 2) # bypasses the cache +```julia +using Pkg +Pkg.add("Cached") +``` -# choose the strategy per function and argument type -Cached.CacheStyle(::typeof(combine), a::Int, b::Int) = TaskLocalCache{LRU}() +```jldoctest +using Cached -cache_info(combine) # hit/miss statistics -cache_info(MyPackage) # ... of all cached functions of a module, empty_caches! likewise -set_cache_size!(combine, 1_000) -``` +@cached function fib(n::Int)::BigInt + n < 2 ? BigInt(n) : fib(n - 1) + fib(n - 2) +end; -- [`@cached`](@ref) works on any method definition: positional, default, varargs and keyword - arguments, `where` clauses, qualified names, operators and callable objects. -- Each function has one cache, whose `maxsize` (in entries or bytes) is the budget for all its - signatures together. Calls stay type-stable, and cache hits do not allocate. -- [`CacheStyle`](@ref) selects the strategy per function and argument type: a shared - [`GlobalCache`](@ref) (the default, a [`ClockCache`](@ref)), a [`TaskLocalCache`](@ref), or - [`NoCache`](@ref). -- Default sizes are configured per package and per function through preferences; see - [Configuration](configuration.md). -- With SQLite.jl loaded, [`DiskCacheStyle`](@ref) keeps results on disk as well, below the - RAM cache, in one database per function and node; see [Disk caching](disk.md). -- With Tachikoma.jl loaded, [`cache_dashboard`](@ref) opens a live terminal dashboard; see - [Dashboard](dashboard.md). -- With TimerOutputs.jl loaded, [`enable_cache_timers!`](@ref) times lookups and computations - per package; see [Timing](timing.md). - -## Sharing results between inputs - -Specialize [`Cached.cachekey`](@ref) to share results between equivalent inputs. Return a -canonical key to group calls, including calls to different methods or argument types, or use -[`Hashed`](@ref) to customize hashing and equality without changing the input type. +fib(100) -```julia -@cached allocation_shape(x; copies = 1) = (length(x), copies) -Cached.cachekey(::typeof(allocation_shape), x; copies = 1) = (length(x), copies) +# output -allocation_shape([1, 2]) -allocation_shape((3, 4)) # same key, shares the cached result +354224848179261915075 ``` -The function body receives the original arguments. See [Custom cache keys](keys.md) for -working examples, keyword handling, equality requirements, and RAM and disk behavior. +Calling `fib(100)` again reuses the stored result. +By default, each function gets a shared Clock cache with room for 10,000 entries across all its cached methods. + +Continue with [Usage](usage.md) for strategies and cache management, or [Why Cached?](comparison.md) for the motivation and comparison with other packages. diff --git a/docs/src/interface.md b/docs/src/interface.md index 0f27325..c58bb72 100644 --- a/docs/src/interface.md +++ b/docs/src/interface.md @@ -4,9 +4,8 @@ CurrentModule = Cached ``` -A [`Cached.AbstractCache`](@ref) is the container behind [`GlobalCache`](@ref) and -[`TaskLocalCache`](@ref). Besides [`LRU`](@ref) and [`ClockCache`](@ref), any subtype -`C{K, V} <: Cached.AbstractCache{K, V}` implementing these methods can be used: +A [`Cached.AbstractCache`](@ref) is the container behind [`GlobalCache`](@ref) and [`TaskLocalCache`](@ref). +Besides [`LRU`](@ref) and [`ClockCache`](@ref), any subtype `C{K, V} <: Cached.AbstractCache{K, V}` implementing these methods can be used: | Method | Contract | | :----- | :------- | @@ -16,8 +15,7 @@ A [`Cached.AbstractCache`](@ref) is the container behind [`GlobalCache`](@ref) a | [`resize!(c; maxsize)`](@ref resize!(::Cached.AbstractCache)) | Set the size limit, evicting entries until they fit. | | [`Cached.cache_stats(c)`](@ref) | `(; hits, misses, length, currentsize, maxsize, by)` | -They may be called from any task, so they must be thread-safe, and `get!` must not hold a lock -while it calls `default`, which may recurse into the same cache or throw. +They may be called from any task, so they must be thread-safe, and `get!` must not hold a lock while it calls `default`, which may recurse into the same cache or throw. `show` is derived from `cache_stats`; the other `AbstractDict` methods are optional. ## Example: first in, first out @@ -82,5 +80,5 @@ only(cache_info(square)).second FIFO{Any, Any}(2/2 entries, 1 hits, 5 misses) ``` -`TaskLocalCache{FIFO}()` works the same way. This `FIFO` boxes its keys, so unlike `LRU` and -`ClockCache` its hits allocate. +`TaskLocalCache{FIFO}()` works the same way. +This `FIFO` boxes its keys, so unlike `LRU` and `ClockCache` its hits allocate. diff --git a/docs/src/keys.md b/docs/src/keys.md index a6d88f8..029156f 100644 --- a/docs/src/keys.md +++ b/docs/src/keys.md @@ -4,15 +4,13 @@ CurrentModule = Cached ``` -A computation may depend on only part of its input: an array's shape, a tensor space's sector -structure, or a normalized name. Specialize [`Cached.cachekey`](@ref) to let equivalent inputs -share a cached result. The function body, return-type inference, [`CacheStyle`](@ref), and -[`DiskCacheStyle`](@ref) still use the original arguments. +A computation may depend on only part of its input: an array's shape, a tensor space's sector structure, or a normalized name. +Specialize [`Cached.cachekey`](@ref) to let equivalent inputs share a cached result. +The function body, return-type inference, [`CacheStyle`](@ref), and [`DiskCacheStyle`](@ref) still use the original arguments. ## The default key -Without an overload, the key contains every positional argument and, when present, the -keyword `NamedTuple`: +Without an overload, the key contains every positional argument and, when present, the keyword `NamedTuple`: ```jldoctest using Cached @@ -25,13 +23,13 @@ using Cached ``` The cached method fills in default positional and keyword values before calling the hook. -Its overload must accept the resulting arguments, including keywords. The function itself -is the first argument, so each function can define its own mapping. +Its overload must accept the resulting arguments, including keywords. +The function itself is the first argument, so each function can define its own mapping. ## Returning a canonical key -A canonical key contains the properties that determine the result. This example computes -the same allocation shape for any input of a given length: +A canonical key contains the properties that determine the result. +This example computes the same allocation shape for any input of a given length: ```jldoctest canonical-key using Cached @@ -48,9 +46,9 @@ b = allocation_shape((3, 4)) ((2, 1), (2, 1), 1) ``` -The vector and tuple share an entry because both calls produce the key `(2, 1)`. Cached does -not retain the original argument types alongside a custom key. Keywords that affect the -result belong in the key too: +The vector and tuple share an entry because both calls produce the key `(2, 1)`. +Cached does not retain the original argument types alongside a custom key. +Keywords that affect the result belong in the key too: ```jldoctest canonical-key allocation_shape([5, 6]; copies = 2) @@ -61,8 +59,8 @@ length(only(cache_info(allocation_shape)).second) 2 ``` -An overload can also combine positional defaults, varargs, and arbitrary keywords. Accept -them as an ordinary Julia function would and return a key containing the relevant values. +An overload can also combine positional defaults, varargs, and arbitrary keywords. +Accept them as an ordinary Julia function would and return a key containing the relevant values. Returning a compact tuple is often sufficient; no wrapper is required. ## Sharing across methods @@ -93,21 +91,19 @@ b = nitems((3, 4)) (2, 2, 1) ``` -Only the first method body runs. All calls merged by a key must accept the same cached result, -including its type. For example, methods returning `Vector{Int}` and `Vector{Float64}` should -use separate keys if callers require those types. Include `typeof(x)` in a custom key when -the original input type matters. +Only the first method body runs. +All calls merged by a key must accept the same cached result, including its type. +For example, methods returning `Vector{Int}` and `Vector{Float64}` should use separate keys if callers require those types. +Include `typeof(x)` in a custom key when the original input type matters. -[`LRU`](@ref) and [`ClockCache`](@ref) distinguish key types, so use a common key type for calls -that should share an entry. They also need to select the same cache container. Task-local -caches share only within a task; typed dictionaries such as `TaskLocalCache{Dict}()` have -separate containers for each key and inferred value type. +[`LRU`](@ref) and [`ClockCache`](@ref) distinguish key types, so use a common key type for calls that should share an entry. +They also need to select the same cache container. +Task-local caches share only within a task; typed dictionaries such as `TaskLocalCache{Dict}()` have separate containers for each key and inferred value type. ## Custom hashing and equality with `Hashed` -When forming a canonical representation is expensive, [`Hashed`](@ref) keeps the original -value and supplies the comparison functions instead. The hash callable takes `(value, -seed::UInt)`; the equality callable takes two wrapped values and returns a `Bool`. +When forming a canonical representation is expensive, [`Hashed`](@ref) keeps the original value and supplies the comparison functions instead. +The hash callable takes `(value, seed::UInt)`; the equality callable takes two wrapped values and returns a `Bool`. ```jldoctest hashed-key using Cached @@ -128,9 +124,9 @@ b = wrapped_shape([3, 4]) ((2, 1), (2, 1), 1) ``` -Length is cheap to compute, so a canonical key is simpler here. For a more complex structure, -custom hashing and equality can inspect the relevant fields directly without constructing -a separate representation. The implementation continues to receive `x` itself. +Length is cheap to compute, so a canonical key is simpler here. +For a more complex structure, custom hashing and equality can inspect the relevant fields directly without constructing a separate representation. +The implementation continues to receive `x` itself. `Hashed` also works as a key in ordinary dictionaries, and `parent` retrieves its value: @@ -146,28 +142,27 @@ d = Dict(a => "two items") ``` `Hashed(x)` uses `Base.hash` and `Base.isequal`; `Hashed(x, hashf)` changes only hashing. -The callables can be functions, closures, or callable structs. Two wrappers compare equal -only when both callables are identical under `===`. Closures with different captured settings -therefore remain separate even if they have the same function type. +The callables can be functions, closures, or callable structs. +Two wrappers compare equal only when both callables are identical under `===`. +Closures with different captured settings therefore remain separate even if they have the same function type. -The wrapper's type includes the wrapped value type. Consequently, wrappers around a vector -and a tuple do not share an entry in Cached's RAM containers. Return a canonical key of a -common type when sharing across those inputs is needed. +The wrapper's type includes the wrapped value type. +Consequently, wrappers around a vector and a tuple do not share an entry in Cached's RAM containers. +Return a canonical key of a common type when sharing across those inputs is needed. ## Choosing valid equality -The equality function must be reflexive, symmetric, and transitive. Whenever it considers -two values equal, their hash functions must produce equal hashes for every seed. Hash -collisions between unequal values are fine: the cache still checks equality. +The equality function must be reflexive, symmetric, and transitive. +Whenever it considers two values equal, their hash functions must produce equal hashes for every seed. +Hash collisions between unequal values are fine: the cache still checks equality. -Approximate comparisons such as `isapprox` generally are not transitive. For tolerance-based -grouping, define explicit buckets and use the bucket identifier as the canonical key instead. +Approximate comparisons such as `isapprox` generally are not transitive. +For tolerance-based grouping, define explicit buckets and use the bucket identifier as the canonical key instead. -Keep every property used by hashing or equality stable while its key is stored. A `Hashed` -wrapper retains its input rather than copying it, so mutating relevant fields can invalidate -the cache's lookup assumptions. A canonical key made from immutable properties avoids that -dependency. Cached results themselves are shared objects; mutations of a cached result are -visible to other callers sharing its key. +Keep every property used by hashing or equality stable while its key is stored. +A `Hashed` wrapper retains its input rather than copying it, so mutating relevant fields can invalidate the cache's lookup assumptions. +A canonical key made from immutable properties avoids that dependency. +Cached results themselves are shared objects; mutations of a cached result are visible to other callers sharing its key. ## RAM and disk caching @@ -178,11 +173,10 @@ The same key hook feeds both cache levels, but they match entries differently: | Canonical value | Key type, `hash`, and `isequal` | Serialized key bytes | | `Hashed` wrapper | Wrapper type, custom hash, and custom equality with matching policies | Serialized wrapper, including its value and callables | -Custom equality alone does not merge disk entries. To share them, return a canonical key -that serializes to the same bytes for equivalent inputs. Disk caching still needs SQLite.jl -and a [`DiskCacheStyle`](@ref) overload; see [Disk caching](disk.md). +Custom equality alone does not merge disk entries. +To share them, return a canonical key that serializes to the same bytes for equivalent inputs. +Disk caching still needs SQLite.jl and a [`DiskCacheStyle`](@ref) overload; see [Disk caching](disk.md). -After changing a key mapping, call [`empty_caches!`](@ref) for the function. If it uses disk -caching, also change [`Cached.diskversion`](@ref) or empty its disk cache so entries created -under the previous mapping are no longer used. [`uncached`](@ref) bypasses the key hook and -all caches; with positional defaults, supply the full arguments to the implementation. +After changing a key mapping, call [`empty_caches!`](@ref) for the function. +If it uses disk caching, also change [`Cached.diskversion`](@ref) or empty its disk cache so entries created under the previous mapping are no longer used. +[`uncached`](@ref) bypasses the key hook and all caches; with positional defaults, supply the full arguments to the implementation. diff --git a/docs/src/timing.md b/docs/src/timing.md index 4cc201e..a2f663f 100644 --- a/docs/src/timing.md +++ b/docs/src/timing.md @@ -4,7 +4,7 @@ CurrentModule = Cached ``` -With [TimerOutputs.jl](https://github.com/KristofferC/TimerOutputs.jl) loaded, [`enable_cache_timers!`](@ref) records the time spent in the cached functions of a package, in two nested sections per function: +With [TimerOutputs.jl](https://github.com/KristofferC/TimerOutputs.jl) loaded, [`enable_cache_timers!`](@ref) records the time spent in the cached functions of a package, in sections per function: - `lookup f` wraps the whole cache lookup, hit or miss; - `compute f` wraps the call of the implementation: nested in `lookup f` on a miss, and on its own under [`NoCache`](@ref); @@ -26,9 +26,9 @@ disable_cache_timers!(@__MODULE__) to ``` -Timing is a debugging tool, and works like TimerOutputs' `enable_debug_timings`. -When it is off, the default, it costs nothing: a cache hit has not even an extra branch. -Enabling or disabling a package recompiles the callers of its cached functions, and takes effect from the next top-level statement on: a function that enables the timers and then calls cached functions needs `invokelatest` for those calls. +Timing is off by default and adds no overhead while off. +Enabling or disabling it recompiles callers and takes effect from the next top-level statement. +If a function enables timers and calls cached functions in the same invocation, use `invokelatest` for those calls. Enabling a package again replaces its timer. Do not enable timers during precompilation. diff --git a/docs/src/usage.md b/docs/src/usage.md new file mode 100644 index 0000000..c485822 --- /dev/null +++ b/docs/src/usage.md @@ -0,0 +1,139 @@ +# Usage + +```@meta +CurrentModule = Cached +``` + +## Memoizing a method + +Put [`@cached`](@ref) before a method definition. +By default, its positional and keyword arguments form the cache key, and its body runs only on a miss. + +```jldoctest powers +using Cached + +@cached function powers(x::T, n::Int = 3; offset::T = zero(T))::Vector{T} where {T} + [x^i + offset for i in 1:n] +end; + +powers(2; offset = 1) + +# output + +3-element Vector{Int64}: + 3 + 5 + 9 +``` + +The macro also supports varargs, qualified names, operators, and callable objects. +Anonymous functions are not supported. +Methods without `@cached` keep their usual behavior. + +A return annotation is optional and may depend on `where` parameters. +Without one, Cached uses the inferred return type, falling back to `Any` if it is not concrete. +A type-stable body keeps calls inferred even though the shared cache holds results of different types. + +Use [`uncached`](@ref) to call the body without reading or filling RAM or disk caches. +Supply all arguments, including defaults: defaults belong to the wrapper, not the uncached body. + +```jldoctest powers +uncached(powers, 2, 3; offset = 1) + +# output + +3-element Vector{Int64}: + 3 + 5 + 9 +``` + +## Choosing a strategy + +Define [`CacheStyle`](@ref)`(f, args...)` for the function and positional argument types you want to customize. +A strategy based on types can be resolved by the compiler. + +| Strategy | Use it when | +| :-- | :-- | +| [`GlobalCache()`](@ref GlobalCache) | Tasks should share results (the default). | +| [`GlobalCache{LRU}()`](@ref GlobalCache) | You want exact least-recently-used eviction. | +| [`TaskLocalCache()`](@ref TaskLocalCache) | Tasks should keep separate caches, reducing contention on shared storage. | +| [`NoCache()`](@ref NoCache) | Computing is cheaper than caching, or results should not be reused. | + +For example, give each task its own cache for a recursive computation: + +```jldoctest +using Cached +@cached fib(n::Int)::BigInt = n < 2 ? BigInt(n) : fib(n - 1) + fib(n - 2); + +Cached.CacheStyle(::typeof(fib), ::Int) = TaskLocalCache{LRU}(); + +fib(10) + +# output + +55 +``` + +To skip caching floating-point calls to `powers`: + +```julia +Cached.CacheStyle(::typeof(powers), ::AbstractFloat, ::Int) = NoCache() +``` + +`GlobalCache()` and `TaskLocalCache()` use the [configured container](configuration.md), Clock by default. +Clock gives recently used entries a second chance during eviction; LRU evicts the least recently used entry. +[`GlobalLRUCache()`](@ref GlobalLRUCache) is an alias for `GlobalCache{LRU}()`. + +Task-local caches live for the task's lifetime and are absent from the global management API and dashboard. +Built-in task-local containers are bounded, but `TaskLocalCache{Dict}()` is unbounded. +Task-local caches can be separate per key and value type; their limits are not a single budget for the whole function. + +[`DiskCacheStyle`](@ref) selects disk caching independently. +`NoCache()` as the RAM strategy can still read and fill a disk cache; see [Disk caching](disk.md). + +## Inspecting and limiting caches + +Global caches are created on first use. +Normally, one cache holds all cached methods and argument types of a function. +If its strategy selects several container types, each gets a separate cache with the function's size limit. + +```julia +cache_info(powers) # live f => cache pairs, with size and statistics +cache_info(MyPackage) # functions owned by this module and submodules +cache_info() # all global caches +set_cache_size!(powers, 1000) # count entries +set_cache_size!(powers, 2^20; by = Cached.cachesize) # measure values in bytes +empty_caches!(powers) # clear entries, keep hit/miss counters +empty_caches!(MyPackage) # clear the package's global caches +``` + +Shrinking a cache evicts entries; changing its size measure discards it. +A value larger than the limit is returned but not retained by the built-in containers. +Byte limits measure values, not keys or container overhead, and are not a process-wide memory budget. +See [Configuration](configuration.md) for persistent defaults and custom size measures. + +## Keys, results, and invalidation + +- Default keys use `hash` and `isequal`, with an additional type check: `f(3)` and `f(3.0)` are different entries. + Keyword values are included; forwarded keyword order can also distinguish keys. +- Keep keys unchanged while cached. + Mutating an array used as a key can invalidate its hash. +- RAM results are shared objects. + Copy a mutable result before changing it. +- Use caching for computations determined by their arguments. + Changes to external state or method definitions (including through Revise) do not automatically invalidate results. + Clear the global cache with `empty_caches!`; for disk results, update [`Cached.diskversion`](@ref). +- Built-in shared caches are thread-safe. + Computations run outside the cache lock, allowing recursion; simultaneous misses may compute the same key more than once. + Exceptions are propagated without storing a result. + +Use [`Cached.cachekey`](@ref) to map equivalent inputs to a common key, or [`Hashed`](@ref) to customize hashing and equality. +See [Custom cache keys](keys.md) for examples and the requirements for sharing results. + +## How it works + +The macro moves the body to [`Cached.implementation`](@ref) and leaves the original method as a wrapper. +Strategy dispatch, lookup, and computation use ordinary Julia code; the call path defines no methods at runtime. +Caches are created lazily, so definitions work in precompiled packages. +To supply a different container, see [Implementing a cache](interface.md). diff --git a/src/call.jl b/src/call.jl index fb3aaf3..8124c35 100644 --- a/src/call.jl +++ b/src/call.jl @@ -10,6 +10,7 @@ function implementation end uncached(f, args...; kwargs...) Call the [`@cached`](@ref) function `f` without consulting or filling any cache. +Supply all positional and keyword arguments, including those with defaults in `f`. """ uncached(f, args...; kwargs...) = implementation(f, args...; kwargs...) diff --git a/src/macro.jl b/src/macro.jl index dda85b5..103f582 100644 --- a/src/macro.jl +++ b/src/macro.jl @@ -8,9 +8,13 @@ cache selected by [`CacheStyle`](@ref)`(f, args...)`. The key is chosen by [`Cached.cachekey`](@ref), which defaults to the tuple of positional arguments, followed by a `NamedTuple` of the keyword arguments if there are any. Custom keys -can merge calls with different argument types. The value type is the return type annotation -if present (it may depend on `where` parameters), and the inferred return type otherwise -(or `Any` if inference does not give a concrete type). +can merge calls with different argument types. Shared caches hold all key and value types +of a function together (one cache per container type). The return type annotation, if +present, may depend on `where` parameters; otherwise the value type is inferred, or `Any` +if inference does not give a concrete type. Keys of different types are distinct entries. + +Use stable keys and treat cached mutable results as shared objects. Cache entries are not +automatically invalidated when a method or external state changes. Works for any method definition: qualified names (`function Base.f(...)`), operators and callable objects (`(x::Foo)(args...)`). Use [`uncached`](@ref) to bypass the cache. From d8e5c100dc7f240958427c4cd61887a47726e89b Mon Sep 17 00:00:00 2001 From: lkdvos Date: Mon, 5 Oct 2026 14:00:38 -0400 Subject: [PATCH 2/9] docs: highlight disk caching and dashboard in README --- README.md | 83 ++++++++++++++++++++++++++++--------------------------- 1 file changed, 43 insertions(+), 40 deletions(-) diff --git a/README.md b/README.md index 9f49482..8e67b4e 100644 --- a/README.md +++ b/README.md @@ -6,67 +6,70 @@ [![Code Style: Runic](https://img.shields.io/badge/code_style-%F0%9F%AA%A8_Runic-9558B2)](https://github.com/fredrikekre/Runic.jl) [![Aqua](https://raw.githubusercontent.com/JuliaTesting/Aqua.jl/master/badge.svg)](https://github.com/JuliaTesting/Aqua.jl) -Transparent, strategy-aware memoization for Julia functions. +Memoization for Julia, with bounded memory, persistent results, and a live view of your caches. -> [!WARNING] -> This package is under active development and its API may still change. -> The design is described in [`research/design.md`](research/design.md). +- **Disk-backed caching:** reuse expensive results across runs with SQLite.jl, or ship precomputed results as artifacts. +- **Live dashboard:** watch hit rates and sizes, browse RAM and disk caches, and clear or resize RAM caches from your terminal. +- **Fast RAM hits:** preserve return-type inference and avoid allocations with the built-in caches and concrete keys. +- **Flexible strategies:** share results across tasks, keep them local to a task, or skip caching for selected argument types. +- **Custom keys:** reuse results for equivalent inputs with `Cached.cachekey`, or customize hashing and equality with `Hashed`. +- **Bounded storage:** Clock and LRU eviction, with limits in entries or bytes across a function's methods. +- **Package controls:** set defaults per package or function, inspect caches programmatically, and profile lookups and computations with TimerOutputs.jl. ```julia using Cached -@cached function combine(a, b; normalize = true) - # expensive computation +@cached function fib(n::Int)::BigInt + n < 2 ? BigInt(n) : fib(n - 1) + fib(n - 2) end -combine(1, 2) # computed -combine(1, 2) # looked up in combine's cache; the result is still inferred -uncached(combine, 1, 2) # bypasses the cache - -# choose the strategy per function and argument type -Cached.CacheStyle(::typeof(combine), a::Int, b::Int) = TaskLocalCache{LRU}() - -cache_info(combine) # hit/miss statistics -cache_info(MyPackage) # ... of all cached functions of a module, empty_caches! likewise -set_cache_size!(combine, 1_000) +fib(100) # compute and cache +fib(100) # reuse the result +cache_info(fib) # size and hit/miss statistics +set_cache_size!(fib, 1000) # bound the cache ``` -Default sizes can be configured per package and per function through `LocalPreferences.toml`; see the [configuration docs](https://lkdvos.github.io/Cached.jl/dev/configuration/). +## Keep results across runs -Specialize `Cached.cachekey` to share results between equivalent inputs while the function -body receives the original arguments: +Load SQLite.jl and opt a function into disk caching: ```julia -Cached.cachekey(::typeof(combine), a, b; normalize = true) = - (Hashed(a, customhash, customequal), b, normalize) +using SQLite +Cached.DiskCacheStyle(::typeof(fib), ::Int) = DiskCache() +fib(200) # new results are kept in RAM and on disk ``` -The hook may also return a canonical key, allowing calls to different methods or argument -types to share an entry. Merged calls must accept the same cached result and return type. -`Hashed` customizes RAM equality; disk entries are matched by serialized key bytes. -See the [custom cache keys guide](https://lkdvos.github.io/Cached.jl/dev/keys/) for examples. +Calls look in RAM, then on disk, before computing. +Processes on the same machine share the database. +You control result versions and disk cleanup; see [disk caching](https://lkdvos.github.io/Cached.jl/dev/disk/). -With [SQLite.jl](https://github.com/JuliaDatabases/SQLite.jl) loaded, results can also be kept on disk, below the RAM cache, in one database per function and node: +## See what your caches are doing + +Load Tachikoma.jl to browse caches while your program runs: ```julia -using SQLite -Cached.DiskCacheStyle(::typeof(combine), a, b) = DiskCache() -disk_cache_info(combine) # its database on this node: path, entries, size +using Tachikoma +cache_dashboard() ``` -See the [disk caching docs](https://lkdvos.github.io/Cached.jl/dev/disk/). - -With [TimerOutputs.jl](https://github.com/KristofferC/TimerOutputs.jl) loaded, `enable_cache_timers!(MyPackage, to)` times the lookups and computations of the cached functions owned by `MyPackage` and its submodules, and `disable_cache_timers!(MyPackage)` brings back the zero-cost default. See the [timing docs](https://lkdvos.github.io/Cached.jl/dev/timing/). - -With [Tachikoma.jl](https://github.com/kahliburke/Tachikoma.jl) loaded, `cache_dashboard()` opens a live terminal dashboard to browse, empty and resize the caches, with a second tab for the disk caches; see the [dashboard docs](https://lkdvos.github.io/Cached.jl/dev/dashboard/). +The dashboard shows recent hit rates, storage use, and activity, with separate RAM and Disk tabs: ```text [RAM] │ Disk - 4 caches · 325 entries · 175KiB in byte-measured caches · recent hit rate 70.3% - Name ▲ Hit rate Size Kind Activity -▌Main.fib █████60%░░░░░ ▏░░░░░░░3/10k░░░░░░░░░ LRU+disk ███████ - Main.label ████100%█████ ▏░░░░░░░6/10k░░░░░░░░░ Clock ▇█ ▁▃▄▅▇ + Name Hit rate Size Kind Activity + Main.fib █████60%░░░░░ ▏░░░░░░░3/10k░░░░░░░░░ Clock+disk ███████ Main.matrix ████100%█████ ███▊175KiB/1.0MiB░░░░░ Clock ▆▇█▁▂▃▄▄ - Main.weights █████67%▊░░░░ ▋░░░░░░276/10k░░░░░░░░ Clock ▄█▆▃▇▅▂▆ - ⇥ tab ↑↓ select ⏎ resize e empty s sort r reverse / filter g refresh q quit + ⇥ tab ↑↓ select ⏎ resize e empty s sort / filter q quit ``` + +See the [dashboard guide](https://lkdvos.github.io/Cached.jl/dev/dashboard/) for controls and statistics. + +## Why Cached? + +Choose Cached when memoization needs ongoing management: memory limits, reuse across runs, or visibility into a running workload. +It brings these tools together with strategies chosen by function and argument type. +For a single in-memory cache, Memoize.jl or Memoization.jl with an LRU container may already cover your needs. +See [the comparison and tradeoffs](https://lkdvos.github.io/Cached.jl/dev/comparison/). + +Requires Julia 1.10 or later. +Start with [usage](https://lkdvos.github.io/Cached.jl/dev/usage/), [configuration](https://lkdvos.github.io/Cached.jl/dev/configuration/), or [timing](https://lkdvos.github.io/Cached.jl/dev/timing/) in the [documentation](https://lkdvos.github.io/Cached.jl/dev/). From 7daf6bb02df098519609cbc7e50cb94d3eb59882 Mon Sep 17 00:00:00 2001 From: lkdvos Date: Mon, 5 Oct 2026 14:00:39 -0400 Subject: [PATCH 3/9] docs: remove design notes and streamline repository guidance --- CLAUDE.md | 84 +++++++++++------------------ research/README.md | 21 -------- research/architecture.md | 75 -------------------------- research/cache-management.md | 73 ------------------------- research/design.md | 93 -------------------------------- research/global-cache-binding.md | 63 ---------------------- research/open-questions.md | 61 --------------------- research/origin.md | 56 ------------------- test/test_macro.jl | 2 +- 9 files changed, 32 insertions(+), 496 deletions(-) delete mode 100644 research/README.md delete mode 100644 research/architecture.md delete mode 100644 research/cache-management.md delete mode 100644 research/design.md delete mode 100644 research/global-cache-binding.md delete mode 100644 research/open-questions.md delete mode 100644 research/origin.md diff --git a/CLAUDE.md b/CLAUDE.md index 565d1bb..1ffd75b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,70 +1,48 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. +# Repository guidance ## Layout -This checkout is the `main` worktree of a bare repository: `../Cached.jl` is the bare git dir and -sibling directories of `main/` are other worktrees. Create new worktrees next to `main/` -(`git worktree add ../ -b `), never inside it. +This repository uses sibling worktrees with the bare Git repository at `../Cached.jl`. +Create additional worktrees beside the current checkout, never inside it. -## Commands +- `src/`: memoization macro, strategy dispatch, cache containers, registry, and configuration. +- `ext/`: optional SQLite persistence, Tachikoma dashboard, and TimerOutputs integration. +- `test/`: tests discovered by ParallelTestRunner. +- `docs/src/`: user guides and API reference. +- `benchmark/`: cache container and disk benchmarks. -No `Manifest.toml` is checked in (it is gitignored), so instantiate first: +See `docs/src/usage.md` for behavior and `docs/src/interface.md` for the custom cache contract. -```bash -julia --project -e 'using Pkg; Pkg.instantiate()' -``` +## Development + +Manifests are ignored; instantiate the environment before running tests or building docs. ```bash -# Run all tests (ParallelTestRunner picks up every test/*.jl except runtests.jl) -julia --project -e 'using Pkg; Pkg.test()' +# Install dependencies and run tests +julia --project -e 'using Pkg; Pkg.instantiate(); Pkg.test()' # Run a single test file -julia --project=test test/test_aqua.jl +julia --project=test test/test_macro.jl -# Build docs +# Prepare and build documentation +julia --project=docs -e 'using Pkg; Pkg.develop(path=pwd()); Pkg.instantiate()' julia --project=docs docs/make.jl -# Format (Runic). Not a project dependency; CI checks it through a reusable workflow. -julia --project=/tmp/runic -e 'using Pkg; Pkg.add("Runic"); using Runic; Runic.main(["--inplace", "src", "ext", "test", "docs"])' +# Benchmark containers +julia --project=benchmark -t 8 benchmark/containers.jl ``` -## Architecture - -`research/design.md` is the spec; the other notes in `research/` describe the archived prototype -(branch `archive/prototype`) and are background only. +Format Julia files with Runic; CI checks formatting. +Keep Markdown prose to one sentence per line. -- `src/macro.jl`: `@cached` moves the body to a method of `Cached.implementation(::typeof(f), ...)` - and leaves `f` with its original signature, calling `Cached.call`. ExprTools does the parsing. -- `src/call.jl`: `call` picks the value type `V` (annotation, or `return_type`) and dispatches on - the `CacheStyle` (`NoCache`, `GlobalCache{C}`, `TaskLocalCache{C}`). The lookup and the - implementation call are wrapped in the internal `instrument(f, Val(phase), thunk, owner)` hook, - with `owner` the package owning `f` (or its module, outside packages); its default `thunk()` compiles away. -- `src/registry.jl`: global caches, one untyped `C{Any,Any}` per function (its budget), looked up - lock-free in an atomically published `IdDict` snapshot. -- `src/containers/`: `interface.jl` is the public `AbstractCache` interface (`docs/src/interface.md`); - `slots.jl` implements it once for the internal `SlotCache` (storage in field `slots`), of which - `LRU` and `ClockCache` are eviction policies (hooks `admit!`, `touch!`, `victim`, `forget!`). - Keys are stored as `Key{Any}` and probed with a concretely typed `Key{K}`, so hits never box. -- `src/preferences.jl`: default settings from Preferences.jl, resolved once per function - (runtime > function > package > `[Cached]` > built-in), and `set_cache_preferences!`. -- `src/api.jl`: `cache_info`, `empty_caches!`, `set_cache_size!`, and the `cache_dashboard` and - `enable_cache_timers!`/`disable_cache_timers!` stubs with their error hints (registered in `__init__`). -- `src/disk.jl`: `DiskCacheStyle` (default `NoCache()`) and `DiskCache{S}` (serializer type `S`), - `diskversion`, `disk_artifact`, the process-wide switch, and `_diskcall`, the miss function of - `get!` under a `DiskCache`; the lookup and management functions are stubs for the SQLite extension. -- `ext/CachedSQLiteExt.jl`: the disk caches. Per function type an optional read-only artifact and - one SQLite database per node (WAL, `keyhash`+`key` → `value`), opened on first use; disk errors - warn once and the call continues. Lookup order: RAM, artifact, node database, compute. -- `ext/CachedTachikomaExt.jl`: the dashboard. It renders from a copy of the statistics, refreshed - every `interval`; tests render it headlessly with Tachikoma's `TestBackend`. Two tabs (`Tab` - structs, Tachikoma's `TabBar`): RAM, and Disk, whose rows come from `Cached.disk_cache_stats()` - (in-memory counters of the SQLite extension) and whose entries and sizes (`disk_cache_info`, - a `count(*)`) are read for all rows on a background task while it is shown, at most every 10 s. -- `ext/CachedTimerOutputsExt.jl`: `enable_cache_timers!(M)` evals a method of `instrument` for - the owner of `M` into the extension, `disable_cache_timers!(M)` deletes it (deleting an - overwritten method would revive the old one, so enabling deletes before defining). +## Implementation constraints -Hot-path invariants, checked by the tests: a cache hit is fully inferred and allocates nothing. -Benchmark with `julia --project=benchmark -t 8 benchmark/containers.jl`. +- `@cached` preserves the method signature and moves the body to `Cached.implementation`. +- The default global cache shares one size limit across a function's cached methods, with a separate cache per container type. +- Preserve inferred return types and allocation-free RAM hits for concrete keys with the built-in containers. +- Built-in RAM containers distinguish key types and use `hash` and `isequal`. +- `Cached.cachekey` may merge equivalent inputs; merged calls must accept the same result and return type. +- Compute misses outside cache locks so recursion works; concurrent misses may compute twice. +- Keep the call path safe for precompiled packages, without runtime method definitions. +- Disk lookup order is RAM, read-only artifact, local database, then computation. +- Dashboard rendering uses copied statistics; database counts and sizes are read in the background. diff --git a/research/README.md b/research/README.md deleted file mode 100644 index 4ec1b23..0000000 --- a/research/README.md +++ /dev/null @@ -1,21 +0,0 @@ -# Research notes - -Ideas and lessons from the first prototype of Cached.jl (March 2026), kept as inspiration for the restart. -None of this is a spec, and the restart is free to throw any of it away. - -The prototype code itself lives on the `archive/prototype` branch: - -- `c683263` *initial implementation* — one `LRU{Any,Any}` per function, multi-argument and vararg support. -- `3d11e28` *reorganize files* — split into files, one typed `LRU{K,V}` per signature bound to a module-level `const`. -- `0510573` *WIP* — lazy per-concrete-argument-type `LRU{T,V}`, created on first call through `Core.eval`. Does not run as-is. - -## Contents - -| File | What it covers | -|:-|:-| -| [design.md](design.md) | **The agreed design for the rewrite** | -| [origin.md](origin.md) | Where the design comes from (TensorKit's internal `@cached`) and what the package was meant to fix | -| [architecture.md](architecture.md) | The `CacheStyle` dispatch pattern and the methods the macro generates | -| [global-cache-binding.md](global-cache-binding.md) | How the generated method finds its cache: four approaches tried, with trade-offs | -| [cache-management.md](cache-management.md) | Sizing, preferences, the registry, and introspection | -| [open-questions.md](open-questions.md) | Known bugs, pitfalls, and design questions that were never settled | diff --git a/research/architecture.md b/research/architecture.md deleted file mode 100644 index 667ce54..0000000 --- a/research/architecture.md +++ /dev/null @@ -1,75 +0,0 @@ -# Prototype architecture - -## Strategy dispatch - -The core idea is to decide *how* to cache at call time, through a trait that users can specialize: - -```julia -abstract type CacheStyle end -struct NoCache <: CacheStyle end -struct TaskLocalCache{D <: AbstractDict} <: CacheStyle end -struct GlobalLRUCache <: CacheStyle end - -CacheStyle(args...) = GlobalLRUCache() # default -``` - -Users override it per function and per argument type: - -```julia -Cached.CacheStyle(::typeof(expensive), ::BigMatrix) = NoCache() -Cached.CacheStyle(::typeof(expensive), ::ThreadSafeKey) = TaskLocalCache{Dict{Any,Any}}() -``` - -They can also bypass dispatch explicitly with `expensive(NoCache(), x)`. - -Because `CacheStyle` is a pure function of types in the common case, the compiler can constant-fold it and the dispatch costs nothing. - -## Generated methods - -`@cached function f(x::T)::R where {T} body end` produced four methods: - -| Method | Role | -|:-|:-| -| `f(x)` | dispatch wrapper: `f(CacheStyle(f, x), x)::R` | -| `f(::NoCache, x)` | the user's body, verbatim | -| `f(::TaskLocalCache{D}, x)` | `get!(get!(D, task_local_storage(), key), x) do f(NoCache(), x) end::R` | -| `f(::GlobalLRUCache, x)` | `get!(lru, x) do f(NoCache(), x) end::R` | - -The return-type annotation is optional. -When present, it is asserted at every entry point, which matters because the global LRU tends to be `Any`-valued. - -## Macro parsing - -`splitdef` peeled off the three optional layers of a function head in order: - -``` -function f(args...) where {P...} :: ReturnType - ↑ :call ↑ :where ↑ ::(2-arg) -``` - -(In the AST the `where` is outermost, then the `::`, then the `:call`.) - -Argument forms handled were `x`, `x::T`, `::T` (with a gensym'd name), and in the first version also `x...` and `x::T...`. -Multi-argument keys were `(args...)` tuples, and single-argument keys were the bare argument. - -Rejected outright: - -- keyword arguments -- default argument values - -The reason is that the generated wrapper methods would multiply the defaults, and kwargs would need to become part of the key. - -The last iteration also restricted the macro to **single-argument functions**. -This was partly to simplify typed LRU inference and partly because of an arity clash: see "strategy as first argument" in [open-questions.md](open-questions.md). - -## Duplicate registration - -Each `(function, signature)` could be `@cached` only once. -Registering it again errors, because a second `@cached` would silently replace the methods while leaving a stale cache registered. -Detection was keyed on a signature *string* such as `"f(::Int)"`, which has its own problems (see open questions). - -## Things that worked well - -- The strategy trait is the right abstraction. It covers "don't cache trivially cheap keys", "cache per task to avoid lock contention", and "share globally" with one mechanism. -- Asserting the return type at every entry point kept call sites inferable even with `Any`-valued caches. -- Embedding helper functions by value in generated code (`$(_tasklocal_key)(f)` rather than `Cached._tasklocal_key`) made the macro work regardless of what the calling module imported. diff --git a/research/cache-management.md b/research/cache-management.md deleted file mode 100644 index f759cd5..0000000 --- a/research/cache-management.md +++ /dev/null @@ -1,73 +0,0 @@ -# Cache management - -## Sizing - -The prototype used two modes, both built on LRUCache.jl's `by` function: - -- **Count-based**: `by = _COUNT_BY`, where `const _COUNT_BY = Returns(1)` is a sentinel. - Testing `lru.by === _COUNT_BY` was how the code told the two modes apart, which is cheap but a hack. -- **Byte-based**: `by = Base.summarysize` by default, replaceable through `GLOBALCACHE_SIZE_FUNCTION[] = sizeof`. - -The defaults were 64 GiB byte-based, or 10 000 entries in count mode. - -Lessons: - -- `Base.summarysize` on **every insert** is expensive for large nested values and can easily cost more than the computation it caches. -- A 64 GiB default per signature is effectively unbounded, and the total across caches was never bounded. - A process-wide memory budget may be what users actually want. -- Switching an existing LRU between count and byte mode means emptying it, because the stored sizes become meaningless. - -## Runtime API - -```julia -set_cache_size!(f, 10_000) # count-based, all caches of f -set_cache_bytesize!(f, 500_000_000; by = sizeof) # byte-based, clears entries -empty_globalcaches!() # clear all (also resets hit/miss counters) -caches_for(f) # the LRU objects for f -global_cache_info() # summary table -``` - -Missing pieces: - -- per-function `empty!` -- hit/miss reset independent of clearing -- task-local caches, which none of these functions touch - -## Preferences - -The defaults were loaded from `LocalPreferences.toml` at module load time, through `@load_preference("globalcache_bytesize")` and `@load_preference("globalcache_size")`. -`set_default_cache_bytesize!` wrote them and needed a restart to take effect. - -The open question is whose preferences these should be. -Preferences are keyed on the package that calls `@load_preference`, so as written they configure *Cached.jl itself*, globally, for all downstream packages. -Per-package defaults, for example "TensorKit's caches default to 2 GiB", would need the macro to call `@load_preference` inside the user's module. - -## Registry - -There were three iterations: - -1. `GLOBAL_CACHES::IdDict{Function, LRU{Any,Any}}`. -2. `PER_SIG_CACHES::Dict{String, Any}` (signature string to LRU), plus `_FUNC_TO_SIG_KEYS::IdDict{Function, Vector{String}}`. -3. `GLOBAL_CACHE_TABLE::IdDict{Function, IdDict{Type, Any}}` (function, then concrete argument type, then LRU), plus `_REGISTERED_STATIC_SIGS::Set{String}` for duplicate detection. - -The registry has two jobs that are worth keeping separate: **introspection** (list everything, with stats) and **duplicate detection** (at macro evaluation time). - -Precompilation caveat: a registry that is mutated at top level by downstream modules only contains entries from modules loaded in the *current* session. -For precompiled packages, registration has to happen in `__init__`, or the registry has to be reconstructed from the module's consts. - -## Introspection output - -`global_cache_info()` printed a box-drawn table: - -``` -Cache Usage Summary (2 caches) -┌──────────────┬──────┬────────┬──────────┬──────┬───────────────────┐ -│ Signature │ Hits │ Misses │ Hit rate │ Fill │ Memory │ -├──────────────┼──────┼────────┼──────────┼──────┼───────────────────┤ -│ expensive(…) │ 1234 │ 56 │ 95.7% │ 42 │ 3.2 MB / 64.0 GB │ -└──────────────┴──────┴────────┴──────────┴──────┴───────────────────┘ -``` - -In byte mode, Fill shows the entry count and Memory shows `current / max`. -In count mode, Fill shows `current/max` and Memory shows `summarysize(lru)`. -This was hand-rolled with `Printf`. Consider instead a `show` method on a returned `CacheInfo` vector, which can be consumed programmatically and also prints nicely. diff --git a/research/design.md b/research/design.md deleted file mode 100644 index f7bba68..0000000 --- a/research/design.md +++ /dev/null @@ -1,93 +0,0 @@ -# Design - -The decisions behind Cached.jl and the invariants the code relies on. User-facing details are -in `docs/src`; the prototype notes in this folder hold the rejected alternatives. - -## Goals - -- Replace TensorKit's internal `@cached` and the hand-rolled caches of SUNRepresentations.jl. -- Type stability: a type-stable function called with concrete arguments infers to its value - type, and a cache hit allocates nothing. -- Full argument support: positional, default, varargs and keyword arguments. -- Precompilation-safe: no `Core.eval` or runtime method definitions in the call path. -- Small and maintainable: the macro only rewrites syntax; all logic is ordinary code. - -Non-goal: a memory budget per process. - -## Macro - -`@cached` moves the body into `Cached.implementation(::typeof(f), args...; kw...)`, with -defaults removed, and leaves `f` with its original signature calling -`Cached.call(f, CacheStyle(f, args...), V, args, kw)`. Julia's lowering handles defaults and -keywords, and dispatch on `typeof(f)` covers qualified names, operators and callable objects. - -- `cachekey(f, args...; kw...)` defaults to the tuple of positional arguments, plus the - keyword `NamedTuple` if any. Custom keys can merge calls of different argument types or - methods; those calls must admit the same result, including its return type. Computation - and strategy selection still use the original arguments. -- `Hashed(value, hashf, eqf)` supplies custom hashing and equality without redefining them - on the input type. Wrappers require identical hash and equality callables (`===`) to - compare equal. Canonical keys can also merge disk entries; wrapper equality alone cannot, - because disk caching compares serialized bytes. -- `V` is the return annotation, or `Core.Compiler.return_type` of `implementation`, falling back - to `Any` when not concrete. The result is asserted `::V`, which keeps calls inferred. - -## Strategies - -`CacheStyle(f, args...)` picks `NoCache`, `GlobalCache{C}` or `TaskLocalCache{C}` per function -and argument type; the default is `GlobalCache()`, whose container is the compile-time -`container` preference (`ClockCache`). `DiskCacheStyle` (below) is independent of it. - -- Each function has **one** global `C{Any,Any}` cache, which is its memory budget. Typed - sub-caches per signature bounded memory poorly (TensorKit reached ~50 per function). -- Task-local caches live in `task_local_storage()` and are not registered. - -## Containers - -- The public `AbstractCache` interface is only what the machinery calls (`docs/src/interface.md`). - `AbstractCache <: AbstractDict` is kept so `LRU` and `ClockCache` stay dictionaries. -- `LRU` and `ClockCache` share an internal `SlotCache` implementation on a `Slots` field and - differ only in four eviction hooks (`admit!`, `touch!`, `victim`, `forget!`). -- The index is a `Dict{Key{Any},Int}` holding each key with its hash; lookups probe with a - concrete `Key{K}` whose `isequal` checks the type first, so hits never box the key. Keys of - different types are different entries (`f(3)` and `f(3.0)`). -- One lock per cache; `get!` computes outside it, so functions may recurse into their own cache. -- Lock contention on shared caches with cheap keys is real in microbenchmarks but did not show in - TensorKit's workloads. Lock-free `ClockCache` hits are parked on the `lockfree-clock` branch - (#9); `TaskLocalCache` is the workaround. - -## Registry - -Caches are created on a function's first call. The hot path reads an immutable `IdDict` -snapshot through an atomic field, without locking; creating a cache republishes the snapshot -under a lock. For singleton functions the lookup key is a constant type, so its hash is cached. - -## Configuration - -`maxsize` and `measure` (and the disk settings) come from Preferences.jl, resolved once per -function at runtime: runtime calls, then `[.Cached.]`, `[.Cached]`, -`[Cached]`, built-in defaults. `` owns `parentmodule(typeof(f))`. Only `container` is -compile-time. See `docs/src/configuration.md`. - -## Extensions - -- **Timing** (`CachedTimerOutputsExt`, `docs/src/timing.md`). Calls go through an internal - `instrument(f, phase, thunk, ::Val{owner})` that compiles to `thunk()`. Enabling a package - evals one method for its owner tuple into the extension (recompiling only its callers); - disabling deletes it. Enabling first deletes any existing method, since deleting an - overwriting method revives the old one. The owner is computed outside the closures: - otherwise Julia 1.10 boxes them. -- **Dashboard** (`CachedTachikomaExt`, `docs/src/dashboard.md`). Renders from copied statistics, - never holding a cache lock across frames. Tabs for RAM and disk; the Disk tab uses in-memory - counters, and reads entries and sizes (`disk_cache_info`) in a background task, never on a - frame. Emptying and resizing act on RAM only. -- **Disk** (`CachedSQLiteExt`, `docs/src/disk.md`). Below RAM: RAM, then artifact, then the - node's SQLite database, then compute. One WAL-mode database per function, version and host, - shared by the node's processes. `diskversion(f)` is the only version: keeping stored results - valid (including across Julia versions) is the user's responsibility. Functions without a - disk style compile exactly as before. -- Each extension's entry points are stubs in the core with an error hint. - -## Open questions - -- Should redefining a cached method (Revise) empty its caches? diff --git a/research/global-cache-binding.md b/research/global-cache-binding.md deleted file mode 100644 index 4a79f22..0000000 --- a/research/global-cache-binding.md +++ /dev/null @@ -1,63 +0,0 @@ -# How the global method finds its cache - -The `GlobalLRUCache` method must reach an LRU object. -How it gets that reference controls type stability, precompilation safety, and lookup cost. -The prototype tried four approaches. - -## 1. One `LRU{Any,Any}` per function, behind a lookup (`c683263`) - -```julia -const GLOBAL_CACHES = IdDict{Function, LRU{Any,Any}}() -_global_cache(f) = GLOBAL_CACHES[f] - -f(::GlobalLRUCache, args...) = get!($(_global_cache)(f), (args...)) do ... end -``` - -- ✅ Precompilation-safe: no live objects embedded in code. -- ❌ An `IdDict` lookup on every call, plus type-unstable keys and values. -- ❌ All signatures of `f` share one cache. - -## 2. One typed `const` per signature (`3d11e28`, the `@cached` default) - -```julia -const _cached_f__Int = _make_typed_global_lru(Int, Float64) # K, V from the macro -f(::GlobalLRUCache, x::Int) = get!(_cached_f__Int, x) do ... end::Float64 -_register_per_sig_cache!("f(::Int)", _cached_f__Int, f) -``` - -- ✅ Precompilation-safe. The const binding resolves to a direct pointer, with no lookup. -- ✅ Typed when the signature is concrete. -- ❌ `K` and `V` must be known **at macro-expansion time**. They fall back to `Any` when the argument type mentions a where-parameter, or when the return type is computed (`_fsdicttype(K)`), which is exactly the TensorKit use case. -- ❌ The const name is derived from the type expression as a string, so it can collide and is ugly. - -## 3. Embed the LRU by value with `Core.eval` (`@cached_direct`, mentioned in `3d11e28`) - -The LRU object is spliced directly into the method body as a literal, which makes it a true closure. - -- ✅ The cheapest possible lookup. -- ❌ Not precompilation-safe, because a serialized method would point at an object that only existed in the precompile process. Usable only in the REPL or in scripts. - -## 4. Lazy per-concrete-type LRU, specialized on first call (`0510573`, WIP) - -```julia -function f(s::GlobalLRUCache, x) # generic fallback - lru = _ensure_global_lru!(f, typeof(x), V) # GLOBAL_CACHE_TABLE[f][T] - _eval_global_method!(f, typeof(x), lru, @__MODULE__) # Core.eval f(::GlobalLRUCache, ::T) with lru embedded - return Base.invokelatest(f, s, x) -end -``` - -- ✅ Each concrete argument type gets its own `LRU{T,V}`, so keys are concrete. -- ✅ After the first call, the lookup is as cheap as approach 3. -- ❌ Not precompilation-safe. It also evals into *other people's modules* at runtime, which fails if that module is closed (precompiled packages) and is generally hostile. -- ❌ Every first call for a new type defines a method, which **invalidates** callers and requires `invokelatest`. The extra world-age hop runs on the first call for each type. -- ❌ `V` still comes only from the annotation, so it is not truly typed when the return type depends on `T`. -- ❌ The specialized method dropped the return-type assertion. - -## Ideas not tried yet - -- **Lookup keyed on the type**, without method definition. Generate `f(::GlobalLRUCache, x::T) where T = get!(_cache_for(f, T), x)`, where `_cache_for` is a `@generated` function or a `Base.@assume_effects :foldable` lookup in a `const` table. Done right, the compiler could fold the table access per `T`. Precompile behaviour still needs investigating. -- **A `const` cache container per function** holding `Dict{Type, LRU}` with a function barrier: `_lookup(lru::LRU{K,V}, x::K)::V`. One dynamic dispatch per call, but no codegen at runtime and fully precompilable. -- **Compute `V` with `Core.Compiler.return_type(f, Tuple{NoCache, T})`** at cache creation instead of requiring an annotation. This has the usual caveats about relying on inference results, and should probably be an opt-in. -- **Let the user name the cache type** in the macro, for example `@cached LRU{Key,Val}(maxsize=1000) function f(x) ... end`, or with a `CacheStyle` that carries the container, and skip inference entirely. -- **Store the cache in a callable struct** instead of adding methods to `f`. For example, `const f = CachedFunction(_f_impl, LRU{K,V}())` keeps the cache in a field, so it is type-stable and precompilable for free. The cost is that `f` becomes a struct rather than a generic function, which changes how users extend it. diff --git a/research/open-questions.md b/research/open-questions.md deleted file mode 100644 index 4727c45..0000000 --- a/research/open-questions.md +++ /dev/null @@ -1,61 +0,0 @@ -# Pitfalls and open questions - -## Bugs in the archived WIP (`0510573`) - -These are recorded so the same mistakes are not repeated: - -- `_cached_tasklocal_def` uses a literal `D` and `Dvar()` inside the quote instead of interpolating `$Dvar`, assigns to `cache` but reads `$cachevar`, and uses `d.arg.name`, which is not a field of `Expr`. -- `_cached_global_def` calls `_sig_string`, which only existed in the previous revision. -- `src/try.jl` is a scratch version of the macro that refers to undefined names. -- In `3d11e28`, an untyped signature's key used the *argument name*: `f(x)` became `"f(x)"`, so `@cached f(x)` followed by `@cached f(y)` was not detected as a duplicate. - -## Semantics - -- **Key equality is `isequal`/`hash`**, so `f(3)` and `f(3.0)` share an entry and can return the wrong type. - Should keys include the type, as in `(typeof(x), x)`, or should the cache be per concrete type, as approach 4 does? -- **Mutable keys**: an array that is mutated after being used as a key corrupts the cache silently. - Options: document it, `deepcopy` keys, or refuse mutable keys under some trait. -- **Returned values are shared**: mutating a cached result mutates the cache. - TensorKit relies on callers not doing this. -- **Revise or redefinition**: redefining the body does not invalidate entries. - Could the world age or method instance be tracked, or should redefinition just empty that function's caches? -- **Recursion**: a cached function that calls itself (memoized Fibonacci) re-enters `get!` on the same LRU. - Check whether LRUCache.jl holds its lock while running the default function; if it does, that deadlocks or serializes. -- **Exceptions in the body** must not leave a half-inserted entry. - -## Concurrency - -- `GlobalLRUCache` relies on LRUCache.jl's internal lock. Under heavy multithreading this is a contention point, which is why `TaskLocalCache` exists. -- Task-local caches are **unbounded, invisible to introspection, and die with the task**. - Any `Threads.@spawn`-heavy code rebuilds them constantly. -- Is a **sharded or per-thread** cache worth adding as a fourth style? Note that `threadid()` is not stable for a task in recent Julia. -- The task-local storage key was `Symbol(:_tasklocal_, parentmodule(f), :_, nameof(f), :_cache)`, which collides for functions with the same name in modules with the same name, and for closures. - -## Macro and API design - -- **Strategy as the first argument** (`f(::NoCache, x)`) puts extra methods on the user's function. - With multiple arguments, the wrapper `f(a, b)` and the implementation `f(::NoCache, a)` have the same arity and can clash. - TensorKit puts the strategy last, which has the mirror-image problem. - Alternatives are an underscored implementation function (`_f`), or a single internal entry point `Cached.call(style, f, args...)` that keeps `f`'s method table clean. -- **Keyword arguments and default values**: users will ask for them. - Defaults are easy: expand to the full-arity call before the wrapper. - Kwargs need a key policy, such as sorting into a `NamedTuple`. -- **Multiple arguments**: the first version supported tuple keys and the last one dropped them. - Is single-argument plus "pack it into a key struct" (TensorKit's `FSPBraidKey` style) a reasonable rule, or too restrictive? -- **Return type annotation** as the source of `V`: required, optional, or inferred? -- **Duplicate registration**: is it an error or a warning? Revise users will hit it constantly. -- **Profiling hooks**: TensorKit wraps lookups and misses in `@timeit_debug`. - Should Cached.jl provide a hook, such as a callback or an extension on TimerOutputs, so packages don't lose that? - -## Precompilation and load time - -- Must `@cached` inside a precompiled package produce caches that work after loading, with correct types and registered for introspection? - This rules out `Core.eval` into foreign modules and runtime method definition. -- Should `@cached` definitions be invalidation-free for downstream code? -- How should the dependency footprint be kept minimal? LRUCache and Preferences are light, but Printf was only there for the table. - -## Scope questions to settle first - -1. Is the primary consumer the TensorKit ecosystem, or general-purpose memoization? -2. Is matching TensorKit's `CacheStyle` API exactly, so that it can be a drop-in replacement, a hard requirement? -3. How much does typed-cache performance actually matter versus `LRU{Any,Any}` plus a return assertion? Benchmark before building machinery. diff --git a/research/origin.md b/research/origin.md deleted file mode 100644 index 76bfb72..0000000 --- a/research/origin.md +++ /dev/null @@ -1,56 +0,0 @@ -# Origin - -The prototype was an extraction and generalization of the `@cached` macro that TensorKit.jl uses internally (`src/auxiliary/caches.jl`). -TensorKit uses it for fusion-tree manipulations (`fsbraid`, `fstranspose`, tree transformers) and space structure, where the same expensive symbolic computation is repeated many times with identical keys. - -## What TensorKit's version does - -```julia -@cached function fsbraid(key::K)::_fsdicttype(K) where {I, N₁, N₂, K <: Union{FSPBraidKey{I, N₁, N₂}, FSBBraidKey{I, N₁, N₂}}} - ... -end - -CacheStyle(::typeof(fsbraid), k::Union{FSPBraidKey{I}, FSBBraidKey{I}}) where {I} = - FusionStyle(I) isa UniqueFusion ? NoCache() : GlobalLRUCache() -``` - -It generates: - -- `_fsbraid(args...)` — the user's body, under an underscored name. -- `fsbraid(args...)` — calls `fsbraid(args..., CacheStyle(fsbraid, args...))`. -- `fsbraid(args..., ::NoCache)` — calls `_fsbraid` directly. -- `fsbraid(args..., ::TaskLocalCache{D})` — looks up a `D` in `task_local_storage()` under `:_tasklocal_fsbraid_cache`. -- `fsbraid(args..., ::GlobalLRUCache)` — uses `const GLOBAL_FSBRAID_CACHE = LRU{Any,Any}(; maxsize = DEFAULT_GLOBALCACHE_SIZE[])`, pushed onto `GLOBAL_CACHES`. - -It also wraps the lookup and the miss path in `@timeit_debug` sections, labeled "bookkeeping" or "symmetry", so cache costs show up in TimerOutputs profiles. - -Notable details: - -- **The strategy goes last** (`f(args..., ::Style)`), not first as in the prototype. -- **The return type can depend on where-parameters** (`_fsdicttype(K)`), so the value type is not known at macro-expansion time. - This is the main reason the prototype kept reworking how the typed LRU gets created (see [global-cache-binding.md](global-cache-binding.md)). -- **`CacheStyle` is a per-argument decision**, not just per function: for abelian symmetries it is cheaper to recompute than to hash. -- **Every cache is `LRU{Any,Any}`**, so lookups are type-unstable and rely on the return type assertion. - -## What the prototype was trying to fix - -1. Make it reusable outside TensorKit, for TensorKitSectors, MPSKit, and others, without each package copying the macro. -2. Get **concretely typed caches** so lookups are type-stable without relying only on return-type assertions. -3. Add **memory-aware sizing**, measuring bytes as well as counting entries, because fusion-tree caches can grow very large. -4. Add **better introspection**: a summary table, per-function resizing, and clearing. -5. Keep **precompilation safety**, which TensorKit needs. - -Point 2 conflicts with point 5 whenever the value type depends on runtime types. -Resolving that conflict is the central design problem. - -## Prior art in the General registry - -Look at these again before redesigning: - -- **Memoize.jl** — `@memoize`, with a custom dict type per function. Supports kwargs. Caches live in a global per method. -- **Memoization.jl** — `@memoize`, with a configurable cache type and per-function `empty_cache!`. Handles closures. -- **LRUCache.jl** — the thread-safe LRU the prototype used. It provides `cache_info`, a `by` size function, and `resize!`. -- **Caching.jl**, **CachedFunctions.jl**, **CachedCalls.jl** — smaller variants worth skimming for API ideas. - -None of them offers *per-call-site strategy dispatch* (`CacheStyle`) or task-local caches. -Those are the distinctive features. diff --git a/test/test_macro.jl b/test/test_macro.jl index 8b2588d..78c46d1 100644 --- a/test/test_macro.jl +++ b/test/test_macro.jl @@ -18,7 +18,7 @@ end @cached converted(x)::Float64 = counting(x) -# value type computed from a where parameter, as in TensorKit's `fsbraid` +# value type computed from a where parameter valtype_of(::Type{T}) where {T} = Vector{T} @cached function dependent(x::T)::valtype_of(T) where {T} return counting([x]) From 6f408bce3e975af7015e5c34028cb7b54894b784 Mon Sep 17 00:00:00 2001 From: lkdvos Date: Mon, 5 Oct 2026 14:14:42 -0400 Subject: [PATCH 4/9] ci: post documentation preview links on pull requests --- .github/workflows/Documentation.yml | 30 +++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/.github/workflows/Documentation.yml b/.github/workflows/Documentation.yml index 5c88c8b..2d76984 100644 --- a/.github/workflows/Documentation.yml +++ b/.github/workflows/Documentation.yml @@ -12,6 +12,7 @@ on: permissions: contents: write statuses: write + pull-requests: write jobs: Documentation: @@ -26,3 +27,32 @@ jobs: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # For authentication with GitHub Actions token DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }} # For authentication with SSH deploy key run: julia --project=docs/ docs/make.jl + - name: Comment with documentation preview + if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository + uses: actions/github-script@v9 + with: + script: | + const marker = ''; + const issue_number = context.issue.number; + const url = `https://lkdvos.github.io/Cached.jl/previews/PR${issue_number}/`; + const body = `${marker}\n[Preview the documentation](${url})`; + const comments = await github.paginate(github.rest.issues.listComments, { + ...context.repo, + issue_number, + }); + const comment = comments.find(c => + c.user.login === 'github-actions[bot]' && c.body?.includes(marker) + ); + if (comment) { + await github.rest.issues.updateComment({ + ...context.repo, + comment_id: comment.id, + body, + }); + } else { + await github.rest.issues.createComment({ + ...context.repo, + issue_number, + body, + }); + } From 32d5a10580b00aa240b39afb3336ea1fd3809366 Mon Sep 17 00:00:00 2001 From: lkdvos Date: Mon, 5 Oct 2026 14:24:00 -0400 Subject: [PATCH 5/9] docs: fix configuration and extension table rendering --- docs/src/configuration.md | 2 +- docs/src/dashboard.md | 2 +- docs/src/disk.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/src/configuration.md b/docs/src/configuration.md index 7cdafb8..51579b2 100644 --- a/docs/src/configuration.md +++ b/docs/src/configuration.md @@ -9,7 +9,7 @@ Default cache settings are read with [Preferences.jl](https://github.com/JuliaPa ## Settings | Key | Values | Default | Meaning | -|:-|:-|:-|:-| +| :--- | :--- | :--- | :--- | | `maxsize` | integer ≥ 0 | `10000` | limit of a function's cache, all signatures together, in entries or bytes (see `measure`) | | `measure` | `"count"` or `"bytes"` | `"count"` | count entries, or measure values with [`Cached.cachesize`](@ref) | | `container` | `"ClockCache"` or `"LRU"` | `"ClockCache"` | container of the default `CacheStyle`; only in the `[Cached]` section | diff --git a/docs/src/dashboard.md b/docs/src/dashboard.md index 4cee89a..94b4eaf 100644 --- a/docs/src/dashboard.md +++ b/docs/src/dashboard.md @@ -87,7 +87,7 @@ Narrow terminals shorten names and drop columns. The minimum size is 40×8. | Key | Action | -|:-|:-| +| :--- | :--- | | `Tab`, `1` / `2` | switch to the next tab / the RAM or Disk tab | | `↑` `↓`, `PgUp` `PgDn`, `Home` `End` | select a row | | `Enter` | resize the selected cache, in RAM (see below) | diff --git a/docs/src/disk.md b/docs/src/disk.md index fb317da..8cea261 100644 --- a/docs/src/disk.md +++ b/docs/src/disk.md @@ -26,7 +26,7 @@ Cached.DiskCacheStyle(::typeof(expensive), args...) = DiskCache() Both resolve at compile time, so functions without a disk cache are not affected, and a RAM hit costs the same with or without one. | `CacheStyle` | `DiskCacheStyle` | a call | -|:-|:-|:-| +| :--- | :--- | :--- | | `GlobalCache()` (default) | `NoCache()` (default) | RAM, else compute | | `GlobalCache()` | `DiskCache()` | RAM, else disk, else compute | | `NoCache()` | `DiskCache()` | disk, else compute, on every call | From 299dd24902d1d477211014d9f7605cfe936ebd75 Mon Sep 17 00:00:00 2001 From: lkdvos Date: Mon, 5 Oct 2026 14:24:02 -0400 Subject: [PATCH 6/9] docs: consolidate home page and explain eviction policies --- CLAUDE.md | 2 +- README.md | 4 +- docs/make.jl | 17 ++++- docs/src/comparison.md | 41 ----------- docs/src/eviction.md | 57 +++++++++++++++ docs/src/index.md | 157 ++++++++++++++++++++++++++++++++++++++++- docs/src/usage.md | 139 ------------------------------------ 7 files changed, 230 insertions(+), 187 deletions(-) delete mode 100644 docs/src/comparison.md create mode 100644 docs/src/eviction.md delete mode 100644 docs/src/usage.md diff --git a/CLAUDE.md b/CLAUDE.md index 1ffd75b..474778d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,7 +11,7 @@ Create additional worktrees beside the current checkout, never inside it. - `docs/src/`: user guides and API reference. - `benchmark/`: cache container and disk benchmarks. -See `docs/src/usage.md` for behavior and `docs/src/interface.md` for the custom cache contract. +See `docs/src/index.md` for behavior and `docs/src/interface.md` for the custom cache contract. ## Development diff --git a/README.md b/README.md index 8e67b4e..8f25f63 100644 --- a/README.md +++ b/README.md @@ -69,7 +69,7 @@ See the [dashboard guide](https://lkdvos.github.io/Cached.jl/dev/dashboard/) for Choose Cached when memoization needs ongoing management: memory limits, reuse across runs, or visibility into a running workload. It brings these tools together with strategies chosen by function and argument type. For a single in-memory cache, Memoize.jl or Memoization.jl with an LRU container may already cover your needs. -See [the comparison and tradeoffs](https://lkdvos.github.io/Cached.jl/dev/comparison/). +See [the comparison and tradeoffs](https://lkdvos.github.io/Cached.jl/dev/#why-cached). Requires Julia 1.10 or later. -Start with [usage](https://lkdvos.github.io/Cached.jl/dev/usage/), [configuration](https://lkdvos.github.io/Cached.jl/dev/configuration/), or [timing](https://lkdvos.github.io/Cached.jl/dev/timing/) in the [documentation](https://lkdvos.github.io/Cached.jl/dev/). +Start with [usage](https://lkdvos.github.io/Cached.jl/dev/#usage), [configuration](https://lkdvos.github.io/Cached.jl/dev/configuration/), or [timing](https://lkdvos.github.io/Cached.jl/dev/timing/) in the [documentation](https://lkdvos.github.io/Cached.jl/dev/). diff --git a/docs/make.jl b/docs/make.jl index fc5b9a5..b60d64c 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -13,9 +13,20 @@ makedocs(; assets = String[], ), pages = [ - "Home" => "index.md", "Usage" => "usage.md", "Why Cached?" => "comparison.md", - "Custom cache keys" => "keys.md", "Configuration" => "configuration.md", "Dashboard" => "dashboard.md", - "Timing" => "timing.md", "Disk caching" => "disk.md", "Implementing a cache" => "interface.md", + "Home" => "index.md", + "Caches" => [ + "Configuration" => "configuration.md", + "Eviction policies" => "eviction.md", + "Disk caching" => "disk.md", + ], + "Monitoring" => [ + "Dashboard" => "dashboard.md", + "Timing" => "timing.md", + ], + "Customization" => [ + "Custom cache keys" => "keys.md", + "Implementing a cache" => "interface.md", + ], "Reference" => "reference.md", ], ) diff --git a/docs/src/comparison.md b/docs/src/comparison.md deleted file mode 100644 index 3393a07..0000000 --- a/docs/src/comparison.md +++ /dev/null @@ -1,41 +0,0 @@ -# Why Cached? - -Cached is for memoization that needs ongoing management, especially in long-running or repeated computations. -Its niche is the combination of bounded RAM caches, optional persistence across runs, and tools to inspect and adjust caches while a program runs. -You can apply these controls per function or package without writing cache bookkeeping around each computation. - -For example, a workload can keep frequently used results in RAM, recover older results from disk after a restart, and show which functions benefit from caching in the dashboard. -Strategy dispatch lets the same function share expensive results across tasks, keep independent task-local caches, or recompute cheap argument types. - -## Choosing a package - -| Package | Choose it when | Tradeoff | -| :-- | :-- | :-- | -| [Memoize.jl](https://github.com/JuliaCollections/Memoize.jl#usage) | You want a small `@memoize` API with a dictionary of your choice. | Limits and statistics depend on the container; persistence, a dashboard, and package-level settings require additional integration. | -| [Memoization.jl](https://github.com/marius311/Memoization.jl#readme) | You need to memoize individual calls or closures, as well as method definitions. | Its default cache is unbounded and not thread-safe; a custom container can address those needs, but the management and persistence tools require additional integration. | -| [LRUCache.jl](https://github.com/JuliaCollections/LRUCache.jl#readme) | You want a thread-safe cache dictionary with size limits, statistics, resizing, and eviction callbacks. | You write the function wrapper and management code yourself, or combine it with a memoization macro. | -| Cached.jl | You want bounded caches, per-type strategies, disk persistence, and live monitoring under one API. | The macro requires a method definition; disk results need explicit versioning and cleanup, and shared caches use locks. | - -Memoize.jl and Memoization.jl both accept LRUCache.jl containers, so bounded memory and statistics are available without Cached. -Memoization.jl also preserves return-type inference. -Cached's benefit is that strategies, settings, persistence, and monitoring work together out of the box, with SQLite.jl and Tachikoma.jl loaded for their respective features. - -## Differences that affect behavior - -**Key matching:** Memoize.jl and Memoization.jl default to `IdDict`, matching arguments by identity; both allow a different dictionary. -By default, Cached's built-in caches match by value using `hash` and `isequal`, and distinguish argument types, so `f(3)` and `f(3.0)` occupy separate entries. -Custom cache keys can merge equivalent inputs across argument types; see [Custom cache keys](keys.md). -Value-based matching can reuse separately constructed equal keys, but hashing large keys can cost more than recomputing. - -**Concurrency:** Cached's shared RAM caches are thread-safe by default, but their locks can contend and simultaneous misses may duplicate computation. -Task-local caches avoid sharing storage, at the cost of repeated computations and memory in each task; they are absent from the dashboard and global management API. -Memoization.jl documents thread safety for top-level functions with a thread-safe container, but not for closures or callable objects; see its [limitations](https://github.com/marius311/Memoization.jl#limitations). - -**Invalidation:** Cached requires manual clearing after a method or external state changes. -Memoization.jl handles method redefinition for memoized definitions, though memoized individual calls require clearing; see its [redefinition behavior](https://github.com/marius311/Memoization.jl#limitations). -For Cached's persistent results, you must also update the disk version when results or stored types change. - -**Persistence and limits:** Disk caching adds serialization and file I/O, so it suits results that are expensive to recompute. -Disk databases have no automatic eviction; RAM limits measure entries or values, not total process memory. -The dashboard monitors global caches used in the current process. -See [disk caching](disk.md) and [usage](usage.md) for the details. diff --git a/docs/src/eviction.md b/docs/src/eviction.md new file mode 100644 index 0000000..e33a39b --- /dev/null +++ b/docs/src/eviction.md @@ -0,0 +1,57 @@ +# Eviction policies + +```@meta +CurrentModule = Cached +``` + +When a RAM cache reaches its size limit, it removes entries to make room for new results. +Cached provides two policies: Clock (the default) and least recently used (LRU). +Both work with shared and task-local caches, and with limits in entries or measured value size. + +## Clock: second chances + +[`ClockCache`](@ref) approximates recent use with one reference bit per entry and a hand that walks the storage slots. +A hit sets the entry's bit. +When space is needed, the hand clears set bits and skips those entries, giving them a second chance. +It removes the first entry it finds with a clear bit and continues from that position on the next eviction. + +New entries start with a clear bit: they get a second chance only after a cache hit. +For example, fill a three-entry cache with A, B, and C, then hit A. +Inserting D starts the hand at A, clears A's bit, and evicts B, whose bit is clear. +Unlike LRU, Clock does not track the exact order of hits. +A recently inserted entry that has never been hit again can be evicted before an older entry that has been reused. + +Choose Clock for approximate recency with little bookkeeping on hits: a hit updates a bit rather than a recency list. +Eviction may scan several slots to find a victim. + +## LRU: exact recency + +[`LRU`](@ref) keeps entries ordered by their last use. +A hit moves an entry to the most recently used position, and a new entry starts there too. +When space is needed, the least recently used entry is removed. + +For example, fill a three-entry cache with A, B, and C, then hit A. +The order from least to most recently used becomes B, C, A, so inserting D evicts B. +Unlike Clock, LRU always retains the exact recency order, even after several hits between evictions. + +Choose LRU when that ordering suits the workload. +Maintaining the order requires more bookkeeping on hits, but choosing the next victim does not require a scan. +Neither policy guarantees a better hit rate for every workload; use the [dashboard](dashboard.md) or [timing](timing.md) to evaluate yours. + +## Selecting a policy + +```julia +Cached.CacheStyle(::typeof(expensive), args...) = GlobalCache{LRU}() +Cached.CacheStyle(::typeof(other), args...) = TaskLocalCache{ClockCache}() +``` + +To set the default container for `GlobalCache()` and `TaskLocalCache()`, use the `container` preference in [Configuration](configuration.md). +Custom eviction policies can be supplied through the [cache interface](interface.md). + +## Size limits + +Both policies evict as many entries as needed to fit a new value or a reduced limit. +A value larger than the entire limit is returned without being stored. +Byte limits measure cached values, not keys or container overhead. +Shared caches use locks under either policy; computations run outside those locks. +Disk caches have no eviction policy; see [Disk caching](disk.md) for cleanup. diff --git a/docs/src/index.md b/docs/src/index.md index f4c2af3..ac23aef 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -42,4 +42,159 @@ fib(100) Calling `fib(100)` again reuses the stored result. By default, each function gets a shared Clock cache with room for 10,000 entries across all its cached methods. -Continue with [Usage](usage.md) for strategies and cache management, or [Why Cached?](comparison.md) for the motivation and comparison with other packages. +## [Using Cached](@id usage) + +### Memoizing a method + +Put [`@cached`](@ref) before a method definition. +By default, its positional and keyword arguments form the cache key, and its body runs only on a miss. + +```jldoctest powers +using Cached + +@cached function powers(x::T, n::Int = 3; offset::T = zero(T))::Vector{T} where {T} + [x^i + offset for i in 1:n] +end; + +powers(2; offset = 1) + +# output + +3-element Vector{Int64}: + 3 + 5 + 9 +``` + +The macro also supports varargs, qualified names, operators, and callable objects. +Anonymous functions are not supported. +Methods without `@cached` keep their usual behavior. + +A return annotation is optional and may depend on `where` parameters. +Without one, Cached uses the inferred return type, falling back to `Any` if it is not concrete. +A type-stable body keeps calls inferred even though the shared cache holds results of different types. + +Use [`uncached`](@ref) to call the body without reading or filling RAM or disk caches. +Supply all arguments, including defaults: defaults belong to the wrapper, not the uncached body. + +```jldoctest powers +uncached(powers, 2, 3; offset = 1) + +# output + +3-element Vector{Int64}: + 3 + 5 + 9 +``` + +### Choosing a strategy + +Define [`CacheStyle`](@ref)`(f, args...)` for the function and positional argument types you want to customize. +A strategy based on types can be resolved by the compiler. + +| Strategy | Use it when | +| :-- | :-- | +| [`GlobalCache()`](@ref GlobalCache) | Tasks should share results (the default). | +| [`GlobalCache{LRU}()`](@ref GlobalCache) | You want exact least-recently-used eviction. | +| [`TaskLocalCache()`](@ref TaskLocalCache) | Tasks should keep separate caches, reducing contention on shared storage. | +| [`NoCache()`](@ref NoCache) | Computing is cheaper than caching, or results should not be reused. | + +For example, give each task its own cache for a recursive computation: + +```jldoctest +using Cached +@cached fib(n::Int)::BigInt = n < 2 ? BigInt(n) : fib(n - 1) + fib(n - 2); + +Cached.CacheStyle(::typeof(fib), ::Int) = TaskLocalCache{LRU}(); + +fib(10) + +# output + +55 +``` + +To skip caching floating-point calls to `powers`: + +```julia +Cached.CacheStyle(::typeof(powers), ::AbstractFloat, ::Int) = NoCache() +``` + +`GlobalCache()` and `TaskLocalCache()` use the [configured container](configuration.md), Clock by default. +See [Eviction policies](eviction.md) for how Clock and LRU choose entries to remove. +[`GlobalLRUCache()`](@ref GlobalLRUCache) is an alias for `GlobalCache{LRU}()`. + +Task-local caches live for the task's lifetime and are absent from the global management API and dashboard. +Built-in task-local containers are bounded, but `TaskLocalCache{Dict}()` is unbounded. +Task-local caches can be separate per key and value type; their limits are not a single budget for the whole function. + +[`DiskCacheStyle`](@ref) selects disk caching independently. +`NoCache()` as the RAM strategy can still read and fill a disk cache; see [Disk caching](disk.md). + +### Inspecting and limiting caches + +Global caches are created on first use. +Normally, one cache holds all cached methods and argument types of a function. +If its strategy selects several container types, each gets a separate cache with the function's size limit. + +```julia +cache_info(powers) # live f => cache pairs, with size and statistics +cache_info(MyPackage) # functions owned by this module and submodules +cache_info() # all global caches +set_cache_size!(powers, 1000) # count entries +set_cache_size!(powers, 2^20; by = Cached.cachesize) # measure values in bytes +empty_caches!(powers) # clear entries, keep hit/miss counters +empty_caches!(MyPackage) # clear the package's global caches +``` + +Shrinking a cache evicts entries; changing its size measure discards it. +A value larger than the limit is returned but not retained by the built-in containers. +Byte limits measure values, not keys or container overhead, and are not a process-wide memory budget. +See [Configuration](configuration.md) for persistent defaults and custom size measures. + +### Keys, results, and invalidation + +- Default keys use `hash` and `isequal`, with an additional type check: `f(3)` and `f(3.0)` are different entries. + Keyword values are included; forwarded keyword order can also distinguish keys. +- Keep keys unchanged while cached. + Mutating an array used as a key can invalidate its hash. +- RAM results are shared objects. + Copy a mutable result before changing it. +- Use caching for computations determined by their arguments. + Changes to external state or method definitions (including through Revise) do not automatically invalidate results. + Clear the global cache with `empty_caches!`; for disk results, update [`Cached.diskversion`](@ref). +- Built-in shared caches are thread-safe. + Computations run outside the cache lock, allowing recursion; simultaneous misses may compute the same key more than once. + Exceptions are propagated without storing a result. + +Use [`Cached.cachekey`](@ref) to map equivalent inputs to a common key, or [`Hashed`](@ref) to customize hashing and equality. +See [Custom cache keys](keys.md) for examples and the requirements for sharing results. + +## [Why Cached?](@id why-cached) + +Cached is for memoization that needs ongoing management: bounded RAM caches, reuse across runs, and tools to inspect and adjust caches while a program runs. +Strategies, settings, persistence, and monitoring work together under one function-level API. + +| Package | Choose it when | Tradeoff | +| :--- | :--- | :--- | +| [Memoize.jl](https://github.com/JuliaCollections/Memoize.jl#usage) | You want a small `@memoize` API with a dictionary of your choice. | Limits and statistics depend on the container; persistence, a dashboard, and package settings require additional integration. | +| [Memoization.jl](https://github.com/marius311/Memoization.jl#readme) | You need to memoize individual calls or closures, as well as method definitions. | Its default cache is unbounded and not thread-safe; custom containers can address those needs, but management and persistence tools require additional integration. | +| [LRUCache.jl](https://github.com/JuliaCollections/LRUCache.jl#readme) | You want a thread-safe cache dictionary with limits, statistics, resizing, and eviction callbacks. | You write the function wrapper and management code, or combine it with a memoization macro. | +| Cached.jl | You want bounded caches, per-type strategies, disk persistence, and live monitoring under one API. | The macro requires a method definition; shared caches use locks, and disk results need explicit versioning and cleanup. | + +Memoize.jl and Memoization.jl both accept LRUCache.jl containers, so bounded memory and statistics are available without Cached. +Memoization.jl also preserves return-type inference. +For a single in-memory cache, these packages may already cover your needs. + +**Key matching:** Memoize.jl and Memoization.jl default to identity-based `IdDict` caches and allow other dictionaries. +Cached defaults to value-based matching with distinct entries for different argument types; [custom keys](keys.md) can merge equivalent inputs. +Hashing large keys can cost more than recomputing. + +**Invalidation:** Cached requires manual clearing after code or external state changes. +Memoization.jl handles method redefinition for memoized definitions, though memoized individual calls require clearing; see its [limitations](https://github.com/marius311/Memoization.jl#limitations). +Those limitations also describe thread safety for top-level functions with a thread-safe container, but not closures or callable objects. + +Disk caching adds serialization and file I/O, so it suits expensive computations. +Disk storage has no automatic eviction, and RAM limits are per cache rather than process-wide. +See [Disk caching](disk.md) and [Configuration](configuration.md) for these controls. diff --git a/docs/src/usage.md b/docs/src/usage.md deleted file mode 100644 index c485822..0000000 --- a/docs/src/usage.md +++ /dev/null @@ -1,139 +0,0 @@ -# Usage - -```@meta -CurrentModule = Cached -``` - -## Memoizing a method - -Put [`@cached`](@ref) before a method definition. -By default, its positional and keyword arguments form the cache key, and its body runs only on a miss. - -```jldoctest powers -using Cached - -@cached function powers(x::T, n::Int = 3; offset::T = zero(T))::Vector{T} where {T} - [x^i + offset for i in 1:n] -end; - -powers(2; offset = 1) - -# output - -3-element Vector{Int64}: - 3 - 5 - 9 -``` - -The macro also supports varargs, qualified names, operators, and callable objects. -Anonymous functions are not supported. -Methods without `@cached` keep their usual behavior. - -A return annotation is optional and may depend on `where` parameters. -Without one, Cached uses the inferred return type, falling back to `Any` if it is not concrete. -A type-stable body keeps calls inferred even though the shared cache holds results of different types. - -Use [`uncached`](@ref) to call the body without reading or filling RAM or disk caches. -Supply all arguments, including defaults: defaults belong to the wrapper, not the uncached body. - -```jldoctest powers -uncached(powers, 2, 3; offset = 1) - -# output - -3-element Vector{Int64}: - 3 - 5 - 9 -``` - -## Choosing a strategy - -Define [`CacheStyle`](@ref)`(f, args...)` for the function and positional argument types you want to customize. -A strategy based on types can be resolved by the compiler. - -| Strategy | Use it when | -| :-- | :-- | -| [`GlobalCache()`](@ref GlobalCache) | Tasks should share results (the default). | -| [`GlobalCache{LRU}()`](@ref GlobalCache) | You want exact least-recently-used eviction. | -| [`TaskLocalCache()`](@ref TaskLocalCache) | Tasks should keep separate caches, reducing contention on shared storage. | -| [`NoCache()`](@ref NoCache) | Computing is cheaper than caching, or results should not be reused. | - -For example, give each task its own cache for a recursive computation: - -```jldoctest -using Cached -@cached fib(n::Int)::BigInt = n < 2 ? BigInt(n) : fib(n - 1) + fib(n - 2); - -Cached.CacheStyle(::typeof(fib), ::Int) = TaskLocalCache{LRU}(); - -fib(10) - -# output - -55 -``` - -To skip caching floating-point calls to `powers`: - -```julia -Cached.CacheStyle(::typeof(powers), ::AbstractFloat, ::Int) = NoCache() -``` - -`GlobalCache()` and `TaskLocalCache()` use the [configured container](configuration.md), Clock by default. -Clock gives recently used entries a second chance during eviction; LRU evicts the least recently used entry. -[`GlobalLRUCache()`](@ref GlobalLRUCache) is an alias for `GlobalCache{LRU}()`. - -Task-local caches live for the task's lifetime and are absent from the global management API and dashboard. -Built-in task-local containers are bounded, but `TaskLocalCache{Dict}()` is unbounded. -Task-local caches can be separate per key and value type; their limits are not a single budget for the whole function. - -[`DiskCacheStyle`](@ref) selects disk caching independently. -`NoCache()` as the RAM strategy can still read and fill a disk cache; see [Disk caching](disk.md). - -## Inspecting and limiting caches - -Global caches are created on first use. -Normally, one cache holds all cached methods and argument types of a function. -If its strategy selects several container types, each gets a separate cache with the function's size limit. - -```julia -cache_info(powers) # live f => cache pairs, with size and statistics -cache_info(MyPackage) # functions owned by this module and submodules -cache_info() # all global caches -set_cache_size!(powers, 1000) # count entries -set_cache_size!(powers, 2^20; by = Cached.cachesize) # measure values in bytes -empty_caches!(powers) # clear entries, keep hit/miss counters -empty_caches!(MyPackage) # clear the package's global caches -``` - -Shrinking a cache evicts entries; changing its size measure discards it. -A value larger than the limit is returned but not retained by the built-in containers. -Byte limits measure values, not keys or container overhead, and are not a process-wide memory budget. -See [Configuration](configuration.md) for persistent defaults and custom size measures. - -## Keys, results, and invalidation - -- Default keys use `hash` and `isequal`, with an additional type check: `f(3)` and `f(3.0)` are different entries. - Keyword values are included; forwarded keyword order can also distinguish keys. -- Keep keys unchanged while cached. - Mutating an array used as a key can invalidate its hash. -- RAM results are shared objects. - Copy a mutable result before changing it. -- Use caching for computations determined by their arguments. - Changes to external state or method definitions (including through Revise) do not automatically invalidate results. - Clear the global cache with `empty_caches!`; for disk results, update [`Cached.diskversion`](@ref). -- Built-in shared caches are thread-safe. - Computations run outside the cache lock, allowing recursion; simultaneous misses may compute the same key more than once. - Exceptions are propagated without storing a result. - -Use [`Cached.cachekey`](@ref) to map equivalent inputs to a common key, or [`Hashed`](@ref) to customize hashing and equality. -See [Custom cache keys](keys.md) for examples and the requirements for sharing results. - -## How it works - -The macro moves the body to [`Cached.implementation`](@ref) and leaves the original method as a wrapper. -Strategy dispatch, lookup, and computation use ordinary Julia code; the call path defines no methods at runtime. -Caches are created lazily, so definitions work in precompiled packages. -To supply a different container, see [Implementing a cache](interface.md). From ad3c2c4978c2cc152415005105db46a3c294fc6d Mon Sep 17 00:00:00 2001 From: lkdvos Date: Mon, 5 Oct 2026 14:31:39 -0400 Subject: [PATCH 7/9] docs: split dashboard guide into RAM and disk sections --- docs/src/dashboard.md | 45 +++++++++++++++++++++++++------------------ 1 file changed, 26 insertions(+), 19 deletions(-) diff --git a/docs/src/dashboard.md b/docs/src/dashboard.md index 94b4eaf..468b2a7 100644 --- a/docs/src/dashboard.md +++ b/docs/src/dashboard.md @@ -14,6 +14,25 @@ cache_dashboard(; interval = 0.2, filter = "fs") # faster, only functions whose ``` It has two tabs, for the caches in RAM and on disk. + +Narrow terminals shorten names and drop columns. +The minimum size is 40×8. + +| Key | Action | +| :--- | :--- | +| `Tab`, `1` / `2` | switch to the next tab / the RAM or Disk tab | +| `↑` `↓`, `PgUp` `PgDn`, `Home` `End` | select a row | +| `Enter` | resize the selected cache, in RAM | +| `e` | empty the selected cache, in RAM | +| `s` / `r` | sort by the next column / reverse the order, per tab | +| `/` | filter by function name, on both tabs (`Enter` applies, `Esc` cancels) | +| `g` | refresh now | +| `q`, `Esc` | quit | + +Statistics are copied under brief locks, so the dashboard can run while other tasks use the caches. + +## RAM caches + With a few example caches, the RAM tab looks like this (rendered while building these docs): ```@example dashboard @@ -64,6 +83,13 @@ The header totals entries and byte-measured values; the selected row's details i Recent hit rates cover the last 10 refreshes. Task-local caches are not shown. +In resize mode, `←` / `→` halve or double the limit, `[` / `]` adjust it by 10%, `Enter` applies, and `Esc` cancels. +The preview uses the cache's current measure (entries or bytes); applying calls [`set_cache_size!`](@ref). + +Emptying and resizing act on RAM only. + +## Disk caches + The Disk tab lists the disk caches opened in this process, including functions without a RAM cache: ```@example dashboard @@ -83,23 +109,4 @@ Database sizes and entry counts are read in the background, at most every 10 sec `…` appears until they arrive. The header indicates when [`disable_disk_caches!`](@ref) has turned disk caching off. -Narrow terminals shorten names and drop columns. -The minimum size is 40×8. - -| Key | Action | -| :--- | :--- | -| `Tab`, `1` / `2` | switch to the next tab / the RAM or Disk tab | -| `↑` `↓`, `PgUp` `PgDn`, `Home` `End` | select a row | -| `Enter` | resize the selected cache, in RAM (see below) | -| `e` | empty the selected cache, in RAM | -| `s` / `r` | sort by the next column / reverse the order, per tab | -| `/` | filter by function name, on both tabs (`Enter` applies, `Esc` cancels) | -| `g` | refresh now | -| `q`, `Esc` | quit | - -In resize mode, `←` / `→` halve or double the limit, `[` / `]` adjust it by 10%, `Enter` applies, and `Esc` cancels. -The preview uses the cache's current measure (entries or bytes); applying calls [`set_cache_size!`](@ref). - -Emptying and resizing act on RAM only. Use [`empty_disk_caches!`](@ref) to clear persistent results outside the dashboard. -Statistics are copied under brief locks, so the dashboard can run while other tasks use the caches. From 7c6f99d8965ce09a625ec54be374d4ee5e8d447b Mon Sep 17 00:00:00 2001 From: lkdvos Date: Mon, 5 Oct 2026 14:46:28 -0400 Subject: [PATCH 8/9] docs: execute guide snippets and render real cache results --- docs/examples/package.jl | 34 +++++++++++ docs/src/configuration.md | 47 ++++++++++++--- docs/src/dashboard.md | 2 +- docs/src/disk.md | 117 ++++++++++++++++++++++++++++++-------- docs/src/eviction.md | 32 ++++++++++- docs/src/index.md | 51 +++++++++++++---- docs/src/timing.md | 22 ++++++- 7 files changed, 260 insertions(+), 45 deletions(-) create mode 100644 docs/examples/package.jl diff --git a/docs/examples/package.jl b/docs/examples/package.jl new file mode 100644 index 0000000..daadb84 --- /dev/null +++ b/docs/examples/package.jl @@ -0,0 +1,34 @@ +# A private package and preferences environment for executable documentation examples. +# Loading without precompilation lets it reuse the documentation process's dependencies. +let env = mktempdir(), uid = Base.UUID(rand(UInt128)) + src = joinpath(env, "MyPackage", "src") + mkpath(src) + write( + joinpath(dirname(src), "Project.toml"), + "name = \"MyPackage\"\nuuid = \"$uid\"\n[deps]\nCached = \"$(Base.PkgId(Cached).uuid)\"\n", + ) + write( + joinpath(src, "MyPackage.jl"), + "__precompile__(false)\nmodule MyPackage\nusing Cached\n@cached expensive(x) = x^2\nend\n", + ) + pushfirst!(LOAD_PATH, env) + package = try + Base.require(Base.PkgId(uid, "MyPackage")) + finally + popfirst!(LOAD_PATH) + end + write( + joinpath(env, "Project.toml"), + "[deps]\nCached = \"$(Base.PkgId(Cached).uuid)\"\nMyPackage = \"$uid\"\n", + ) + with_preferences = function (f) + original = Base.ACTIVE_PROJECT[] + Base.ACTIVE_PROJECT[] = joinpath(env, "Project.toml") + try + return Base.invokelatest(cd, f, env) + finally + Base.ACTIVE_PROJECT[] = original + end + end + (; package, with_preferences) +end diff --git a/docs/src/configuration.md b/docs/src/configuration.md index 51579b2..819925a 100644 --- a/docs/src/configuration.md +++ b/docs/src/configuration.md @@ -55,12 +55,34 @@ Unknown keys and invalid values are ignored with a warning. Use [`set_cache_preferences!`](@ref) to write the sections, which merges with what is already there: -```julia +```@example preferences using Cached -set_cache_preferences!(; maxsize = 50_000) # [Cached] -set_cache_preferences!(MyPackage; measure = "bytes") # [MyPackage.Cached] -set_cache_preferences!(MyPackage.expensive; maxsize = 1000) # [MyPackage.Cached.expensive] -set_cache_preferences!(MyPackage.expensive; maxsize = nothing) # remove a setting +sample = include(joinpath(pkgdir(Cached), "docs", "examples", "package.jl")) # hide +MyPackage = sample.package # hide +with_preferences = sample.with_preferences # hide +with_preferences() do # hide +set_cache_preferences!(; maxsize = 50_000) # [Cached] +println(read("LocalPreferences.toml", String)) +end # hide +``` + +For functions defined in your package, use package or function settings: + +```@example preferences +with_preferences() do # hide +set_cache_preferences!(MyPackage; measure = "bytes") # [MyPackage.Cached] +set_cache_preferences!(MyPackage.expensive; maxsize = 1000) # [MyPackage.Cached.expensive] +println(read("LocalPreferences.toml", String)) +end # hide +``` + +Remove a setting by passing `nothing`: + +```@example preferences +with_preferences() do # hide +set_cache_preferences!(MyPackage.expensive; maxsize = nothing) +println(read("LocalPreferences.toml", String)) +end # hide ``` ## Measuring sizes @@ -70,6 +92,17 @@ That traverses the whole value, which can be slow for large nested values, and c Keys and container overhead are excluded; this is a value budget, not a limit on total process memory. Overload it for your own types: -```julia -Cached.cachesize(t::MyTensor) = sizeof(t.data) +```jldoctest +using Cached + +struct MyTensor + data::Vector{Float64} +end +Cached.cachesize(t::MyTensor) = sizeof(t.data); + +Cached.cachesize(MyTensor(zeros(10))) + +# output + +80 ``` diff --git a/docs/src/dashboard.md b/docs/src/dashboard.md index 468b2a7..ee10afe 100644 --- a/docs/src/dashboard.md +++ b/docs/src/dashboard.md @@ -68,7 +68,7 @@ for step in 1:40 # hide foreach(i -> Main.kernel(1 + (5i + step) % 24), 1:(2 + step % 5)) # hide Ext.refresh!(m, Float64(step)) # hide end # hide -foreach(t -> filter!(r -> !occursin("doctest", r.label), t.rows), m.tabs) # functions of the doctests of other pages # hide +foreach(t -> filter!(r -> r.f in (Main.fib, Main.weights, Main.matrix, Main.label, Main.kernel), t.rows), m.tabs) # only this page's examples # hide Ext.rebuild!(m) # hide function preview(m, height) # hide tb = Tachikoma.TestBackend(100, height) # hide diff --git a/docs/src/disk.md b/docs/src/disk.md index 8cea261..e29bdcb 100644 --- a/docs/src/disk.md +++ b/docs/src/disk.md @@ -9,15 +9,31 @@ The disk is a second level below the cache in memory: a call looks in RAM first Disk caching is a package extension on [SQLite.jl](https://github.com/JuliaDatabases/SQLite.jl): load it with `using SQLite`, or, in a package, add SQLite to its dependencies and `import SQLite`. -```julia +```@example disk using Cached, SQLite +using Pkg.Artifacts # hide +diskenv = mktempdir() # hide +write(joinpath(diskenv, "Project.toml"), "[deps]\nCached = \"$(Base.PkgId(Cached).uuid)\"\n") # hide +write(joinpath(diskenv, "LocalPreferences.toml"), "[Cached]\ndisk_path = $(repr(mktempdir()))\n") # hide +push!(LOAD_PATH, diskenv) # hide +computations = Ref(0) @cached function expensive(n::Int)::Matrix{Float64} - sleep(1) # stand-in for an expensive computation + computations[] += 1 [Float64(i == j) for i in 1:n, j in 1:n] end Cached.DiskCacheStyle(::typeof(expensive), args...) = DiskCache() +expensive(2) +``` + +Clear RAM and call again to read the stored result from disk: + +```@example disk +empty_caches!(expensive) +expensive(2) +@assert computations[] == 1 # hide +computations[] # the body ran only once ``` ## Choosing what goes to disk @@ -37,8 +53,15 @@ Each function has one SQLite database per machine, `.-v- p.first === point, Cached.disk_cache_stats())).second.hits == 1 # hide +(p.x, p.y) ``` `writetag` and `OBJECT_TAG` are internals of `Serialization`, used the same way by `Distributed`'s `ClusterSerializer`. @@ -111,8 +149,11 @@ Cached.DiskCacheStyle(::typeof(expensive), args...) = DiskCache(; serializer = M [`disable_disk_caches!()`](@ref disable_disk_caches!) turns all disk caches off in this process, and [`enable_disk_caches!()`](@ref enable_disk_caches!) back on; while off, nothing is read from or written to disk. The `disk = false` preference turns them off per function, package, or globally: -```julia +```@example diskprefs +with_preferences() do # hide set_cache_preferences!(MyPackage; disk = false) +println(read("LocalPreferences.toml", String)) +end # hide ``` `disk` and `disk_path` are read when a function first uses its disk cache in a session. @@ -120,13 +161,21 @@ Disk caches are never used during precompilation. ## Managing disk caches -```julia -disk_cache_info(expensive) # [expensive => (; path, entries, bytes)] -disk_cache_info(MyPackage) # ... of its functions used in this process -empty_disk_caches!(expensive) # remove the entries -Cached.disk_cache_stats() # [expensive => (; hits, misses), ...] in this process +```@example disk +info = only(disk_cache_info(expensive)).second +(; entries = info.entries, bytes = info.bytes) +``` + +Empty the database and check the entry count: + +```@example disk +empty_disk_caches!(expensive) +only(disk_cache_info(expensive)).second.entries ``` +`disk_cache_info(MyPackage)` lists disk caches of functions owned by a module and its submodules. +[`Cached.disk_cache_stats`](@ref) reports hit/miss counters for this process. + [`disk_cache_info`](@ref) and [`empty_disk_caches!`](@ref) act on this machine's current database; files of older versions or other machines are left alone. The Disk tab of the [dashboard](dashboard.md) lists the open disk caches with these numbers. @@ -134,15 +183,37 @@ The Disk tab of the [dashboard](dashboard.md) lists the open disk caches with th A package can ship precomputed results, read-only, as a [Pkg artifact](https://pkgdocs.julialang.org/v1/artifacts/): fill the disk cache, export it with [`export_disk_cache`](@ref), and point [`Cached.disk_artifact`](@ref) at the artifact. -```julia +```@example disk using Pkg.Artifacts -hash = create_artifact() do dir - foreach(expensive, 1:100) # fill the disk cache - export_disk_cache(expensive, dir) # writes .expensive-v.sqlite +artifact_hash = create_artifact() do dir + empty_caches!(expensive) # make every call reach disk + foreach(expensive, 1:3) # fill the disk cache + export_disk_cache(expensive, dir) end -# archive, upload and bind it in Artifacts.toml as usual; then, in the package: -Cached.disk_artifact(::typeof(expensive)) = artifact"expensive" +readdir(artifact_path(artifact_hash)) # the exported database ``` +Point the function at the exported artifact, then empty RAM and the node database: + +```@example disk +Cached.disk_artifact(::typeof(expensive)) = artifact_path(artifact_hash) +try # hide +empty_caches!(expensive) +empty_disk_caches!(expensive) +Base.get_extension(Cached, :CachedSQLiteExt).close_all() # reopen with the artifact # hide +before = computations[] +result = expensive(2) +@assert computations[] == before # hide +println((; result, new_computations = computations[] - before)) +finally # hide +Base.get_extension(Cached, :CachedSQLiteExt).close_all() # hide +filter!(!=(diskenv), LOAD_PATH) # hide +end # hide +nothing # hide +``` + +To ship the artifact, archive, upload, and bind it in your package's `Artifacts.toml`. +Its package definition can then return `artifact"expensive"` from `Cached.disk_artifact`. + A call then looks in RAM, the artifact, and this machine's database, in that order, and only then computes. New results go to the database; the artifact is never written, and holds the results of one version of `f`. diff --git a/docs/src/eviction.md b/docs/src/eviction.md index e33a39b..6308b3f 100644 --- a/docs/src/eviction.md +++ b/docs/src/eviction.md @@ -18,6 +18,18 @@ It removes the first entry it finds with a clear bit and continues from that pos New entries start with a clear bit: they get a second chance only after a cache hit. For example, fill a three-entry cache with A, B, and C, then hit A. Inserting D starts the hand at A, clears A's bit, and evicts B, whose bit is clear. +```@example clockcache +using Cached + +cache = ClockCache{Symbol, Int}(; maxsize = 3) +for (key, value) in zip((:A, :B, :C), 1:3) + cache[key] = value +end +cache[:A] # a hit +cache[:D] = 4 +sort!(collect(keys(cache))) # B was evicted +``` + Unlike LRU, Clock does not track the exact order of hits. A recently inserted entry that has never been hit again can be evicted before an older entry that has been reused. @@ -32,6 +44,18 @@ When space is needed, the least recently used entry is removed. For example, fill a three-entry cache with A, B, and C, then hit A. The order from least to most recently used becomes B, C, A, so inserting D evicts B. +```@example lru +using Cached + +cache = LRU{Symbol, Int}(; maxsize = 3) +for (key, value) in zip((:A, :B, :C), 1:3) + cache[key] = value +end +cache[:A] # a hit +cache[:D] = 4 +sort!(collect(keys(cache))) # B was evicted +``` + Unlike Clock, LRU always retains the exact recency order, even after several hits between evictions. Choose LRU when that ordering suits the workload. @@ -40,9 +64,15 @@ Neither policy guarantees a better hit rate for every workload; use the [dashboa ## Selecting a policy -```julia +```@example policy +using Cached +@cached expensive(x) = x^2 +@cached other(x) = 2x + Cached.CacheStyle(::typeof(expensive), args...) = GlobalCache{LRU}() Cached.CacheStyle(::typeof(other), args...) = TaskLocalCache{ClockCache}() + +(CacheStyle(expensive, 3), CacheStyle(other, 3)) ``` To set the default container for `GlobalCache()` and `TaskLocalCache()`, use the `container` preference in [Configuration](configuration.md). diff --git a/docs/src/index.md b/docs/src/index.md index ac23aef..4f77502 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -117,8 +117,17 @@ fib(10) To skip caching floating-point calls to `powers`: -```julia -Cached.CacheStyle(::typeof(powers), ::AbstractFloat, ::Int) = NoCache() +```jldoctest powers +Cached.CacheStyle(::typeof(powers), ::AbstractFloat, ::Int) = NoCache(); + +powers(2.0; offset = 1.0) + +# output + +3-element Vector{Float64}: + 3.0 + 5.0 + 9.0 ``` `GlobalCache()` and `TaskLocalCache()` use the [configured container](configuration.md), Clock by default. @@ -138,16 +147,38 @@ Global caches are created on first use. Normally, one cache holds all cached methods and argument types of a function. If its strategy selects several container types, each gets a separate cache with the function's size limit. -```julia -cache_info(powers) # live f => cache pairs, with size and statistics -cache_info(MyPackage) # functions owned by this module and submodules -cache_info() # all global caches -set_cache_size!(powers, 1000) # count entries -set_cache_size!(powers, 2^20; by = Cached.cachesize) # measure values in bytes -empty_caches!(powers) # clear entries, keep hit/miss counters -empty_caches!(MyPackage) # clear the package's global caches +```@example management +using Cached + +module MyPackage + using Cached + @cached powers(x) = [x^i for i in 1:3] +end + +MyPackage.powers(2) +cache_info(MyPackage) # functions owned by this module and submodules ``` +Resize a function's cache and inspect it after inserting three keys: + +```@example management +set_cache_size!(MyPackage.powers, 2) # count entries +foreach(MyPackage.powers, 1:3) +cache_info(MyPackage.powers) +``` + +Switch to a byte limit, or clear a module's caches: + +```@example management +set_cache_size!(MyPackage.powers, 2^20; by = Cached.cachesize) +MyPackage.powers(2) +empty_caches!(MyPackage) # keep hit/miss counters +cache_info(MyPackage) +``` + +`cache_info()` and `empty_caches!()` act on all global caches. + + Shrinking a cache evicts entries; changing its size measure discards it. A value larger than the limit is returned but not retained by the built-in containers. Byte limits measure values, not keys or container overhead, and are not a process-wide memory budget. diff --git a/docs/src/timing.md b/docs/src/timing.md index a2f663f..060a385 100644 --- a/docs/src/timing.md +++ b/docs/src/timing.md @@ -36,15 +36,31 @@ What counts is the package that owns the function (including its submodules), no For instance, if `MyPackage` adds `@cached` methods to a function `OtherPackage.f`, then `enable_cache_timers!(OtherPackage)` times them, and `enable_cache_timers!(MyPackage)` does not. A package that wants all of its cached functions timed enables the packages that own them: -```julia +```@example owners +using Cached, TimerOutputs # hide +module MyPackage # hide + function enable_timers! end # hide +end # hide +module OtherPackage # hide + using Cached # hide + @cached f(x) = x^2 # hide +end # hide +TIMER = TimerOutput() # hide function MyPackage.enable_timers!() enable_cache_timers!(MyPackage, TIMER) enable_cache_timers!(OtherPackage, TIMER) end + +MyPackage.enable_timers!() +OtherPackage.f(2) +disable_cache_timers!(MyPackage) +disable_cache_timers!(OtherPackage) +TIMER ``` The labels come from [`Cached.instrument_label`](@ref), `"lookup f"` and `"compute f"` by default; overload it to pick your own: -```julia -Cached.instrument_label(::typeof(expensive), ::Val{:lookup}) = "cache: expensive" +```@example timer +Cached.instrument_label(::typeof(weights), ::Val{:lookup}) = "cache: weights" +Cached.instrument_label(weights, Val(:lookup)) ``` From f8e8fca4d05d4487ab6b88e288a74f03872f248d Mon Sep 17 00:00:00 2001 From: lkdvos Date: Mon, 5 Oct 2026 14:52:38 -0400 Subject: [PATCH 9/9] docs: render home examples without doctest output markers --- docs/src/index.md | 39 +++++---------------------------------- 1 file changed, 5 insertions(+), 34 deletions(-) diff --git a/docs/src/index.md b/docs/src/index.md index 4f77502..acac4ea 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -25,7 +25,7 @@ using Pkg Pkg.add("Cached") ``` -```jldoctest +```@example fib using Cached @cached function fib(n::Int)::BigInt @@ -33,10 +33,6 @@ using Cached end; fib(100) - -# output - -354224848179261915075 ``` Calling `fib(100)` again reuses the stored result. @@ -49,7 +45,7 @@ By default, each function gets a shared Clock cache with room for 10,000 entries Put [`@cached`](@ref) before a method definition. By default, its positional and keyword arguments form the cache key, and its body runs only on a miss. -```jldoctest powers +```@example powers using Cached @cached function powers(x::T, n::Int = 3; offset::T = zero(T))::Vector{T} where {T} @@ -57,13 +53,6 @@ using Cached end; powers(2; offset = 1) - -# output - -3-element Vector{Int64}: - 3 - 5 - 9 ``` The macro also supports varargs, qualified names, operators, and callable objects. @@ -77,15 +66,8 @@ A type-stable body keeps calls inferred even though the shared cache holds resul Use [`uncached`](@ref) to call the body without reading or filling RAM or disk caches. Supply all arguments, including defaults: defaults belong to the wrapper, not the uncached body. -```jldoctest powers +```@example powers uncached(powers, 2, 3; offset = 1) - -# output - -3-element Vector{Int64}: - 3 - 5 - 9 ``` ### Choosing a strategy @@ -102,32 +84,21 @@ A strategy based on types can be resolved by the compiler. For example, give each task its own cache for a recursive computation: -```jldoctest +```@example tasklocal using Cached @cached fib(n::Int)::BigInt = n < 2 ? BigInt(n) : fib(n - 1) + fib(n - 2); Cached.CacheStyle(::typeof(fib), ::Int) = TaskLocalCache{LRU}(); fib(10) - -# output - -55 ``` To skip caching floating-point calls to `powers`: -```jldoctest powers +```@example powers Cached.CacheStyle(::typeof(powers), ::AbstractFloat, ::Int) = NoCache(); powers(2.0; offset = 1.0) - -# output - -3-element Vector{Float64}: - 3.0 - 5.0 - 9.0 ``` `GlobalCache()` and `TaskLocalCache()` use the [configured container](configuration.md), Clock by default.