Repository navigation
Update dependency mr-boxington to v1.19.0 - #114
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
1.15.0→1.19.0v1.22.0(+3)Release Notes
jdx/mr-boxington (mr-boxington)
v1.19.0: : mbx analyze, low-disk collection, and caching for CMake, build-std, and Linux cdylibsCompare Source
The new
mbx analyzecommand explains a slow build: it ranks the build's uncached compiler time by cause and shows the critical path. Collection now responds when free disk space runs low, and you can choose which checkouts' targets are kept and which are removed first. Standard library units from-Zbuild-std, Linuxcdylibcrates (including Node-API addons), and C/C++ compiles undermbx exec cmakeare now cached instead of bypassed.Added
mbx analyzeranks a build's uncached compiler time by cause (#569, #568, @jdx). The command reads the current workspace's last recorded build. It runs nothing. Wherembx explain --lastgives each crate's miss separately,mbx analyzegroups the misses by cause, sorts the causes by cost, and says what would remove each one. If a crate's own key didn't change but it read a changed artifact, its time is charged to the crate where the change started, through any number of levels. One edit therefore shows up as one cause, not dozens:The causes include changed inputs, changed compiler flags (counted per flag), changed environment variables, a new toolchain or linker, a first build, and each bypass reason. Bypassed compilations now record their crate and compiler time, so
mbx tuino longer shows them as unnamed0srows. A unit is now compared only with earlier units that emitted the same outputs. This also stopsmbx explain --lastfrom blaming--emitafter you switch betweencargo checkandcargo build. A new "Analyzing a build" docs page covers the causes and limits.mbx analyzeshows the critical path (#570, @jdx). The report ends with the chain of units the build waited on, build-script runs included, and how long only one unit was running. This shows which crate orbuild.rsis holding the build back. It works with both Cargo build-directory layouts (up to 1.99 and 1.100+). To build this graph, mbx now records timing for more work:wrapper_phases_nsinMBX_STATS_REPORTnow counts those compilations undercompiler.MBX_SCHEDULER=0, the shim now waits for a bypassed compilation instead of exec-ing rustc in its place.Collection keeps a minimum of free disk space (#574, @jdx). Before, budgets were shares of total disk size and sweeps ran at most once an hour, so a disk filled by other programs, or by several agents building at once, stayed full. While a disk has less free space than
gc.min_free_size:gc.max_size, and the most recently used target and anything a running build holds are kept.The cache disk and a separate
target.rootvolume are each checked on their own.mbx gcapplies the same rule, andmbx gc --dry-runshows the most a low disk could remove.Choose which checkouts' targets to keep or remove first (#575, @jdx). Both lists are empty by default.
target.max_size, and they are still removed once their checkout is deleted.~entries cover everything at or under that directory. A relative entry matches wherever its path components appear, so.claude/worktreescovers agent worktrees in every repository.MBX_TARGET_KEEPandMBX_TARGET_EVICT_FIRST, take comma-separated lists.mbx exec cmakecaches C and C++ through CMake compiler launchers (#564, @jdx). On a configure,mbx exec cmakenow setsCMAKE_C_COMPILER_LAUNCHERandCMAKE_CXX_COMPILER_LAUNCHER. This caches two setups that were missed before: a build directory first configured without mbx, and a compiler named by full path (-DCMAKE_C_COMPILER=/usr/bin/clang,CC=gcc-13, or a toolchain file). The compiler recorded for the directory doesn't change, so there's no need to start over in a fresh build directory:mbx execiscmakeitself running a configure, not when a script calls CMake.-Dor in the environment, is left alone.mbx exec cmakekeeps its cache session even when no unversionedccorgccis onPATH.Host
cdyliblinks are cached on Linux (#579, #580, @jdx). C-ABI libraries and Node-API addons were bypassed asunsupported-crate-typeand recompiled on every build. They now use the same native-link cache as binaries and tests. Native Linux links also accept-C link-arg=-Wl,-z,KEYWORDfordefs,lazy,nodelete,nodlopen,noexecstack,norelro,now,origin, andrelro, so napi-build addons (which pass-z,nodelete) are now cached too. In a 15-crate workspace with twocdylibcrates, a warm build with a fresh target dropped from 3.8 s to 2.4 s. Some links still bypass:cdylibscdylibs on macOS and Windows-zkeywords, combined-Wlspellings, and-C link-argsMBX_CACHE_LINKS=0turns this off along with other native-link caching.Fixed
-Zbuild-stdstandard library units are cached (#566, @jdx). mbx now accepts the-Zforce-unstable-if-unmarkedflag, socore,alloc,std, and the other standard library units are no longer bypassed asunknown-flag.RUSTC_BOOTSTRAPis now part of the cache key when it is set. Builds that set it will get one round of cache misses after upgrading. Keys for builds that don't set it are unchanged.mbx setup --statusandmbx doctorcatch a broken Cargo shim on Unix (#576, @jdx). The shim was reported as current even when the mbx executable it points to had been removed, for example by pruning a mise install. It is now reported as outdated, and runningmbx setuprepairs it.mbx setupexplains rust-analyzer'scheck/overrideCommand: unexpected fieldwarning (#573, @jdx). Editors show this message after setup, but checks still run through mbx.mbx setupandmbx setup --statusnow say so.mbx testview works on xterm terminals (#559, @jdx). WithTERM=xterm-256color, colored test result lines were printed raw and theTests (x/y)progress didn't advance. Both work now.Changed
backend: remoteandremote-urlinputs, available in mr-boxington-action v1.6.0 and later (#557, @jdx).Full Changelog: jdx/mr-boxington@v1.18.0...v1.19.0
v1.18.0: : New checkouts start from another checkout's dependency units, and live target directories are prunedCompare Source
Managed targets do more on their own in this release. With Cargo 1.100 or later, a new checkout's first build copies registry and Git dependency units that another checkout already built. Unused units are now removed from target directories that are still in use. An existing
target/is moved under the managed root without a prompt. mbx also supports Cargo 1.100's per-unit build layout and the Rust 1.99 toolchain, and a crate that readsOUT_DIRno longer recompiles on the build after it compiles.Added
A new checkout's first build starts from another checkout's registry units (#550, @jdx). Before, a fresh worktree asked the cache for every unit, one shim call at a time. That added up to seconds across a real dependency graph, and build scripts whose C output is cached only per path (aws-lc-sys, for example) compiled again in every checkout. With Cargo 1.100 or later, the first build of a profile now copies registry and Git dependency units from another checkout's managed target. Cargo treats the copies as fresh, so neither rustc nor the shim runs for them:
On a fresh clone of this repository, the first build dropped from about 13 s to about 1 s. The following limits apply:
On by default. Turn it off with
MBX_TARGET_SEED=0or:Unused build units are removed from live managed target directories (#549, @jdx). Before, a managed target was collected only when its checkout was deleted, when it aged out, or when the size budget needed space. A checkout in daily use kept every unit it had ever built, across every
Cargo.lock, feature, and toolchain change. Collection now also removes units that no build has used fortarget.max_age(30 days by default) plus one day. To decide which units are unused, mbx reads the access times of Cargo's fingerprint files.deps/,.fingerprint/, andbuild/layout is removed as a whole once no build has used any of it. This frees a checkout's old outputs after it moves to Cargo 1.100.noatimemounts, and often NTFS), and for targets that a build is using.target.max_age = "none"keeps all units.mbx gcreports the removal on its own line (removed <count> unused build units from live target directories).mbx gc --jsonaddstargets.removed_unitsandtargets.removed_unit_bytes.Changed
An existing
target/is adopted on the first build without a prompt (#545, @jdx). Before, a checkout that already had a realtarget/stayed unmanaged unless you ranmbx adoptor accepted a prompt in a terminal. Builds run by agents, scripts, and editors never saw the prompt, so their directories were never collected. Now any mbx build outside CI moves the existingtarget/under the managed root when a same-filesystem rename is possible, and keeps every output:target/keep working.--target-dir,CARGO_TARGET_DIR, orbuild.target-dirare never moved.target/that plain Cargo built may recompile some crates.To keep every
target/where Cargo puts it, settarget.views = false(orMBX_TARGET_VIEWS=0).Cache hashing and prefetch planning are faster (#538, #539, #541, #542, #543, #544, #546, #547, @jdx). C/C++ and assembler inputs are hashed faster: scanning for timestamp macros takes up to 79% less time on large headers. Prefetch ranking skips parsing full prediction payloads and was 6-7 times faster in synthetic benchmarks. These are component measurements, not whole-build speedups.
The Get started guide now includes a prompt you can give a coding agent to set up mbx (#540). The comparison page was rewritten to help new users choose between mbx, kache, sccache, CI caches, and incremental compilation (#537).
Fixed
out/, so their cache keys work across checkouts.debugprofile andcargo install:debugprofile (used bycargo install --debug) uses[linker.profiles.dev]unless[linker.profiles.debug]exists.cargo installwithout--debugnow uses[linker.profiles.release]. Before, it looked updev.OUT_DIR(for example throughinclude!(concat!(env!("OUT_DIR"), ...))) no longer recompiles on the next no-change build, along with every crate that depends on it (#551, @jdx). The copies mbx makes of build-script output now keep the original modification times, so Cargo no longer treats them as newer than the unit.MBX_DISPLAY=plain) no longer prints the seven-line mascot before every build (#533, @jdx). Thembx[progress]lines and thembx[cache]summary are unchanged. The animated terminal view still shows the mascot.Full Changelog: jdx/mr-boxington@v1.17.0...v1.18.0
v1.17.0: : Memory-pressure-aware compile scheduling and build-script replay for custombuildpathsCompare Source
The machine-wide scheduler now watches live memory pressure and stops admitting new compilers while the machine is short on memory, with an opt-in Linux mode that can freeze running compiler trees in delegated cgroups. Build scripts declared with a custom
build = "..."path (aws-lc-sys, for example) are now replayed from the cache instead of rerunning on every build, and crates that merely look like build scripts are no longer mistaken for them.Added
Admission control reacts to live memory pressure (#520, #521, @jdx). Until now the pool admitted compilers based on permits and recorded memory estimates, so several concurrent builds could still push a machine into swap when cold compilations had no history. mbx now samples pressure at most every 500 ms and, after two unhealthy samples, defers new admissions, including compilations with no memory estimate, until five healthy seconds pass; admissions then ramp back up over five seconds. Linux uses available-memory headroom plus full-memory PSI stalls from the host and visible cgroup v2 ancestors; macOS uses headroom only. Running compilers are never paused by this setting, an idle pool always lets one compilation start, and probe failures fall back to the previous permit and estimate checks. On by default; opt out with:
or
pressure = falseunder[scheduler].scheduler.memory = "none"orMBX_SCHEDULER=0also disables it.Experimental: suspend compiler trees under pressure in delegated Linux cgroups (#523, #525, @jdx). When admission control alone is not enough, opted-in Linux builds place each eligible
rustc, GCC, or Clang invocation and its descendants (including linkers) in its own cgroup. After pressure persists for two seconds mbx freezes the newest eligible compiler, at most one per second, while the oldest keeps running; on recovery, frozen work resumes oldest first before anything new is admitted. Frozen compilers keep their permits. Off by default and only settable from the environment or the global config, never from a repository.mbx.toml:(
MBX_SCHEDULER_SUSPEND=1,MBX_SCHEDULER_CGROUP_ROOT=...).cgroup_rootmust be an absolute, writable, delegated cgroup v2 directory; pressure control and memory scheduling must be enabled. mbx does not provision delegation or change host memory limits, and if delegation is unavailable or the platform is unsupported it warns once per session and continues with admission-only scheduling. Freezing retains allocated memory, so it cannot rescue a compilation that is too large to run alone, and mbx never kills and retries a compilation. Custom compiler wrappers, build-script binaries, test binaries, and compilers nested inside a supervised compiler are excluded. A supervisor and an independent watchdog run outside the compiler cgroups and thaw everything if either dies or hangs. Suspension events and per-action statistics are written underscheduler/supervision-*/in the cache directory, and compiler completion reports time spent suspended.Fixed
Build scripts with a custom
buildpath are now replayed from the cache (#526, @jdx). mbx only recognized scripts compiled frombuild.rs(cratebuild_script_build), so a package such as aws-lc-sys withbuild = "builder/main.rs"ran its build script in full on every build even on a cache hit, recompiling and archiving its C sources for about 3.5 seconds on the critical path of everything depending on it (reqwest, rustls, and other aws-lc-rs users). Any Cargo build script is now replayed regardless of its source file name. On the jdx/hk benchmark, a warm store with a fresh target went from 4.58 s to 1.02 s. Builds into a different target path than the one that filled the cache still rerun aws-lc-sys's script, because its C objects are cached per target path, as before.Crates that only look like build scripts are no longer treated as one (#528, @jdx). A
[[bin]]target namedbuild-script-buildhad its executable replaced with the build-script launcher, so runningtarget/debug/build-script-buildfailed withexec: .../.mbx-build-script-shims/.../mbx: not found. With native-link caching off (MBX_CACHE_LINKS=0), libraries named like build scripts (for examplebuild-script-helper) went through the execution-only path and recompiled every build. mbx now recognizes a build script only when Cargo is building one: abuild_script_*crate whose only--crate-typeisbinand for whichCARGO_BIN_NAMEis unset. The check works under both Cargo's current target layout and the build-dir layout Cargo 1.100 makes the default, and it reads arguments after expanding@argfileresponse files, so build scripts whose arguments arrive that way are cached too.For library users,
mbx-cache-rustc0.18.1 carries the same change:RustcOutputs::build_script_executablenow keys off the build-script crate name alone rather than thebuild/output directory, and callers are expected to confirm with Cargo's environment asmbxdoes.Full Changelog: jdx/mr-boxington@v1.16.0...v1.17.0
v1.16.0: : Hard-linked restores on ext4, opt-in cargo test scheduling, and fixes for multiple mbx installationsCompare Source
Cache hits on filesystems that cannot clone files (ext4 above all) now hard link the stored object instead of copying every byte, native
.aarchives produced by build scripts no longer change digest on every build because of anartimestamp, andcargo testbinaries can optionally take permits from the machine-wide scheduler. Several fixes cover machines that run more than one mbx installation, including CI runners where per-job installs share one cache directory.Added
cargo testbinaries can run under the machine-wide permit pool (#513, @jdx). mbx already keeps simultaneous builds from oversubscribing the machine, but only for compiler processes; severalcargo testruns reaching their test phase together (for example, coding agents in separate worktrees) each still started a thread per CPU. With the new opt-inscheduler.testssetting, mbx installs itself as Cargo's target runner and each test binary waits for permits before it starts, then runs through whatever runner was configured before (qemu, a wrapper script). Off by default; enable it in.mbx.tomlor your global config, or per run:MBX_SCHEDULER_TESTS=1 mbx test --workspaceA binary's first run asks for
--test-threads=N/RUST_TEST_THREADS=Npermits if stated, otherwise half the pool; after a complete run of at least a second, mbx records its measured average core usage (Unix only) and later runs ask for that many. Nested builds a test starts (trybuild, compile-fail suites) run with scheduling off and count against the test's permits.--no-run,--list, doctests, and commands with--config,-C,--directory, or+toolchainare not wrapped. Also in this change: runner environment keys now turn.into_as Cargo does, which affectscargo runwith custom target names containing a dot.Private compiler shim directories for containers sharing a cache (#501, @jbellis). When several containers share
MBX_CACHE_DIRbut each has its own mbx binary, one container could repoint the shared compiler symlinks at a binary that exists only in its own filesystem, and the others failed with exit 127. The newshims_dirsetting (MBX_SHIMS_DIR) puts the persistent Rust, C/C++, cross-compiler, CMake, andmbx execshims in a private directory while build artifacts stay shared:The default remains
<cache_dir>/shims. Absolute paths are used as given; relative paths resolve beneathcache_dirand are rejected if they traverse above it or normalize to an empty path. The directory must persist across builds and contain no real compilers. Existing CMake trees that recorded an old shim path need reconfiguring (for examplembx exec cmake --fresh -S . -B build). This is a global or environment setting, not a workspace policy.Changed
Restores hard link cached outputs where the filesystem cannot clone them (#511, #514, @jdx). Restores previously tried a reflink and fell back to a byte copy, so on ext4 (most Linux CI runners) every restored byte was written: a warm build of
jdx/hkcopied 1.32 GB across 1565 files. The order is now reflink, then a hard link to the stored object, then a copy. Filesystems with clone support (APFS, Btrfs, XFS with reflink, ZFS) are unaffected. A hard-linked output is the store's object, so it is read-only; mbx unlinks it before a compiler rewrites it, so rebuilding through mbx works as before, but running plaincargodirectly in a target directory mbx filled can reportoutput file ... is not writeablefor a unit it decides to rebuild. Setrestore_hardlink = false(orMBX_RESTORE_HARDLINK=0) to keep every restored output a private writable copy.mbx doctorreplaces itsreflinkcheck with arestorecheck that reports whether a given cache and target directory pair clones, hard links, or copies, and the session summary andMBX_STATS_REPORTgainhardlinked_output_filesandhardlinked_output_bytesnext to the reflinked and copied counters.Native archive timestamps no longer miss every downstream cached action (#506, @jdx). Apple's
arandranlibstamp the current time into an archive, so a build script producing a.a(CMake-based-syscrates in particular, which call/usr/bin/ardirectly) handed Cargo a different digest on every build even when nothing changed; one surveyed cache heldlibz-ng.aunder 7 digests for 7 byte-identical member sets. mbx now setsZERO_AR_DATEfor the build scripts it runs, controlled by the newar_determinismsetting (MBX_AR_DETERMINISM):auto(default) normalizes every profile exceptrelease,alwayscoversreleasetoo, andoffleaves the toolchain alone. AZERO_AR_DATEyou set yourself always wins. The effective value is part of the build-script action key and the action version was bumped, so existing build-script entries are recomputed once after upgrading.mbx doctorgains anarchivescheck that probes the local tools and reports what the configured policy does about them.Fixed
Nested Cargo builds inside tests write to their own target directory (#502, @stewartadam). mbx used to inject
CARGO_TARGET_DIRfor its managed target placement, and tests inherited it, so a test that rancargo buildon a separate guest crate found the guest's artifacts in the parent project'starget/instead ofguest/target/. Placement is now passed to Cargo as an invocation-local--config build.target-dir=..., so tests and build scripts see only theCARGO_TARGET_DIRthe caller set, if any, and their nested builds resolve their own target directories.Multiple mbx installations no longer break each other's C builds (#516, #504, #517, @jdx). Three related defects, all reachable with two mbx binaries on one machine:
HOST_CC/HOST_CXXat one shared<cache>/shims/mbx-csymlink, repointed by whichever session started last; when that job's install directory was cleaned up, every other running build failed withfailed to find tool "/var/cache/mbx/shims/mbx-c": No such file or directory(discussion #515). Session C, C++, and CMake shims now live under<shims_dir>/native/<install id>/, keyed by the mbx binary's path, so each installation owns its own. An in-place upgrade on Unix keeps the same paths; a different install path gets new ones, so cc-rs build scripts re-run once and existing CMake trees inOUT_DIRare moved to the new launcher automatically.ccshim as "the real compiler" and hand a compilation back and forth until the machine ran out of processes, with no object file and no diagnostic. mbx now writes a.mbx-shimsmarker into every directory it installs shims in and skips marked directories when looking for a real compiler, including pins recorded by an older mbx. Do not pointshims_dirat a directory that also holds real compilers, since mbx will then skip it.rust/<id>RUSTC_WRAPPERdirectory that was never collected (one machine had 206). Sessions now hold a lease on their wrapper directory, and on Unix each session start removesnative/andrust/directories that no session has used for 7 days and no running session leases, or whose links all dangle. Directories left by versions before this release are removed only once their links dangle; delete older piles by hand. Windows shims are never collected.Automatic cache cleanup works when mbx is installed as a hard link named
cargo(#505, @jbellis). The detached collector launched that executable withgc --automatic, re-entered Cargo-shim mode, and failed witherror: no such command: gc, leaving an over-budget cache uncollected.Target directory collection no longer removes a view a build just claimed (#512, @jdx). The two-second grace window that protects a checkout whose build started during a sweep was measured from the wall clock at each check rather than from the sweep's start, so a slow sweep narrowed its own window. It now uses one timestamp for the whole sweep; a slow sweep can only widen the window.
Library changes
mbx-cache-core0.18.0 addshardlinked_output_filesandhardlinked_output_bytestoRestoreStatsandAgentStats(#514). This is a semver break for embedders that construct those structs with literals. The shim-agentAGENT_PROTOCOL_VERSIONmoves to 10; since the shim and agent already require an exact version match, nothing that interoperated before stops doing so. Thembx-cache-protocolremote-cache wire contract is unchanged.New Contributors
Full Changelog: jdx/mr-boxington@v1.15.0...v1.16.0
Configuration
📅 Schedule: (in timezone Asia/Tokyo)
* * * * 1)🚦 Automerge: Enabled.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.