Skip to content

ci: consolidate 20 per-distro workflows into one matrix, add a blocking Resolute gate - #38

Open
nbbrooks wants to merge 3 commits into
mainfrom
nbbrooks/consolidate-ci-matrix
Open

ci: consolidate 20 per-distro workflows into one matrix, add a blocking Resolute gate#38
nbbrooks wants to merge 3 commits into
mainfrom
nbbrooks/consolidate-ci-matrix

Conversation

@nbbrooks

@nbbrooks nbbrooks commented Aug 3, 2026

Copy link
Copy Markdown
Member

Why

Every build workflow in this repo has been disabled_inactivity since ~2026-03.

The split is exact — no exceptions in either direction:

count state
workflows with schedule: 20 all disabled_inactivity
workflows without schedule: 11 all active

GitHub auto-disables cron-triggered workflows after 60 days of repo inactivity, and it disables the entire workflow, not just the cron trigger. So pull_request and push stopped firing too. This repo has had zero build coverage on any distro for months.

That is how it ended up un-released from rolling by ros/rosdistro#51542 without a red job anywhere to warn us. Simply re-enabling the workflows puts us back here in another 60 days.

How

20 per-distro build workflows + 2 reusable workflows + the coverage workflow → one build_and_test.yaml, with no schedule:.

The three-tier lookahead ladder is preserved. The binary / semi-binary / source split exists to catch incoming upstream breakage, and that intent is kept — as one matrix instead of 20 files.

Tier 1 — binary, main apt: can a user build this today?

The only jobs that gate merges.

Job Base OS
jazzy-main noble
kilted-main + ccov noble
lyrical-main resolute

lyrical is a blocking Resolute gate. It is released, so its main apt is populated, and I verified every dependency this repo has is there — ros2_control 6.8.0, realtime_tools 5.2.0. This is the rviz_visual_tools#301 call; moveit_visual_tools had to keep lyrical non-blocking only because moveit_core has no Resolute deb, which does not apply to us.

This is the job that would have caught the break that got us un-released.

Tier 2 — binary, testing apt: what breaks at the next sync?

jazzy-testing, kilted-testing, lyrical-testing, rolling-testing. All non-blocking.

Rolling appears only at this tier — Rolling's main apt has no Resolute packages yet.

Non-blocking is deliberate: an upstream regression staged for release is something we want to see, not something that should block an unrelated PR. This repo has already been through one "7 PRs blocked on an ecosystem break" episode.

Tier 3 — semi-binary, immediate deps from source: what breaks in the mid future?

rolling-testing + upstream-source. Non-blocking. Core ROS from debs, but ros2_control and realtime_tools built from their development branches.

⚠️ This tier has never actually worked in this repo. picknik_controllers.rolling.repos was an entirely commented-out placeholder, so the semi-binary job was byte-identical to the binary job — the middle rung of the ladder reported green while testing nothing. It is now populated with the two immediate non-core dependencies.

That makes it the tier that would have caught LoanedCommandInterface::get_value() being removed, weeks ahead of it surfacing as a buildfarm release failure.

What was dropped

The source-build tier (core ROS itself from source, via ros-tooling/action-ros-ci). It was permanently red for a reason unrelated to its value: it fetched its .repos with the deprecated ?token= URL syntax and got an HTTP 404 every run. Rebuilding all of core ROS to test two controllers is also poor value when tier 3 covers the dependencies that can realistically break us. Happy to restore it if you'd rather — it needs the token fix either way. (Same ?token= bug is still live in ros2_robotiq_gripper.)

Also in here

  • Coverage moved into the matrix on kilted. ci-coverage-build.yml ran action-ros-ci on a noble runner outside a container and has been broken since the Resolute transition — its own comment proposed exactly this fix. It also only built picknik_reset_fault_controller, so picknik_twist_controller has never had coverage. Attached to kilted rather than rolling so coverage doesn't stop reporting every time rolling breaks.

  • ROS Lint pinned to lyrical — the newest distro whose build job is blocking — and moved into a ros:lyrical-ros-base container, since GitHub has no 26.04 runner and setup-ros cannot install lyrical on noble (the same mismatch that broke coverage). It now lints the whole repo: the old config named picknik_controllers, which is not a package in this repo, and never linted picknik_twist_controller.

  • ABI check jobs given distinct names. All three used the bare job id abi_check, so all three reported as a check called abi_check. Branch protection matches on the name string, so there was no way to require jazzy's and kilted's while leaving rolling's optional. Now jazzy-abi / kilted-abi / rolling-abi.

  • CONTRIBUTING.md added. ament_copyright requires one at the repo root and validates its contents against its license templates; this is its apache2 template verbatim, matching our LICENSE.

  • humble build + ABI workflows removed from main only. The humble branch carries its own copies, and these were dead weight here — they only trigger on humble-base PRs. humble's hardware_interface has no get_optional(), so main genuinely cannot build there today.

On branches

This keeps the current main = jazzy/kilted/lyrical/rolling split but is shaped so it doesn't entrench it: once main carries source-level distro guards per moveit2#3751, humble comes back as one more line in the matrix and the humble branch retires. No new workflow files.

Verification

  • pre-commit run --all-files --hook-stage manual — clean
  • All three ament linters inside ros:lyrical-ros-base — pass, 8 files checked
  • Full check run green on the first push (before tiers 2–3 were added): jazzy-main, kilted-main + ccov, lyrical-main, rolling-testing, all three linters, Format, all three ABI checks

Conflicts / follow-ups

  • Conflicts with Add per-package build farm status table #25, which deletes .github/workflows/README.md and moves that content into the top-level README. This PR rewrites that file instead. Suggest Add per-package build farm status table #25 rebases to keep its buildfarm status tables in the top-level README and link here for the CI explanation — happy to take it either way.
  • prerelease-check.yml is unusable as written: ros_distro choices are humble/jazzy/rolling (no kilted, no lyrical) and the only branch choice is master, which doesn't exist here. Worth fixing before the next tag, since PRERELEASE: true is the closest thing to a buildfarm dry-run.
  • Issue picknik_reset_fault_controller appears to use hardware_interface directly without declaring it in package.xml #36 (hardware_interface used directly but undeclared) is real on main for both packages and should land before the release.
  • rosdistro's source: entry for humble points at main in this repo, which can no longer build on humble. Stale metadata worth correcting.

Nathan Brooks and others added 3 commits August 3, 2026 00:17
…ng Resolute gate

Every build workflow in this repo has been `disabled_inactivity` since roughly
2026-03. The split is exact: all 20 workflows carrying a `schedule:` trigger were
auto-disabled by GitHub, and all 11 without one are still active. GitHub disables
the whole workflow, not just its cron, so this repo has had zero build coverage on
any distro for months -- which is how it ended up un-released from rolling by
ros/rosdistro#51542 without a red job anywhere.

Replaces the 20 per-distro build workflows and both reusable workflows with a
single step-based industrial_ci matrix, following the shape moveit_visual_tools
and rviz_visual_tools converged on. No `schedule:` triggers, so this cannot
silently disable itself again.

Matrix on `main`: jazzy, kilted (+ccov), lyrical, rolling.

lyrical is a BLOCKING Resolute gate. It is released, so its `main` apt is
populated -- ros2_control 6.8.0 and realtime_tools 5.2.0 are both there, and this
repo depends on nothing else that is missing on Resolute. rviz_visual_tools#301
made the same call; moveit_visual_tools had to keep lyrical non-blocking only
because moveit_core has no Resolute deb. Rolling stays non-blocking on
ROS_REPO: testing, since Rolling's `main` apt has no Resolute packages yet.

Also:

- Coverage moves into the matrix on kilted. The old ci-coverage-build.yml ran
  ros-tooling/action-ros-ci on a noble runner outside a container and has been
  broken since the Resolute transition; its own comment proposed exactly this
  fix. It also only built picknik_reset_fault_controller, so
  picknik_twist_controller has never had coverage. Attached to kilted rather
  than rolling so coverage does not stop reporting whenever rolling breaks.

- ROS Lint pinned to lyrical -- the newest distro whose build job is blocking --
  and moved into a ros:lyrical-ros-base container, since GitHub has no Ubuntu
  26.04 runner and setup-ros cannot install lyrical on noble. It now lints the
  whole repo: the old config named `picknik_controllers`, which is not a package
  here, and never linted picknik_twist_controller.

- Add CONTRIBUTING.md. ament_copyright requires one at the repository root and
  validates its contents; this is its apache2 template verbatim, matching the
  repo's LICENSE.

- Drop the 8 `.repos` files. All are comment-only placeholders, so no
  UPSTREAM_WORKSPACE is needed -- and therefore neither is the safe.directory
  workaround that moveit_visual_tools#157 required.

- humble build + ABI workflows are removed from `main` only; the `humble` branch
  carries its own copies. This is temporary: once `main` has source-level distro
  guards per moveit2#3751, humble returns as one more matrix entry.

Verified locally before pushing: pre-commit clean, and all three ament linters
pass inside ros:lyrical-ros-base.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All three ABI workflows used the bare job id `abi_check` with no `name:`, so all
three reported as a check literally called `abi_check`. Branch protection matches
required status checks on the name string, so three identically-named checks
cannot be told apart -- there is no way to require jazzy's and kilted's while
leaving rolling's optional.

Names them jazzy-abi / kilted-abi / rolling-abi to match the build jobs' style.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…time

The previous commit collapsed the matrix to tier 1 and lost the point of the
original layout: the binary / semi-binary / source split is an escalating
lookahead for *incoming* upstream breakage, not redundancy. Restores it.

Tier 1 -- binary, `main` apt: can a user build this today? jazzy, kilted (+ccov),
lyrical. Only these gate merges.

Tier 2 -- binary, `testing` apt: what breaks at the next sync? jazzy, kilted,
lyrical, rolling. Non-blocking on purpose -- an upstream regression staged for
release is something to see, not something that should block an unrelated PR.
This repo has already had a "7 PRs blocked on an ecosystem break" episode.

Tier 3 -- semi-binary: what breaks in the mid future? Core ROS from debs, but
ros2_control and realtime_tools built from their development branches.

Tier 3 has never actually worked here. `picknik_controllers.rolling.repos` was
an entirely commented-out placeholder, so the semi-binary job was byte-identical
to the binary job -- the middle rung reported green while testing nothing. It is
now populated with the two immediate non-core dependencies, which makes this the
tier that would have caught LoanedCommandInterface::get_value() being removed,
weeks before it surfaced as a buildfarm release failure.

Carries the `AFTER_INIT` safe.directory workaround, which is required once an
upstream_ws cache exists (moveit_visual_tools#157 hit this) and was correctly
unnecessary while there was no real UPSTREAM_WORKSPACE.

The old source-build tier stays dropped. It was permanently red for a reason
unrelated to its value -- it fetched its .repos with the deprecated `?token=`
URL syntax and 404'd every run -- and rebuilding all of core ROS to test two
controllers is poor value when tier 3 covers the deps that can realistically
break us.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant