Problem
integration_tests is diff-gated (.github/workflows/test.yml):
- uses: technote-space/get-diff-action@v6.1.2
with:
PATTERNS: |
**/**.sol
**/**.go
go.mod
go.sum
tests/integration_tests/**
- name: Run integration tests
run: make run-integration-tests
if: env.GIT_DIFF
When GIT_DIFF is empty the step is skipped, and a skipped step still makes the job report success. As a required check, that is indistinguishable from a real pass.
Effect
Every dependency bump reports integration_tests (unmarked): success without running anything. Between 2026-07-30 and 2026-09-16 the suite did not execute on develop at all, yet the branch showed green the whole time:
success Run technote-space/get-diff-action@v6.1.2
skipped Run integration tests
(from run 34244306239, develop @ 2253b15)
Because the suite also pulls its fixtures from upstream ethereum/execution-apis, real breakage accumulated invisibly and then failed on the next PR that happened to touch a .go file — #1099, whose diff had nothing to do with it.
Suggested fix
Make "did not run" distinguishable from "passed". Options:
- A final aggregation job that the branch protection requires, which asserts each upstream job actually ran rather than treating
skipped as satisfied.
- Drop the path filter for jobs used as required checks, and control cost with concurrency/caching instead.
- Keep the filter but have the job fail when it is required and
GIT_DIFF is empty.
Related: #1101 (stale results satisfying the gate). The fixture pin in #1100 removes the upstream moving target but leaves this reporting hole.
Problem
integration_testsis diff-gated (.github/workflows/test.yml):When
GIT_DIFFis empty the step is skipped, and a skipped step still makes the job reportsuccess. As a required check, that is indistinguishable from a real pass.Effect
Every dependency bump reports
integration_tests (unmarked): successwithout running anything. Between 2026-07-30 and 2026-09-16 the suite did not execute ondevelopat all, yet the branch showed green the whole time:(from run 34244306239,
develop@ 2253b15)Because the suite also pulls its fixtures from upstream
ethereum/execution-apis, real breakage accumulated invisibly and then failed on the next PR that happened to touch a.gofile — #1099, whose diff had nothing to do with it.Suggested fix
Make "did not run" distinguishable from "passed". Options:
skippedas satisfied.GIT_DIFFis empty.Related: #1101 (stale results satisfying the gate). The fixture pin in #1100 removes the upstream moving target but leaves this reporting hole.