Skip to content

ci(tests): the resource-monitor test can see a sample written mid-line (flaky) #1888

Closed

Description

@JarryShaw

tests/project/test_resource_monitor_1740.py::TestTheSamplesReachTheTestsLog::test_samples_stream_into_the_tests_log_and_the_stop_step_ends_the_sampler failed on #1881's Python 3.12 job (117297298108), at test_resource_monitor_1740.py:177. The PR changes only docs/source/conf.py and docs tests, so the failure is unrelated to it.

The assertion checks that every sample line starts on its own line, even after pytest's unfinished progress line. One sample was written straight after the stub run's progress characters:

'uuu.u[monitor]    2438 692764 948892      02:05 /opt/hostedtoolcache/Python/3.12.15/x64/bin/pyt'

So the step that interleaves the sampler's output with the tests' output does not always start a sample on a new line. It depends on when the sample lands relative to pytest's writes, which makes it a timing race. The same test passes in local tests/project runs, including today's combined tree on main, so it is intermittent. A flaky blocking test turns any PR red at random.

Fix: make the start-of-line guarantee independent of timing in the workflow step and in the test's stub, or make the assertion tolerate the race only where the real step cannot cause it. Add a test that forces the interleaving deterministically, and check that it fails without the fix. The resource-monitor steps live in .github/workflows/experimental.yml.

Activity

  1. added
    ciPull requests that change CI or workflow configuration (ci: subject prefix)
    pendingAccepted and unstarted - nobody is on it and nothing blocks it
    wipWork in flight - a covering PR is open or an agent is actively on it
    and removed
    pendingAccepted and unstarted - nobody is on it and nothing blocks it
    on Oct 17, 2026
  2. JarryShaw commented on Oct 17, 2026

    @JarryShaw
    OwnerAuthor

    This flake hit a second PR. #1891's Python 3.14 job (117337423158) failed the same assertion with the same mid-line shape ('uuu.u[monitor] 2443 533152 711292 …'), and #1891 does not touch the monitor. #1890 fixes it, and its review is GOOD TO GO. I re-ran #1891's failed jobs.

  3. removed
    wipWork in flight - a covering PR is open or an agent is actively on it
    on Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ciPull requests that change CI or workflow configuration (ci: subject prefix)

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions