DRAFT: Add Zstd GPU CI test infrastructure (Phase 1)#108
Open
brohan203 wants to merge 6 commits into
Open
Conversation
brohan203
force-pushed
the
users/rohanborkar/ds_testInfra_phase1
branch
from
89bc1c9 to
9a677ac
Compare
Add zstdgpu_ci_tests: a thin GTest wrapper that shells out to zstdgpu_demo.exe for correctness and performance validation of GPU Zstd decompression. Tests are parameterized over .zst content files discovered at runtime via --content-path. Test cases: - SimulationCheck (--chk-gpu --chk-cpu --sim-gpu) - D3D12DebugLayer (--d3d-dbg) - ExternalMemory (--ext-mem) - GraphicsQueue (--d3d-gfx) - OverallThroughput (--prf-lvl 0) - PerStageTiming (--prf-lvl 2) Also adds scripts/generate_histogram.py for converting performance CSV output into histogram PNGs suitable for CI reporting. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ectness failure Adds a translation-unit-only override of the ZSTDGPU_BREAK macro for the demo's CPU-sim shader path: instead of the library's __debugbreak() -- which kills the demo with STATUS_BREAKPOINT (0x80000003) on bad input before any [FAIL] message can flush -- breaks now log a [ZGBRK] line and increment a correctness-failure counter. VALIDATE / VALIDATE_CND also increment the counter, the per-frame validation failure block and uncompressed-size early-out increment it, and wmain returns 2 if the counter is non-zero. The validate-on-CPU path early-returns before DecompressSequences if any prior validation failed -- avoids feeding sentinel FSE table indices into the OOB-prone shader code. All changes are local to zstdgpu_demo/main.cpp. zstdgpu library is untouched. Performance CSV emission is provided upstream by PR microsoft#118 (zstdgpu_demo's built-in --out-csv flag); this PR no longer adds a separate CSV writer.
brohan203
force-pushed
the
users/rohanborkar/ds_testInfra_phase1
branch
from
5115e9b to
1456908
Compare
added 3 commits
July 1, 2026 18:28
…icro) Six groups of fixes to the CI test infrastructure: A. Validate all required inputs in main() with hard exit codes instead of silently skipping tests. Also use the non-throwing overload of std::filesystem::recursive_directory_iterator so a single unreadable subdirectory logs a warning and continues instead of aborting the entire process before any test runs. B. Cache the discovered .zst file list on TestConfig in main(); reuse in GetTestFiles(). Avoids walking the content tree twice. C. Remove result.stdOut from ASSERT_EQ / EXPECT_EQ failure messages - it was already printed unconditionally as the [DEMO OUT] block above, which caused each failing test's stdout to appear twice in the log. D. Remove GTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST(ZstdGpuDemoTests). Group A now hard-fails if the content path has zero .zst files, so empty instantiation can no longer happen at test time. Keeping the macro would let a mis-typed --content-path silently pass green with zero coverage. E. Call CancelIoEx(hReadPipe, nullptr) after TerminateProcess on the timeout path so any pending ReadFile in the drain thread returns and readerThread.join() cannot hang the test runner indefinitely. Trust that CancelIoEx alone unblocks the reader; no bounded-wait fallback. F. Replace placeholder GUID {A1B2C3D4-...} in zstd.sln with a real one: {17F60A53-4A7F-4107-AF0A-914497018D67}. G. Nits: drop the "Spec implies a global" and "Spec inlines the args" comments in the header, and the stray blank line. Regression tested locally on RTX 2080 against a 20-file fuzz subset with --run-count 5 (all 6 scenarios per file): Flavor Before wall After wall Pass/fail counts ------------- ----------- ---------- ---------------- r1+fflush 223.6s 225.5s unchanged (58/125) vanilla+fflush 543.1s 528.2s unchanged (58/125) Wall time within run-to-run noise (+/-0.8%). Marker-count reductions ([FAIL] -50%, [INFO] -14.5%) are the expected consequence of Group C removing the duplicate stdout print on failure - same content, half the log lines when tests fail.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft - not ready for merge. Putting this up for early visibility and code review.
Adds a thin GTest wrapper (
zstdgpu_ci_tests) that shells out tozstdgpu_demo.exeto validate Zstd GPU decompression shaders in CI. Also adds pipeline YAML and a histogram generation script for perf results.What's here
zstd/zstdgpu_ci_tests/— GTest parameterized test fileszstd/scripts/generate_histogram.py— Generates histogram PNGs from perf CSV output. This can probably be improvedOpen Q's
What's not here
zstdgpu_demo.exe. ATG owns. This PR does not include--output-csvflag and non-zero exit code changesATG dependencies
--output-csvflag onzstdgpu_demo.exefor perf CSV outputzstdgpu_demo.exewhen validation fails (correctness tests currently pass vacuously)