Repository navigation
Rework test harness - #12
Merged
Merged
Conversation
- Every register size (128, 256, 512 and 1024 bit) is tested, each built as its own pair of dispatch libraries to keep compile memory down - Every compiled feature set up to the CPU's is compared with the SCALAR reference, plain and relaxed. Non relaxed results must match bit for bit, relaxed ones within per test ULP/absolute tolerances - Plain C++ oracles check the SCALAR reference itself, and relaxed SCALAR (std:: maths) checks FastSIMD's own accuracy - Inputs start with every combination of special values (+-0, denormals, infinities, NaN payloads, rounding ties, int limits, mask patterns), then seeded random values from per test input specs - Known failures are explicit, scoped by tier, feature set range and optionally input values, and reported when no longer seen - Exit code, ctest registration and a Release ctest step in CI - Options: --seed, --count, --filter, --width, --max-feature-set, --stats
MSVC 19.51 (VS 2026) optimises xor( ifFalse, and( mask, xor( ifTrue, ifFalse ) ) ) wrongly where FS::ATan2 inlines it, giving wrong results on the SSE2 feature set. The mirrored xor( ifTrue, andnot( mask, xor( ifTrue, ifFalse ) ) ) is the same 3 instructions and register use, and compiles correctly.
- Input generation moves to test_inputs.h/.cpp, test.cpp keeps test collection, the run loop, the summary and main - Tests are collected with a runtime loop over the compiled feature sets, only collection depends on the register size - A tier is one kind (non relaxed, relaxed or oracle) and every setting is read from the SCALAR non relaxed reference. Tiers registering different known failure counts, bit exactness or tolerances are rejected, and duplicate names are caught in every tier - Known failures are named values scoped once and registered with .Known( ... ), RegisterTests is split by category - Known failure input conditions see a lane's arguments the same way the oracle does - Each argument has one typed input buffer, mask lanes are stored as the type they're compared in - Comparing a call returns its outcome, the return type is switched on once per test - Input specs default to every value of the argument's type, Finite and NoNaN narrow it. In::Any() is gone so specs written for the old default don't compile Inputs and results are unchanged for every seed. Checked by fingerprinting every test's inputs and tier results against the previous harness on MSVC x64 (full count), Linux GCC/Clang and AArch64 GCC/Clang, GCC only differs in which NaN payload propagates. "Generating Tests" now prints in ascending feature set order and the MSVC Debug f32 abs known failure is one entry.
- One TestValue trait says how each kind of argument and result (scalar, register, mask register) is loaded, passed to the oracle and stored, replacing eight per shape trait families - Failure inputs and known failure lane arguments are read from the typed input buffers at runtime, TestArgType has each argument's lane count, so neither is instantiated for every test and tier any more - Collected tests keep the SCALAR non relaxed registration whole and only the test function of the other tiers. Tiers are checked against the reference while collecting, known failures are compared by value rather than count, and every test's inputs are validated, not only those matching --filter Inputs and results are unchanged for every seed. Checked against the previous harness with NaN payloads canonicalised on MSVC x64 (65536 and full count), Linux GCC/Clang and AArch64 GCC/Clang. Clang matches bit for bit, MSVC and GCC only differ in which NaN payload propagates.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebuilds the test harness so it covers every register size and feature set, checks results against exact references, reports pass/fail, and runs in CI. The library bugs it found were fixed in #9.
Harness
std::maths, so it also checks FastSIMD's own accuracy (ScalarRelaxedAccuracy), e.g. Sin/Cos/ATan/Exp/Log/Pow polynomials.KnownFailure,KnownZeroSign,KnownFailureIf). Known failures that are no longer seen are reported.FASTSIMD_CTEST_INPUT_COUNT), and a Release ctest step in every CI job. Emscripten Debug links without debug info, which would otherwise make the single file output over 1 GB.--seed,--count,--filter,--width,--max-feature-set,--stats(largest relaxed errors against their tolerances).f32 pow zero basetest from Fix Log precision and Pow returning 1 for a zero base #10 is kept, asPow( 0, b )with an exact oracle.MSVC 19.51 workaround
The Windows runners now use MSVC 19.51 (VS 2026), which miscompiles the SSE2 f32
Select(xor( ifFalse, and( mask, xor( ifTrue, ifFalse ) ) )) whereFS::ATan2inlines it, so the SSE2 atan2 tests failed. The second commit uses the mirroredxor( ifTrue, andnot( mask, xor( ifTrue, ifFalse ) ) ), same instructions and register use. Verified with 19.51 and 19.44 (628 passed, 0 failed).Known library behaviour, documented as known failures
Modulusisa - trunc( a / b ) * b, not exact likestd::fmod.std::atan2special cases, the AVX512 native masked add zero sign, and unoptimised MSVC'sstd::absquieting signalling NaNs.Testing
On current master:
Before the rebase onto master, the same harness also passed full count runs on Win32, ClangCL, MinGW and Emscripten.