Skip to content

AVX512: merge-masked MaskedIncrement/MaskedDecrement for i32x16 - #14

Open
Auburn wants to merge 1 commit into
masterfrom
perf/avx512-masked-increment
Open

Auburn wants to merge 1 commit into
masterfrom
perf/avx512-masked-increment

Conversation

@Auburn

@Auburn Auburn commented Oct 17, 2026 •

Copy link
Copy Markdown
Owner

Problem

MaskedIncrement and MaskedDecrement have no AVX512 overload, so i32<16> uses the generic a - Masked( mask, Register( -1 ) ) from Generic/Functions.h. MSVC compiles that as written: a zero-masked move of the -1 constant (vmovdqa32 zmm {k}{z}), then vpsubd, with one more register live. GCC 16 converts the mask with vpmovm2d and then subtracts. Clang already turns both forms into vpmovm2d + vpsubd in most places.

FastNoise2 uses the int versions in its grid loops (AxisReset), the 4D Simplex rank counting and the Simplex domain warps.

Change

Add i32x16 overloads next to MaskedAdd: _mm512_mask_sub_epi32( a, mask, a, set1( -1 ) ) for increment and _mm512_mask_add_epi32( a, mask, a, set1( -1 ) ) for decrement. Each is one merge-masked add/sub with a broadcast constant. The result is the same in every lane. 1024-bit registers still split in the generic code, and each half now uses the new overload.

f32x16 stays on the generic version. I tried the same float overloads, and they weren't worth it:

  • With MSVC 19.51 on FastNoise2 master, SuperSimplex 3D/4D, Simplex 3D and FBm 3D got 8-12% slower. SuperSimplex 3D/4D spill more.
  • On top of the native copies PR (MSVC AVX512: copy 512 bit registers and masks through their native type #13), they gained only another 0.2-0.5%. MSVC 19.51 also compiled the surrounding float maths differently, so FastNoise2 output moved by up to 1.5e-4 relative.
  • MaskedDecrement would keep -0 in masked off lanes, where the generic a + Masked( m, -1 ) gives +0. The f32 masked decrement test checks for +0.

Effect

I built FastNoise2 master (8c2d080) against this branch and against FastSIMD master, then timed both on a Ryzen 9 7950X. Both builds are loaded in one process and timed interleaved on one pinned core. Each figure is the median of 3-5 processes with alternating order, over the default 63 node/dimension cases of the AVX512 feature set. AVX2 is unchanged and serves as a control. The numbers are new/old time:

Compiler AVX512 AVX2 (control)
MSVC 19.51 (VS 2026) 0.971 0.999
MSVC 19.44 (VS 2022) 0.950 1.000
clang-cl 23 1.001 0.998
GCC 16 (Linux) 0.998 1.000
  • MSVC 19.51: Simplex 4D 0.82, DomainWarpSimplex 4D 0.82-0.87, FBm Simplex 4D 0.83, White 4D 0.88, Value 3D/4D 0.90, SuperSimplex 3D/4D 0.97. No case is slower than 1.02.
  • MSVC 19.44: Value 3D/4D 0.72-0.73, White 3D/4D 0.74-0.75, Perlin 3D/4D 0.82-0.83, Simplex 3D/4D 0.86-0.87, SuperSimplex 3D/4D 0.88-0.91. No case is slower than 1.01.
  • Clang and GCC: within noise. GCC's DomainWarpSimplex 4D is 0.97.

Most of the MSVC gain comes from fewer stack spills. With 19.51:

  • GenUniformGrid 3D/4D has 610 → 576 and 664 → 613 stack references.
  • Simplex 4D Gen has 1041 → 970 instructions, with 357 → 310 stack references.

In the whole FastNoise2 DLL:

  • MSVC zero-masked moves go from 768 to 666, and merge-masked vpsubd/vpaddd from 184 to 286.
  • GCC vpmovm2d goes from 114 to 12.
  • Clang vpmovm2d goes from 154 to 130, and only 3 functions change.

On a FastNoise2 development branch with restructured grid and Simplex loops, MSVC 19.51 measured 1.002 with these overloads (on a busy machine). So the gain depends on how much register pressure the caller has.

Testing

  • FastSIMD tests at full count (4,194,304 inputs), seeds 24301, 1 and 2:
    • clang-cl 23, MSVC 19.51 and GCC 16: 628 passed, 52 known failures, 0 failed. Same list as master.
    • The changed ops are covered by i32 masked increment, i32 masked increment int mask and i32 masked decrement at 512 and 1024 bit.
    • As a check, I changed the decrement to add +1. The 512 and 1024 bit i32 masked decrement tests then fail on AVX512 and AVX512 RELAXED.
  • FastNoise2 output is bit-identical to FastSIMD master with clang-cl 23, MSVC 19.51, MSVC 19.44 and GCC 16. That covers all 1212 node, feature set and dimension cases.
  • Only x86/512/i32x16.h changes, so ARM and WASM builds aren't affected.

Other PRs

The generic fallback a - Masked( mask, -1 ) costs a zero-masked move plus the
add/sub on compilers that do not fold it (MSVC). Use _mm512_mask_sub/add with
a -1 broadcast instead; arithmetic is identical in every lane.

f32x16 keeps the generic version: with MSVC 19.51 the merge-masked float form
made SuperSimplex 3D/4D and Simplex 3D slower and changed the results of the
surrounding float maths, and MaskedDecrement would keep -0 in masked off lanes
where the generic a + Masked( mask, -1 ) gives +0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant