Skip to content

MSVC AVX512: copy 512 bit registers and masks through their native type #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Auburn merged 2 commits into from
Oct 17, 2026
Merged

MSVC AVX512: copy 512 bit registers and masks through their native type #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
79a9838
MSVC AVX512: copy f32x16/i32x16 through the native vector
Auburn Oct 14, 2026
9a765fa
MSVC AVX512: copy masks through the native mask, AnyMask with kortest
Auburn Oct 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions include/FastSIMD/ToolSet/x86/512/f32x16.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,12 @@ namespace FS
FS_FORCEINLINE Register() = default;
FS_FORCEINLINE Register( NativeType v ) : native( v ) { }
FS_FORCEINLINE Register( float v ) : native( _mm512_set1_ps( v ) ) { }
#if defined( _MSC_VER ) && !defined( __clang__ )
// MSVC copies trivially copyable 64 byte structs as two 32 byte halves, reloading those as a 512 bit register
// stalls store forwarding. Copying the native vector keeps register copies in zmm registers
FS_FORCEINLINE Register( const Register& v ) : native( v.native ) { }
FS_FORCEINLINE Register& operator =( const Register& v ) { native = v.native; return *this; }
#endif

FS_FORCEINLINE NativeType GetNative() const
{
Expand Down
6 changes: 6 additions & 0 deletions include/FastSIMD/ToolSet/x86/512/i32x16.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,12 @@ namespace FS
FS_FORCEINLINE Register() = default;
FS_FORCEINLINE Register( NativeType v ) : native( v ) { }
FS_FORCEINLINE Register( std::int32_t v ) : native( _mm512_set1_epi32( v ) ) { }
#if defined( _MSC_VER ) && !defined( __clang__ )
// MSVC copies trivially copyable 64 byte structs as two 32 byte halves, reloading those as a 512 bit register
// stalls store forwarding. Copying the native vector keeps register copies in zmm registers
FS_FORCEINLINE Register( const Register& v ) : native( v.native ) { }
FS_FORCEINLINE Register& operator =( const Register& v ) { native = v.native; return *this; }
#endif

FS_FORCEINLINE NativeType GetNative() const
{
Expand Down
15 changes: 15 additions & 0 deletions include/FastSIMD/ToolSet/x86/512/mNx16.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,12 @@ namespace FS

FS_FORCEINLINE Register() = default;
FS_FORCEINLINE Register( NativeType v ) { this->native = v; }
#if defined( _MSC_VER ) && !defined( __clang__ )
// MSVC writes copies of a 32 lane composite mask with two 16 bit stores and reads them back with one 32 bit load,
// which can't be store forwarded. Copying the native mask keeps the halves in registers
FS_FORCEINLINE Register( const Register& v ) { this->native = v.native; }
FS_FORCEINLINE Register& operator =( const Register& v ) { this->native = v.native; return *this; }
#endif

FS_FORCEINLINE NativeType GetNative() const
{
Expand Down Expand Up @@ -67,4 +73,13 @@ namespace FS
{
return static_cast<BitStorage<16>>( a.native );
}

#if defined( _MSC_VER ) && !defined( __clang__ )
// MSVC moves both mask halves to general purpose registers to OR them
template<FastSIMD::FeatureSet SIMD, std::size_t N, bool B, typename = EnableIfNative<Register<Mask<N, B>, 16, SIMD>>>
FS_FORCEINLINE bool AnyMask( const Register<Mask<N, B>, 32, SIMD>& a )
{
return !_mm512_kortestz( a.v0.native, a.v1.native );
}
#endif
}
Loading