[jethac]

Native Windows cannot build the existing POSIX host layer. This adds a native MinGW UCRT64 CPU server and tested CPU executable artifacts for Windows x86-64, Linux x86-64/ARM64 and macOS ARM64.

Refs #3.

Changes

• Add Win32 socket/file mapping support, native subprocess argument handling, Windows cache defaults, secure binary temporary files and cache replacement. Preserve UNC share roots when creating cache directories.
• Keep the model graph, tokenizer, quantization and native/OpenAI HTTP contracts unchanged. Preserve existing accelerator build targets.
• Build CPU artifacts on native runners, run correctness checks before upload, verify executable format/architecture and package checksums, source revision, toolchain provenance and runtime license notices. Pin Actions and the public test model revision/hash.

Verification

All four native CI jobs passed at 2cba339.

Hosted environment	Compiler	Minimum embedding cosine
Ubuntu 24.04 x86-64	GCC 13.3.0, Ubuntu 13.3.0-6ubuntu2~24.04.1	0.999871254
Ubuntu 24.04 ARM64	Same GCC package	0.999874592
macOS 15 ARM64	Apple clang 17.0.0, clang-1700.0.13.5; Xcode 16.4 toolchain	0.999860704
Windows Server 2022 x86-64	MSYS2 UCRT64 GCC 16.2.0-4; CRT/winpthreads 14.0.0.r426.g4564ee4b5-1	0.999871254

CI commands on fresh checkouts:

make check && make
make test
# Windows UCRT64 additionally:
make check && make && make test-windows-download
make test
python scripts/check-binary-platform.py PLATFORM build/embeddinggemma[.exe]

All targets passed the 314-tensor manifest, exact tokenizer vectors (10/10), embedding goldens (10/10, cosine >=0.999), packed-batch parity (1.0), model-free units and HTTP/health/native/OpenAI/Matryoshka checks. Windows also passed downloader path/fallback/cleanup and synthetic UNC/binary-temp/cache-replacement tests. Windows imports were inspected to exclude MSYS/Cygwin/winpthreads/compiler runtime DLL dependencies.

Model revision: 8dd0ca2a66a8f14470acb0e2a71f801afbc5fb73; SHA256: 50d28e22432a148f6f8a86eab3700f92add5d1f54baf7790675a2a4dadbccf26. Model weights are excluded from artifacts and commits. Resolved rolling MSYS2 package versions are recorded, not represented as a fixed package snapshot.

Downloaded final Windows and Linux x86-64 artifacts independently passed:

python testdata/test_http_dimensions.py --binary PATH_TO_DOWNLOADED_EXECUTABLE --model model/embeddinggemma-300M-qat-Q4_0.gguf --backend cpu

All downloaded artifacts matched their checksums/source revision. Physical CPU models were not captured; runner architectures and executable formats were verified. No kernel optimization or performance claim is made.

The literal contributor clean-build check also passed natively at the same commit:

make clean-cpu && make test

Local host: Windows 11 Enterprise 10.0.26200, Intel Xeon W-2135 (6 cores/12 threads), MSYS2 MINGW64 GCC 16.1.0 Rev5 (POSIX threads), GNU Make 4.4.1 and Python 3.11.9. Tokenizer 10/10, embeddings 10/10 with minimum cosine 0.999869943, batch parity 1.0, units and HTTP checks passed.

Limits

Metal/CUDA/ROCm/XPU were not built or tested for these shared host-code changes. Intel macOS, Windows ARM64/GPU/MSVC, older OS/distribution compatibility, real SMB shares and arbitrary Unicode/extended-length Windows paths are unverified; UNC tests use fictional strings.

These are CI artifacts, not an official release. No version/tag, installer, seven-backend release matrix, release staging or release publication change is included. Linux Ubuntu 24.04 builds do not establish the runbook's older-glibc release compatibility. Explicit Darwin codesign/vtool and post-strip staged HTTP release gates were not run. Those release requirements remain separate.

Summary by CodeRabbit

• New Features
  • Added native Windows support for CPU-based inference, including model loading, downloads, and server operation.
  • Added CPU binaries for Linux x86-64 and ARM64, macOS ARM64, and Windows x86-64.
  • x86-64 builds can select optimized AVX2 processing at runtime when supported. Set EI_CPU_ISA to auto, baseline, or avx2 to control CPU instruction selection.
• Documentation
  • Updated installation and CPU performance guidance, including platform requirements and instructions for building and testing on Windows.

[coderabbitai]

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: fac40fb4-3962-4957-a1e2-d93d2d2cd0fc

📥 Commits

Reviewing files that changed from the base of the PR and between d8f9e33 and 7a5f816.

📒 Files selected for processing: 25

• .github/workflows/ci.yml
• CONTRIBUTING.md
• Makefile
• PORT_SPEC.md
• README.md
• perf/optimization_status.md
• scripts/check-avx2-isolation.py
• scripts/check-binary-platform.py
• src/common.h
• src/gguf.c
• src/inference_service.c
• src/kernels.c
• src/kernels_avx2.c
• src/kernels_avx2.h
• src/parallel.c
• src/quants.c
• src/server.c
• src/test_cpu_dispatch.c
• src/test_downloader_fixture.c
• src/test_gemma2.c
• src/test_inference_service.c
• src/test_kernels.c
• src/test_windows_paths.c
• src/windows_compat.h
• testdata/test_windows_download.py

+-----------------------------------------------------------------------------------------------------+
| Write code that writes code. Code generators increase your productivity and help avoid duplication. |
+-----------------------------------------------------------------------------------------------------+
  \
   \     /)/)
        ( '')
        /  |
      o(___)

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[ehartford]

Thanks for this, it's a really solid PR. The Windows host layer is careful, and the runtime AVX2 dispatch is a big win on its own: shipped x86 binaries were running the Q4 dots in scalar C.

Main had moved a long way (the EmbeddingGemma 2 profile landed), so I rebased your branch onto current main and force-pushed it to your fork's main using maintainer edits. Sorry for rewriting your default branch. To sync your local clone: git fetch origin && git reset --hard origin/main.

What changed

• Your 7 commits are kept, with your authorship. The conflicts in server.c, parallel.c, Makefile, CONTRIBUTING.md and perf/optimization_status.md are resolved to keep both sides.
• ei_cpu_count() is new on main, so I gave it the same Windows CPU-count code you used.
• I added one follow-up commit (7a5f816) for things main introduced after your base:
  • a setenv shim in windows_compat.h, because the server now maps --threads and --cuda-devices onto env vars
  • --profile embeddinggemma-300m in test_windows_download.py, because the default profile is now embeddinggemma-2
  • test_gemma2 now reads its UTF-8 text arguments from GetCommandLineW, because the CRT argv is in the ANSI code page and the Japanese golden failed
  • removed a duplicate windows_compat.h include in inference_service.c

Testing (the rebased tree is identical to what was tested)

• Linux x86-64 (Ryzen 9 9950X): make test and make test-gemma2 pass. Goldens min cosine 0.99986, SSE2/AVX2 parity 0.99987, AVX2 2.4-2.7x.
• Windows 10 22H2 VM, MSYS2 UCRT64 GCC 16.2: make test, make test-windows-download and make test-gemma2 (text, vision, audio, HTTP) pass. AVX2 2.1-2.2x. The .exe imports only system DLLs.

Small things, not blocking:

• send_header_and_body: the POSIX path changed sent < 0 to sent <= 0, so a zero return now checks a stale errno. I'd revert it to < 0.
• model_fingerprint: ftell returns a 32-bit long on Windows, so files over 2 GB would need _ftelli64.
• On Windows the persistent cache is flushed on Ctrl+C, but not on a kill or service stop, since there is no SIGTERM.
• Follow-up for later: main's EmbeddingGemma 2 x86 kernels pick their ISA independently and don't honor EI_CPU_ISA yet.

When you're happy with it, please take it out of draft.