Skip to content

[Engine] avoid hanging on PleReader initialization failures - #3385

Draft
j-luwierski wants to merge 1 commit into
Niko1221:mainfrom
j-luwierski:codex/ple-reader-init-beta
Draft

j-luwierski wants to merge 1 commit into
Niko1221:mainfrom
j-luwierski:codex/ple-reader-init-beta

Conversation

@j-luwierski

@j-luwierski j-luwierski commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem and result

On current upstream main, PleReader::open() still hangs during cleanup if read-buffer allocation fails or if a reader is reopened with a table range beyond EOF. The failed open() never returns its existing error. After this change, it closes cleanly and a later valid open/read succeeds.

Related discussion

Retested the report in #3045, which was closed during the 0.2.0-beta tracker reset. The original PR #3046 was also closed in that reset.

Approach

Publish the in-flight slot count only after the buffers and free-slot list are ready, and reset the count when closing. Add Linux CPU-only lifecycle coverage for normal reads, failed reopen, and injected allocation failure in caller-thread and I/O-worker modes. The allocation failure uses link-time posix_memalign wrapping and does not add production fault-injection behavior.

Validation

Linux x86_64, GCC 14.2.0, CUDA 13.4.92, NVIDIA RTX 4070 Ti SUPER; base revision e88ee39d1d6be6e3364183af3b39b62a7ed6f211, candidate revision 9f4b65fc.

Passed:

  • CPU: cmake -S . -B build/ple-reader-beta -DSTRATA_BUILD_TESTS=ON -DSTRATA_WERROR=ON; cmake --build build/ple-reader-beta --target ple_reader_lifecycle_test direct_file_async_test -j2; and ctest --test-dir build/ple-reader-beta --output-on-failure -R '^(ple_reader_(reads|reopen|alloc)_(caller|threaded)|direct_file_async_test)$': targets built and all 7 tests passed.
  • CUDA: cmake -S . -B build/ple-reader-cuda -DSTRATA_BUILD_TESTS=ON -DSTRATA_WERROR=OFF -DSTRATA_ENABLE_CUDA=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.4/bin/nvcc; cmake --build build/ple-reader-cuda --target strata ple_reader_lifecycle_test direct_file_async_test -j2: all targets built.
  • CUDA CTest: ctest --test-dir build/ple-reader-cuda --output-on-failure -R '^(ple_reader_(reads|reopen|alloc)_(caller|threaded)|direct_file_async_test)$': all 7 tests passed.
  • Reproduced on unchanged base revision e88ee39d: both read controls passed; reopen and injected alloc cases timed out after 2 seconds in caller and threaded modes (4 hangs).
  • git diff --check origin/main...HEAD: passed.

Failed:

  • With STRATA_WERROR=ON, the CUDA build command cmake --build build/ple-reader-cuda --target strata ple_reader_lifecycle_test direct_file_async_test -j2 stopped on -Werror=misleading-indentation in unchanged src/core/layer.cpp. The CUDA build passed with WERROR disabled; warnings are outside this change.

Not run:

  • HIP and SYCL builds: hipcc, dpcpp and icpx are unavailable.
  • Windows runtime checks and real model inference.

Limits

The reproduction uses a synthetic file and real DirectFile I/O on Linux; it verifies initialization and recovery behavior, not a production memory-pressure incident or model inference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant