[naruto-lgtm]

Describe your changes

Cause: H5P__dcrt_layout_dec() decodes a chunked layout from an encoded property list (reached from the public H5Pdecode()). It reads the chunk rank as a single byte and only rejects zero, then writes that many dimensions into tmp_layout.u.chunk.dim[], a fixed array of H5O_LAYOUT_NDIMS (33) entries. A rank above that count overruns the stack struct and runs past the end of the input buffer. The object header layout message decoder (H5O__layout_decode()) already rejects ndims > H5O_LAYOUT_NDIMS; the property-list decoder did not.

Fix: reject a rank greater than H5O_LAYOUT_NDIMS before the decode loop, keeping the bound in the decoder where the untrusted count is consumed.

Repro: with ASan, H5Pdecode() on an encoded DCPL whose chunk-rank byte is set to 200 faults with a heap-buffer-overflow in H5P__dcrt_layout_dec(); after the change the same buffer is rejected with an error. A regression test covering this is added to test/enc_dec_plist.c.

Issue ticket number (GitHub or JIRA)

N/A

Checklist before requesting a review

• My code conforms to the guidelines in CONTRIBUTING.md
• I made an entry in release_docs/CHANGELOG.md (bug fixes, new features)
• I added a test (bug fixes, new features)

[github-actions]

Review Checklist

This PR touches the following areas. Each needs a sign-off
from its listed owners before merging.

• src — @jhendersonHDF
• test — @jhendersonHDF
  • ⚠️ Changes requested by @bmribler
• release_docs — @hyoklee

[bmribler]

Suggest to follow the same error handling approach as the existing test functions in the file.

[naruto-lgtm]

Done in d40ce6e. The test now follows test_encode_decode(): TEST_ERROR/PUTS_ERROR, returns -1 on failure and cleans up in the error block, with TESTING/PASSED moved into main. That also fixes a real slip, since the old version returned 1 on failure and main only checks < 0, so a regression would have gone unnoticed.