Skip to content

fix: size YUY2, UYVY and NV12 frames of odd width in whole pixel pairs - #422

Open
devopvoid wants to merge 1 commit into
mainfrom
fix/packed-yuv-buffer-size
Open

devopvoid wants to merge 1 commit into
mainfrom
fix/packed-yuv-buffer-size

Conversation

@devopvoid

Copy link
Copy Markdown
Owner

YUY2 and UYVY rows, and NV12 and NV21 chroma rows, always end with a whole pixel pair. With an odd width, CalcBufferSize undercounted YUY2/UYVY frames, so a conversion wrote (or read) past the checked buffer, and ConvertFromI420 wrote these formats with other strides than ConvertToI420 reads them with.

  • One layout for both directions: sizes and strides rounded up to whole pairs, matching libyuv's ConvertToI420.
  • Sizes computed in 64 bits; empty frames, widths above INT_MAX / 4 and unknown formats are rejected.
  • Boundary tests for widths 1, 3 and 5 with guard bytes, round trips, and oversized frames.
  • Buffer sizing in the converter guide and Javadoc corrected.

YUY2 and UYVY rows, and NV12 and NV21 chroma rows, end with a whole pair,
so a frame of odd width was written past the buffer it was checked against,
and written with other strides than it is read with. Sizes are now computed
in 64 bits, and empty or oversized frames and unknown formats are rejected.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant