Skip to content

dat: size vertex arrays from display lists - #3653

Open
itsgrimetime wants to merge 7 commits into
doldecomp:masterfrom
itsgrimetime:codex/ribbanya-dat-vertex-sizing
Open

dat: size vertex arrays from display lists#3653
itsgrimetime wants to merge 7 commits into
doldecomp:masterfrom
itsgrimetime:codex/ribbanya-dat-vertex-sizing

Conversation

@itsgrimetime

@itsgrimetime itsgrimetime commented Oct 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Size vertex arrays from the largest index in their display lists, retaining the largest count across shared arrays and descriptor lists. Continues Ribbanya's wip/dat-vtx-size and preserves his commit. Descriptors without display-list bindings retain their DAT_EXTENT fallback; exact shape-animation index-list sizing remains a TODO.

Uses the generated readers merged in #3658. Expressions, byte-valued bindings and recounts compile into C readers. Shared helpers decode GX bytes and manage storage; archive.c remains unaware of DAT annotations. Independent vertex fixtures compile through the production emitter.

Validation: matching GALE01 MWCC binary; all six native tests, including traversal/readback parity across 7,050 archives and production HSD dispatch for 27 fighters. The feature patch is unchanged by the rebase; its prior validation includes 62 Rust tests with actual game DWARF, Clippy, 605/605 matching samples in 838 objdiff units (99.99% archive-byte coverage), and ASan/UBSan fixtures and five real archives. Generated C and objdiff inspected; generated files remain uncommitted. All eight CI build/check jobs pass on be6fad4.

@itsgrimetime itsgrimetime added the ai-assisted Utilizes a LLM to do the heavy lifting label Oct 6, 2026
@itsgrimetime
itsgrimetime force-pushed the codex/ribbanya-dat-vertex-sizing branch from 5885e34 to 450eed9 Compare October 6, 2026 15:54
@itsgrimetime
itsgrimetime marked this pull request as ready for review October 6, 2026 16:00
@itsgrimetime itsgrimetime added the portability Improves non-matching builds label Oct 6, 2026
ribbanya
ribbanya reviewed Oct 6, 2026
View reviewed changes
Comment thread config/GALE01/dat_symbols.txt Outdated
ribbanya
ribbanya reviewed Oct 6, 2026
View reviewed changes
Comment thread src/melee/ft/types.h Outdated
@itsgrimetime
itsgrimetime force-pushed the codex/ribbanya-dat-vertex-sizing branch 6 times, most recently from ac38932 to 09e2137 Compare October 8, 2026 21:49
ribbanya
ribbanya reviewed Oct 9, 2026
View reviewed changes
Comment thread tools/dat-cli/native/src/archive.c

@ribbanya ribbanya Oct 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is a horrible mess, which is my fault, but sometime soon it needs to be refactored. It's intended to be a thin runtime layer to load archives for a native port, and the complex per-type parsing is generated as C by the Rust code. Instead it's directly duplicating a large portion of the Rust archive code, which is introducing an insane amount of churn and bloat to this file. The runtime codegen was hastily thrown together and I think my Claude misunderstood what I wanted to do with archive.c, and now yours is just running wild with it. I would fix it myself but I really won't have time anytime soon.

Maybe fix in a separate PR, but just know that this is causing issues in every subsequent iteration of the tool until it's fixed.

@ribbanya ribbanya Oct 9, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

archive.c should have absolutely no idea what anything in the dwarf markup is. It should be a native port of baselib/archive.c and lbarchive.c and then as I said, more specific interpretations of public symbols can be generated during the cmake build process.

@itsgrimetime itsgrimetime Oct 10, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can do the refactor in a different PR, I understand what you're going for, then will update this one after that refactor lands

@itsgrimetime itsgrimetime Oct 10, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactor in #3658

@itsgrimetime itsgrimetime mentioned this pull request Oct 10, 2026
Merged
@itsgrimetime
itsgrimetime force-pushed the codex/ribbanya-dat-vertex-sizing branch 2 times, most recently from 0c64ac7 to cb33ec4 Compare October 11, 2026 02:09
@ribbanya
ribbanya marked this pull request as draft October 11, 2026 16:56
ribbanya and others added 5 commits October 11, 2026 12:02
@ribbanya @claude @itsgrimetime
dat: size vertex arrays from the display lists
9ee0c65 
Expressions evaluate to bytes as well as integers: a pointer field whose
own DAT_COUNT or DAT_TERMINATED gives its length is the data it points
to, which calls and DAT_BIND take. GXMaxIndex decodes a display list
with its vertex descriptors and returns an attribute's largest index;
HSD_PObjDesc binds its display lists and descriptors, and
HSD_VtxDescList.vertex counts (largest index + 1) * stride. Descriptor
lists several shapes share recount their arrays for each shape, so a
shared array is as long as the longest. Mirrored in the native walker,
with unit tests in both.

Also create obj/<dir> before compiling a unit's samples.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@itsgrimetime
dat: retain vertex extents without display-list bindings
cce14d0
@itsgrimetime
dat: preserve CPU sizing with byte-valued expressions
e2dbfc7
@itsgrimetime
dat: clarify vertex sizing and extent fallback documentation
0f05a8e
@itsgrimetime @ribbanya
dat: generate native vertex array sizing
be6fad4 
Compile byte-valued bindings and expressions into C readers. Recount shared descriptor arrays under each display-list context, retaining extent fallbacks without restoring the annotation interpreter.

Co-authored-by: Robin Avery <robin@ribbanya.dev>
@itsgrimetime
itsgrimetime force-pushed the codex/ribbanya-dat-vertex-sizing branch from cb33ec4 to be6fad4 Compare October 11, 2026 19:02
@itsgrimetime
itsgrimetime marked this pull request as ready for review October 13, 2026 22:51

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ribbanya ribbanya ribbanya left review comments

Assignees

No one assigned

Labels

ai-assisted Utilizes a LLM to do the heavy lifting portability Improves non-matching builds

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@itsgrimetime @ribbanya