json: Serialise GapToJsonString directly in the kernel#36
Open
ChrisJefferson wants to merge 1 commit into
Open
json: Serialise GapToJsonString directly in the kernel#36ChrisJefferson wants to merge 1 commit into
ChrisJefferson wants to merge 1 commit into
Conversation
GapToJsonString previously dispatched through the _GapToJsonStreamInternal GAP operation once per element, which dominated its runtime on record-heavy data. Serialise integers, booleans, strings, lists and records directly in the kernel instead, writing into a self-managed output buffer, and fall back to the GAP operation only for types the C code does not handle (floats, large integers, component objects, and user-installed methods). The buffer tracks the string bag together with its length and capacity as plain integers, materialising the data pointer freshly after each growth, so no raw pointer is ever held across an allocation. Records are sorted by component name via SortPRecRNam before output, matching the previous Set(RecNames(...)) behaviour and keeping the emitted JSON byte-identical. AI disclosure: this change was prepared with the assistance of Claude Code (Anthropic). Claude explored the existing serialiser, proposed and iterated on the C port and its GC-safety design, wrote the kernel and GAP code, and ran the regression, round-trip and garbage-collection stress tests. All work was done under the author's direction and review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #36 +/- ##
==========================================
+ Coverage 96.42% 96.50% +0.07%
==========================================
Files 6 6
Lines 140 143 +3
==========================================
+ Hits 135 138 +3
Misses 5 5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GapToJsonString previously dispatched through the _GapToJsonStreamInternal GAP operation once per element, which dominated its runtime on record-heavy data. Serialise integers, booleans, strings, lists and records directly in the kernel instead, writing into a self-managed output buffer, and fall back to the GAP operation only for types the C code does not handle (floats, large integers, component objects, and user-installed methods).
The buffer tracks the string bag together with its length and capacity as plain integers, materialising the data pointer freshly after each growth, so no raw pointer is ever held across an allocation. Records are sorted by component name via SortPRecRNam before output, matching the previous Set(RecNames(...)) behaviour and keeping the emitted JSON byte-identical.
AI disclosure: this change was prepared with the assistance of Claude Code (Anthropic). Claude explored the existing serialiser, proposed and iterated on the C port and its GC-safety design, wrote the kernel and GAP code, and ran the regression, round-trip and garbage-collection stress tests. All work was done under the author's direction and review.