fix(prompts): include real page number in figure/chart image tags - #475
Open
SomSamantray wants to merge 2 commits into
Open
fix(prompts): include real page number in figure/chart image tags#475SomSamantray wants to merge 2 commits into
SomSamantray wants to merge 2 commits into
Conversation
Threads the actual page number into build_no_anchoring_v4_yaml_prompt's instructed image tag (page_startx... -> <page_num>_startx...) instead of asking the model to invent a page number it has no way to know. Updates the three call sites that already have a real per-page number in scope (pipeline.py, bench run_server.py, grpo_train.py); the two remaining callers that only ever process an isolated single page keep the new default of page_num=1, matching this repo's existing single-page convention.
Adds a direct unit test for build_no_anchoring_v4_yaml_prompt's new page_num parameter, and strengthens test_grpo.py's existing mocked test to assert the grpo_train.py call site actually passes page_num through (previously the mock accepted any/no arguments silently).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #384
Changes proposed in this pull request:
page_startx_starty_width_height.png), so a caller couldn't tell which page a figure came from. The model was also never told the actual page number anywhere in its prompt or image input, so the originally-proposed fix (renaming the placeholder to literalpage_number) wouldn't have worked — there was nothing for it to substitute.build_no_anchoring_v4_yaml_prompt(the live-pipeline prompt) now takespage_numand interpolates the real value into the instructed tag:<page_num>_startx_starty_width_height.png.olmocr/pipeline.py,olmocr/bench/runners/run_server.py,olmocr/train/grpo_train.py.olmocr/train/dataloader.pyandscripts/hf_local_test.pyunchanged — they only ever process an isolated single page, so the newpage_num=1default already matches this repo's existing single-page convention there.build_no_anchoring_v4_yaml_prompt's interpolation, and strengthenedtest_grpo.py's existing mocked test to assert thegrpo_train.pycall site actually passespage_numthrough (previously the mock accepted any/no arguments silently).Before submitting
section of the
CONTRIBUTINGdocs.Writing docstrings section of the
CONTRIBUTINGdocs.