Skip to content

test(lance): cover the runtime config boundary - #1162

Open
Moviw wants to merge 1 commit into
NVIDIA:mainfrom
Moviw:test/lance-runtime-config-contract
Open

test(lance): cover the runtime config boundary#1162
Moviw wants to merge 1 commit into
NVIDIA:mainfrom
Moviw:test/lance-runtime-config-contract

Conversation

@Moviw

@Moviw Moviw commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Background

lance declares a single runtime test, test_lance_vl_pipeline, gated on
REQUIRES_TRT,REQUIRES_GPU. Its VL preprocessing config boundary therefore has
no coverage in the public CPU tier.

That boundary is unusual. lance is the only family that resolves its
preprocessing config from two bundle sections at once — plugin.cpp hands
lance_parse_preprocess_config() both config.json and
preprocessor_config.json:

const std::string config_text = bundle_section_text(ctx.bundle, "config.json");
const std::string preproc_text =
    bundle_section_text(ctx.bundle, "preprocessor_config.json");
auto vl_preprocess = lance_parse_preprocess_config(config_text, preproc_text);

The two sources do not merge in a single direction.
apply_preprocessor_config_overrides() runs before
apply_config_image_norm_overrides(), so:

Keys Winning source
patch_size, merge_size, temporal_patch_size preprocessor_config.json
image_mean, image_std config.json

Nothing pinned either direction. Both are invisible at the call site, and both
fail by producing plausible wrong pixels rather than an error.

Exit Criteria

  • The lance VL preprocessing config boundary is covered in the CPU tier, using
    the same entry point plugin.cpp calls.
  • Both precedence directions are pinned, so reordering the two override steps
    fails the suite.
  • The resample fallback is pinned as suppressed whenever config.json states
    an interpolation of its own.

Non-goals: no production behavior is changed, and image decoding, pixel
preprocessing and mrope position building stay out of scope — they need image
fixtures rather than a config contract.

Implementation

Add tests/cpp/models/lance/test_lance_runtime_config_contract.cpp and register
it in src/runtime/models/lance/MODEL.toml as a CPU test that compiles
image_preprocessor.cpp directly, mirroring the entry already used by
deepseek_ocr and internvl.

The fixture uses non-default values throughout, so a consumer that stopped
reading a key and fell back to the struct default fails. Beyond the two
precedence directions the test pins:

  • the escaped-newline decode applied to vl_prompt_template;
  • resample 0/2/3 resolving to nearest/bilinear/bicubic, and an unknown value
    keeping the default, only when config.json states no interpolation;
  • the surprising half of the first direction — a preprocessor_config.json that
    omits the geometry keys still overrides them, resetting config.json's values
    to 14/2/2 rather than preserving them;
  • an absent preprocessor_config.json section leaving config.json's geometry
    standing.

No production source is touched.

Change categories

  • Model or runtime behavior
  • Public API
  • ABI
  • Bundle or artifact format
  • Dependencies
  • Documentation only
  • CI or developer tooling

Validation

Commands and Results

Compilation and unit:

cmake --build build-cpu4 --target test_lance_runtime_config_contract
./build-cpu4/test_lance_runtime_config_contract        # exit 0

Repository gates:

clang-format --dry-run --Werror \
  tests/cpp/models/lance/test_lance_runtime_config_contract.cpp   # clean
python tools/model_ci.py validate                                 # exit 0, 89 families
python -m pytest tests/tools/test_family_model_toml_loader.py \
  tests/tools/test_model_ci.py tests/tools/test_model_checks.py   # 175 passed

Mutation check, three ways — each mutation applied to
src/runtime/models/lance/image_preprocessor.cpp, then reverted:

Mutation Result
Swap apply_preprocessor_config_overrides() and apply_config_image_norm_overrides() FAIL: image mean comes from config.json, FAIL: image std comes from config.json
Drop the stated-interpolation guard in maybe_apply_resample_fallback() FAIL: stated interpolation survives the resample fallback
Make the absent geometry overrides preserve config.json instead of resetting FAIL: an absent patch_size override resets to 14 and the two sibling checks

After reverting all three, the test returns to exit 0.

Hardware, Environment, and Revisions

  • Repository head: 3ee90b6e11d8929849271aaf067ade6bd627e416 (upstream/main).
  • Host: x86_64 Ubuntu 22.04, GCC 11.4.0, CMake + Ninja, clang-format 22.1.8.
  • Build: CPU-only configuration, TRTMC_BUILD_BACKEND_TRT=OFF,
    TRTMC_BUILD_BACKEND_RTX=OFF. No TensorRT SDK and no GPU are used by this
    test.
  • No model checkpoint, dataset or bundle is downloaded; the fixture is inline
    JSON text.

Not Run / Remaining Gaps

  • test_lance_vl_pipeline was not run: it stays gated on REQUIRES_TRT,REQUIRES_GPU
    and is unchanged by this PR.
  • Image decoding, pixel preprocessing and lance_build_mrope_positions() are not
    covered here; they need image fixtures rather than a config contract.
  • Internal CI has not run on this branch.

Notes For Future Readers

The opposite precedence between the geometry keys and the normalization triplets
is the thing worth remembering. It is a consequence of statement order in
lance_parse_preprocess_config(), not of an explicit policy, so if the merge is
ever unified in one direction this test is the place that will say so.

The geometry reset is pinned as current behavior, not endorsed as correct: a
preprocessor_config.json that omits patch_size silently discards
config.json's value and substitutes 14. If that turns out to be wrong for a
real checkpoint, the fix belongs in apply_preprocessor_config_overrides() and
this test should be updated in the same change.

Suggested review order: MODEL.toml registration, then the fixture, then the
assertions.

Risk level

  • Low
  • Medium
  • High

Test-only. No production source is modified, and the new test runs in the CPU
tier without TensorRT or a GPU.

Lance declared only a REQUIRES_TRT,REQUIRES_GPU runtime test, so its VL
preprocessing config boundary had no coverage in the public CPU tier.

Lance is the only family that resolves that config from two bundle sections at
once: plugin.cpp hands lance_parse_preprocess_config() both config.json and
preprocessor_config.json. The two sources do not merge in a single direction.
apply_preprocessor_config_overrides() runs before
apply_config_image_norm_overrides(), so patch_size, merge_size and
temporal_patch_size are taken from preprocessor_config.json while image_mean and
image_std are taken from config.json. Nothing pinned either direction, and the
opposite precedence is invisible at the call site.

Add a CPU consumer contract over lance_parse_preprocess_config() covering both
directions, the escaped-newline decode applied to vl_prompt_template, and the
resample fallback that must stay suppressed whenever config.json states an
interpolation of its own. Also pin the surprising half of the first direction: a
preprocessor_config.json that omits the geometry keys still overrides them,
resetting config.json's values to 14/2/2 rather than preserving them.

Mutation-checked three ways. Swapping the two override calls fails the image
mean and std checks; dropping the stated-interpolation guard fails the resample
check; making the absent overrides preserve config.json instead of resetting
fails the three reset checks. Each mutation was reverted and the test returns to
green.

Follows the CPU contract pattern already established for internvl, qwen_vl,
qwen3_5, locateanything and deepseek_ocr.

Signed-off-by: Moviw <xvzimo@gmail.com>
@Moviw
Moviw requested a review from yifeif-nv as a code owner September 4, 2026 13:02
@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a42ad133-7c70-46c5-b2e2-619865c09e26

📥 Commits

Reviewing files that changed from the base of the PR and between 3ee90b6 and af18ee5.

📒 Files selected for processing (2)
  • src/runtime/models/lance/MODEL.toml
  • tests/cpp/models/lance/test_lance_runtime_config_contract.cpp

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


📝 Summary

Summary

Adds a CPU-only Lance contract test for lance_parse_preprocess_config().

The test verifies:

  • Geometry and image normalization precedence.
  • Geometry reset behavior.
  • Preservation of config.json geometry when the override file is absent.
  • vl_prompt_template escaped-newline decoding.
  • resample fallback mappings.
  • Suppression of fallback mapping when config.json specifies interpolation.

The test compiles image_preprocessor.cpp directly and is registered in the Lance model configuration. No production source changes are included.

Architecture impact

  • Family-owned files: Lance model configuration and Lance-specific test code.
  • Changed shared surfaces: The test consumes the shared image_preprocessor.cpp implementation and the lance_parse_preprocess_config() boundary.
  • New dependency directions: The Lance test depends directly on the image preprocessing implementation and third_party/stb headers.
  • Affected consumers: CPU Lance preprocessing configuration coverage only.
  • Unresolved blast-radius questions: GPU, TensorRT, and image-processing runtime paths are not covered by this change.

Review status

PASS — CPU compilation, unit tests, formatting, model validation, repository tests, and mutation checks passed. The test detects the documented precedence, interpolation-guard, and geometry-reset mutations.

Walkthrough

Changes

Lance runtime configuration

Layer / File(s) Summary
Configuration contract test
tests/cpp/models/lance/test_lance_runtime_config_contract.cpp, src/runtime/models/lance/MODEL.toml
Adds a CPU-only executable test for Lance preprocessing configuration. The test validates cross-file precedence, prompt-template newline decoding, interpolation fallback, absent-key resets, and behavior without an override section. The model configuration registers the test with image_preprocessor.cpp and the third_party/stb include path.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to af18e

This adds CPU coverage for Lance preprocessing configuration precedence, fallback, reset, and prompt-template behavior without changing production behavior. No current merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 8 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 1 files. (1 skipped: 1… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (8 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the Lance runtime configuration boundary test, which is the main change.
Description check ✅ Passed The description covers the background, exit criteria, implementation, change category, validation commands and results, environment, remaining gaps, notes, and risk rationale. It provides sufficient d…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Family Ownership Boundary ✅ Passed PASS. The change adds only src/runtime/models/lance/MODEL.toml:9 and tests/cpp/models/lance/test_lance_runtime_config_contract.cpp. The test includes runtime/models/lance/image_preprocessor.h at…
Shared Semantic Neutrality ✅ Passed PASS. The pull request changes only src/runtime/models/lance/MODEL.toml and tests/cpp/models/lance/test_lance_runtime_config_contract.cpp. The first is a model-owned runtime test registration, and…
Benchmark Validation Integrity ✅ Passed PASS. The pull request adds one CPU runtime contract test and one MODEL.toml registration. It does not add or change benchmark timing, performance metrics, workload accounting, reference comparison,…
Shared Change Blast Radius ✅ Passed PASS: The pull request is family-local and does not alter a shared surface. The committed diff contains only src/runtime/models/lance/MODEL.toml and the new `tests/cpp/models/lance/test_lance_runtim…
Full details: Docstring Coverage

Explanation

Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 1 files. (1 skipped: 1 unsupported.)


Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant