Skip to content

Nightly spec-drift: strict contract tests failing (since 2026-07-05) #467

Description

@github-actions

Nightly strict contract tests failed

The scheduled Spec Drift Detection run promoted additive drift to an error.
This means upstream OpenAPI/AsyncAPI changed and the SDK contract suite no longer matches.

To resolve: locally run uv run python scripts/sync_spec.py + uv run python scripts/generate.py, reconcile models/maps, and open a PR with Closes #<this issue>.

Failing test output (last 200 lines)

        Plain ``"Foo"`` looks up ``components.schemas.Foo``.
        Dotted ``"Foo.bar.items"`` walks ``components.schemas.Foo.properties.bar.items``
        where each segment after the first navigates ``properties[segment]`` unless the
        segment is the literal ``"items"`` (in which case it descends into the array
        items schema). Used to address inline object schemas that the spec doesn't
        name at the top level — e.g., the per-entry shapes inside ``Batch*Orders*``
        response wrappers.
        """
        schemas = spec.get("components", {}).get("schemas", {})
        head, *rest = schema_name.split(".")
        node = schemas.get(head)
        if node is None:
>           pytest.fail(f"Schema '{head}' not found in OpenAPI spec")
E           Failed: Schema 'Announcement' not found in OpenAPI spec

tests/test_contracts.py:316: Failed
_______________ TestSpecDrift.test_required_drift[Announcement] ________________

self = <tests.test_contracts.TestSpecDrift object at 0x7f7e57facdd0>
entry = ContractEntry(sdk_model='kalshi.models.exchange.Announcement', spec_schema='Announcement', ignored_fields=frozenset(), notes='')

    @pytest.mark.parametrize(
        "entry",
        CONTRACT_MAP,
        ids=[e.sdk_model.rsplit(".", 1)[1] for e in CONTRACT_MAP],
    )
    def test_required_drift(self, entry: ContractEntry) -> None:
        """Fail when SDK fields are optional but spec marks them required.
    
        Hard-fail since #172. Was warn-only while the SDK kept ~134 fields
        Optional[T] | None against spec's required: true. The deviation is
        either resolved (drop None default) or recorded in EXCLUSIONS with
        kind='server_omits_despite_required' citing a demo+prod observation.
        """
>       spec_fields = _get_schema_fields(self.spec, entry.spec_schema)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

tests/test_contracts.py:783: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_contracts.py:347: in _get_schema_fields
    schema = _resolve_schema(spec, schema_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

spec = {'openapi': '3.0.0', 'info': {'title': 'Kalshi Trade API Manual Endpoints', 'version': '3.23.0', 'description': 'Manua...endpoints was last validated: GetBalance, GetOrder(s), GetFills, GetPositions', 'tags': ['exchange'], ...}}, ...}, ...}
schema_name = 'Announcement'

    def _resolve_schema(spec: dict[str, Any], schema_name: str) -> dict[str, Any]:
        """Resolve a schema by name, supporting dotted-path syntax for inline schemas.
    
        Plain ``"Foo"`` looks up ``components.schemas.Foo``.
        Dotted ``"Foo.bar.items"`` walks ``components.schemas.Foo.properties.bar.items``
        where each segment after the first navigates ``properties[segment]`` unless the
        segment is the literal ``"items"`` (in which case it descends into the array
        items schema). Used to address inline object schemas that the spec doesn't
        name at the top level — e.g., the per-entry shapes inside ``Batch*Orders*``
        response wrappers.
        """
        schemas = spec.get("components", {}).get("schemas", {})
        head, *rest = schema_name.split(".")
        node = schemas.get(head)
        if node is None:
>           pytest.fail(f"Schema '{head}' not found in OpenAPI spec")
E           Failed: Schema 'Announcement' not found in OpenAPI spec

tests/test_contracts.py:316: Failed
______________________ TestSpecDrift.test_schema_coverage ______________________

self = <tests.test_contracts.TestSpecDrift object at 0x7f7e57e353d0>

    def test_schema_coverage(self) -> None:
        """Every mapped schema must resolve (supports dotted-path syntax)."""
        for entry in CONTRACT_MAP:
            # _resolve_schema fails the test internally if the path doesn't resolve.
>           _resolve_schema(self.spec, entry.spec_schema)

tests/test_contracts.py:796: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

spec = {'openapi': '3.0.0', 'info': {'title': 'Kalshi Trade API Manual Endpoints', 'version': '3.23.0', 'description': 'Manua...endpoints was last validated: GetBalance, GetOrder(s), GetFills, GetPositions', 'tags': ['exchange'], ...}}, ...}, ...}
schema_name = 'Announcement'

    def _resolve_schema(spec: dict[str, Any], schema_name: str) -> dict[str, Any]:
        """Resolve a schema by name, supporting dotted-path syntax for inline schemas.
    
        Plain ``"Foo"`` looks up ``components.schemas.Foo``.
        Dotted ``"Foo.bar.items"`` walks ``components.schemas.Foo.properties.bar.items``
        where each segment after the first navigates ``properties[segment]`` unless the
        segment is the literal ``"items"`` (in which case it descends into the array
        items schema). Used to address inline object schemas that the spec doesn't
        name at the top level — e.g., the per-entry shapes inside ``Batch*Orders*``
        response wrappers.
        """
        schemas = spec.get("components", {}).get("schemas", {})
        head, *rest = schema_name.split(".")
        node = schemas.get(head)
        if node is None:
>           pytest.fail(f"Schema '{head}' not found in OpenAPI spec")
E           Failed: Schema 'Announcement' not found in OpenAPI spec

tests/test_contracts.py:316: Failed
_______ TestRequestParamDrift.test_sync_params_match_spec[announcements] _______

self = <tests.test_contracts.TestRequestParamDrift object at 0x7f7e57f67740>
entry = MethodEndpointEntry(sdk_method='kalshi.resources.exchange.ExchangeResource.announcements', http_method='GET', path_template='/exchange/announcements', request_body_schema=None)

    def test_sync_params_match_spec(self, entry: MethodEndpointEntry) -> None:
>       self._assert_params_match(entry, async_=False)

tests/test_contracts.py:1121: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_contracts.py:1148: in _assert_params_match
    spec_params_list = _resolve_path_params(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

spec = {'openapi': '3.0.0', 'info': {'title': 'Kalshi Trade API Manual Endpoints', 'version': '3.23.0', 'description': 'Manua...endpoints was last validated: GetBalance, GetOrder(s), GetFills, GetPositions', 'tags': ['exchange'], ...}}, ...}, ...}
path_template = '/exchange/announcements', http_method = 'GET'
max_ref_depth = 8

    def _resolve_path_params(
        spec: dict[str, Any],
        path_template: str,
        http_method: str,
        *,
        max_ref_depth: int = 8,
    ) -> list[dict[str, Any]]:
        """Return the merged list of parameter objects for a specific operation.
    
        Walks BOTH path-level ``parameters`` (shared across operations on that path)
        AND operation-level ``parameters`` (specific to the given HTTP method).
        Operation-level entries override path-level by the ``(name, in)`` key —
        OpenAPI permits the same parameter name in different locations (e.g., a
        path param and a query param both named ``ticker``), so dedup must consider
        both dimensions.
        Any ``$ref`` entries are resolved via ``_resolve_ref``.
    
        Raises ``KeyError`` if the path does not exist in the spec, or if the
        operation (HTTP method) does not exist on that path. Fail loud: a
        mismatched ``METHOD_ENDPOINT_MAP`` entry should surface at test time, not
        silently return partial data.
        """
        paths = spec.get("paths", {})
        if path_template not in paths:
>           raise KeyError(f"path {path_template!r} not found in spec")
E           KeyError: "path '/exchange/announcements' not found in spec"

tests/_contract_support.py:1736: KeyError
______ TestRequestParamDrift.test_async_params_match_spec[announcements] _______

self = <tests.test_contracts.TestRequestParamDrift object at 0x7f7e57e8fe90>
entry = MethodEndpointEntry(sdk_method='kalshi.resources.exchange.ExchangeResource.announcements', http_method='GET', path_template='/exchange/announcements', request_body_schema=None)

    def test_async_params_match_spec(self, entry: MethodEndpointEntry) -> None:
>       self._assert_params_match(entry, async_=True)

tests/test_contracts.py:1124: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_contracts.py:1148: in _assert_params_match
    spec_params_list = _resolve_path_params(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

spec = {'openapi': '3.0.0', 'info': {'title': 'Kalshi Trade API Manual Endpoints', 'version': '3.23.0', 'description': 'Manua...endpoints was last validated: GetBalance, GetOrder(s), GetFills, GetPositions', 'tags': ['exchange'], ...}}, ...}, ...}
path_template = '/exchange/announcements', http_method = 'GET'
max_ref_depth = 8

    def _resolve_path_params(
        spec: dict[str, Any],
        path_template: str,
        http_method: str,
        *,
        max_ref_depth: int = 8,
    ) -> list[dict[str, Any]]:
        """Return the merged list of parameter objects for a specific operation.
    
        Walks BOTH path-level ``parameters`` (shared across operations on that path)
        AND operation-level ``parameters`` (specific to the given HTTP method).
        Operation-level entries override path-level by the ``(name, in)`` key —
        OpenAPI permits the same parameter name in different locations (e.g., a
        path param and a query param both named ``ticker``), so dedup must consider
        both dimensions.
        Any ``$ref`` entries are resolved via ``_resolve_ref``.
    
        Raises ``KeyError`` if the path does not exist in the spec, or if the
        operation (HTTP method) does not exist on that path. Fail loud: a
        mismatched ``METHOD_ENDPOINT_MAP`` entry should surface at test time, not
        silently return partial data.
        """
        paths = spec.get("paths", {})
        if path_template not in paths:
>           raise KeyError(f"path {path_template!r} not found in spec")
E           KeyError: "path '/exchange/announcements' not found in spec"

tests/_contract_support.py:1736: KeyError
=========================== short test summary info ============================
FAILED tests/test_contracts.py::TestSpecDrift::test_additive_drift[Announcement] - Failed: Schema 'Announcement' not found in OpenAPI spec
FAILED tests/test_contracts.py::TestSpecDrift::test_required_drift[Announcement] - Failed: Schema 'Announcement' not found in OpenAPI spec
FAILED tests/test_contracts.py::TestSpecDrift::test_schema_coverage - Failed: Schema 'Announcement' not found in OpenAPI spec
FAILED tests/test_contracts.py::TestRequestParamDrift::test_sync_params_match_spec[announcements] - KeyError: "path '/exchange/announcements' not found in spec"
FAILED tests/test_contracts.py::TestRequestParamDrift::test_async_params_match_spec[announcements] - KeyError: "path '/exchange/announcements' not found in spec"
================== 5 failed, 649 passed in 243.69s (0:04:03) ===================

Metadata

Metadata

Assignees

No one assigned

    Labels

    spec-driftUpstream OpenAPI/AsyncAPI spec changed since last sync

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions