Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ Known V1 error codes are `VALIDATION_ERROR`, `SCHEMA_VERSION_MISMATCH`, `MODEL_V

## Compatibility Policy

The SDK supports only `schema_version="v1"`. `validate_service_metadata()` checks `/healthz` metadata and raises typed compatibility errors before prediction if the service advertises a different schema, an unexpected model version, or mode capabilities outside the recorded V1 contract.
The SDK supports only `schema_version="v1"`. `validate_service_metadata()` checks `/healthz` metadata and raises typed compatibility errors before prediction if the service advertises a different schema, an unexpected model version, mode capabilities outside the recorded V1 contract, or an unsupported `decoding_strategy`.

Callers should pass an expected `model_version` when they already know which deployment artifact they intend to use. A mismatch is treated as a hard compatibility error rather than silently downgrading, guessing, or retrying another model.

Expand Down
7 changes: 4 additions & 3 deletions docs/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ This reference covers the supported public Python surface exported by `jointfm_c
| `DataFrameSchema` | Describes tabular history layout. Fields are `columns`, `time_index_mode`, `time_column`, `time_scale_seconds`, `use_local_normalized_time`, `calendar_id`, and `timezone`. |
| `ForecastRequestMetadata` | Holds `schema_version`, `model_version`, `query_mode`, and `return_mode` for one forecast request. |
| `ForecastRequest` | Validated request object that combines metadata, schema, history rows, query times, requested columns, sample or quantile controls, and `seed`, then emits a JSON-compatible payload with `to_payload()`. |
| `HealthMetadata` | Typed service-health payload with service status, schema and model versions, checkpoint metadata, device, head, advertised modes, time-index encoding, `max_sample_count`, and an optional `data_generation` block carrying advertised capacity limits. The container exposes it on `GET /healthz` for direct local access and as the response to `POST {"request_type": "health"}` on the unstructured prediction route for DataRobot-hosted deployments. |
| `HealthMetadata` | Typed service-health payload with service status, schema and model versions, checkpoint metadata, device, head, `decoding_strategy`, advertised modes, time-index encoding, `max_sample_count`, and an optional `data_generation` block carrying advertised capacity limits. The container exposes it on `GET /healthz` for direct local access and as the response to `POST {"request_type": "health"}` on the unstructured prediction route for DataRobot-hosted deployments. |
| `DataGenerationCapabilities` | Optional service-health block describing the deployed checkpoint's data-generation capacity. Fields are `sampler_type`, `min_features`, `max_features`, `min_targets`, `max_targets`, `t_input`, `t_output`, `n_input`, and `n_output`. |
| `ForecastPlan` | Validated forecast plan returned by `plan_forecast_columns`. Fields are `columns` (ordered `ColumnSpec` tuple), `feature_columns`, `target_columns` (both reflect post-downgrade roles), and `requested_columns` (the caller's original target list). |
| `StructuredError` | One structured JointFM service error with `code`, `message`, and optional `field`. |
Expand Down Expand Up @@ -73,7 +73,7 @@ All SDK-specific exceptions inherit from `JointFMError`.
| `JointFMCompatibilityError` | Base class for fail-fast service compatibility failures. |
| `UnsupportedSchemaVersionError` | The service or response advertises a schema version other than `v1`. |
| `UnsupportedModelVersionError` | The service or response model version differs from the configured or requested version. |
| `UnsupportedServiceContractError` | The service-health payload advertises mode capabilities outside the SDK's V1 contract. |
| `UnsupportedServiceContractError` | The service-health payload advertises mode capabilities or a `decoding_strategy` outside the recorded V1 contract. |

## Public Functions

Expand All @@ -96,7 +96,7 @@ All SDK-specific exceptions inherit from `JointFMError`.
| `build_local_predict_url(service_base_url)` | Build a direct local service `/predict` URL. |
| `build_datarobot_prediction_headers(api_token)` | Build hosted prediction headers: bearer authorization, broad accept header, and JSON content type. |
| `build_forecast_payload(...)` | Build a validated JSON-compatible V1 forecast payload from explicit schema, history rows, query times, and return-mode controls. |
| `validate_service_metadata(metadata, expected_model_version=None)` | Validate the service-health metadata against schema `v1`, the expected model when supplied, and advertised V1 mode capabilities. |
| `validate_service_metadata(metadata, expected_model_version=None)` | Validate the service-health metadata against schema `v1`, the expected model when supplied, advertised V1 mode capabilities, and a supported `decoding_strategy`. |
| `infer_column_specs_from_dataframe(frame, ...)` | Infer ordered `ColumnSpec` objects from a pandas `DataFrame` and explicit role, modality, mapping, nullability, time-value, and bounds hints. |
| `dataframe_to_history_rows(frame, schema)` | Convert a pandas `DataFrame` into server-compatible `history_rows`. |
| `arrays_to_history_rows(values, columns=..., ...)` | Convert a two-dimensional NumPy-like array plus column metadata into `history_rows`. |
Expand Down Expand Up @@ -216,6 +216,7 @@ The string literals are exposed as `PREDICT_REQUEST_TYPE`, `HEALTH_REQUEST_TYPE`
| `checkpoint_path` | Loaded checkpoint path reported by the service. |
| `device` | Device used by inference. |
| `head` | Active forecast head. |
| `decoding_strategy` | Horizon decoding mode advertised by the mounted model. Must be one of `SUPPORTED_DECODING_STRATEGIES`: `parallel_dense`, `parallel_scalable`, or `autoregressive`. Parallel strategies decode every horizon in one pass; `autoregressive` rolls horizons sequentially. |
| `supported_query_modes` | Must match the SDK V1 query modes. |
| `supported_return_modes` | Must match the SDK V1 return modes (`mean`, `samples`, `quantiles`, `log_prob`). |
| `supported_time_index_modes` | Must match the SDK V1 time-index modes. |
Expand Down
2 changes: 1 addition & 1 deletion notebooks/service_health.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
},
"source": [
"# Service Health\n",
"Fetch the typed `/healthz` payload from the configured JointFM deployment and display every field. Use this notebook to diagnose health-endpoint issues: version pins, advertised modes, sample-count budgets, and the optional `data_generation` capability block."
"Fetch the typed `/healthz` payload from the configured JointFM deployment and display every field. Use this notebook to diagnose health-endpoint issues: version pins, advertised modes, `decoding_strategy` (parallel vs autoregressive horizon decoding), sample-count budgets, and the optional `data_generation` capability block."
]
},
{
Expand Down
2 changes: 2 additions & 0 deletions src/jointfm_client/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@
SUPPORTED_COLUMN_MODALITIES,
SUPPORTED_COLUMN_ROLES,
STRUCTURED_ERROR_CODES,
SUPPORTED_DECODING_STRATEGIES,
SUPPORTED_QUERY_MODES,
SUPPORTED_REQUEST_TYPES,
SUPPORTED_RETURN_MODES,
Expand Down Expand Up @@ -206,6 +207,7 @@
"SUPPORTED_COLUMN_MODALITIES",
"SUPPORTED_COLUMN_ROLES",
"STRUCTURED_ERROR_CODES",
"SUPPORTED_DECODING_STRATEGIES",
"SUPPORTED_QUERY_MODES",
"SUPPORTED_RETURN_MODES",
"SUPPORTED_TIME_INDEX_MODES",
Expand Down
37 changes: 36 additions & 1 deletion src/jointfm_client/contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,11 @@
"continuous_float",
"absolute_datetime",
]
DecodingStrategy: TypeAlias = Literal[
"parallel_dense",
"parallel_scalable",
"autoregressive",
]
ColumnModality: TypeAlias = Literal[
"numeric",
"categorical",
Expand Down Expand Up @@ -92,6 +97,11 @@
"continuous_float",
"absolute_datetime",
)
SUPPORTED_DECODING_STRATEGIES: Final[tuple[DecodingStrategy, ...]] = (
"parallel_dense",
"parallel_scalable",
"autoregressive",
)
SUPPORTED_COLUMN_MODALITIES: Final[tuple[ColumnModality, ...]] = (
"numeric",
"categorical",
Expand Down Expand Up @@ -470,6 +480,7 @@ class HealthMetadata:
checkpoint_path: str
device: str
head: str
decoding_strategy: DecodingStrategy
supported_query_modes: tuple[str, ...]
supported_return_modes: tuple[str, ...]
supported_time_index_modes: tuple[str, ...]
Expand Down Expand Up @@ -512,6 +523,9 @@ def from_payload(cls, payload: Mapping[str, Any]) -> Self:
),
device=_require_string(payload.get("device"), field="device"),
head=_require_string(payload.get("head"), field="head"),
decoding_strategy=_require_decoding_strategy(
payload.get("decoding_strategy")
),
supported_query_modes=_string_tuple(
payload.get("supported_query_modes"),
field="supported_query_modes",
Expand Down Expand Up @@ -1088,7 +1102,12 @@ def validate_service_metadata(
*,
expected_model_version: str | None = None,
) -> None:
"""Validate `/healthz` metadata against the SDK's V1 compatibility policy."""
"""Validate `/healthz` metadata against the SDK's V1 compatibility policy.

Requires schema ``v1``, the expected model version when supplied, the
advertised V1 query/return/time-index modes, and a supported
``decoding_strategy``.
"""
schema_version = _required_string(metadata, "schema_version")
if schema_version != SCHEMA_VERSION:
raise UnsupportedSchemaVersionError(
Expand Down Expand Up @@ -1118,6 +1137,7 @@ def validate_service_metadata(
field="supported_time_index_modes",
supported_values=SUPPORTED_TIME_INDEX_MODES,
)
_require_decoding_strategy(metadata.get("decoding_strategy"))


def _required_string(metadata: Mapping[str, Any], field: str) -> str:
Expand All @@ -1130,6 +1150,21 @@ def _required_string(metadata: Mapping[str, Any], field: str) -> str:
return value


def _require_decoding_strategy(value: Any) -> DecodingStrategy:
"""Return a supported horizon decoding strategy from health metadata."""
if not isinstance(value, str) or value == "":
raise UnsupportedServiceContractError(
"JointFM health metadata field 'decoding_strategy' must be a "
"non-empty string"
)
if value not in SUPPORTED_DECODING_STRATEGIES:
raise UnsupportedServiceContractError(
"Unsupported JointFM decoding_strategy: expected one of "
f"{sorted(SUPPORTED_DECODING_STRATEGIES)!r}, got {value!r}"
)
return cast(DecodingStrategy, value)


def _require_exact_values(
metadata: Mapping[str, Any],
*,
Expand Down
1 change: 1 addition & 0 deletions tests/fixtures/health_metadata.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
"checkpoint_path": "/models/jointfm.pt",
"device": "cpu",
"head": "studentt",
"decoding_strategy": "parallel_dense",
"supported_query_modes": ["forecast"],
"supported_return_modes": ["mean", "samples", "quantiles", "log_prob"],
"supported_time_index_modes": [
Expand Down
1 change: 1 addition & 0 deletions tests/test_capabilities.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ def test_health_metadata_parses_data_generation_block(
health = HealthMetadata.from_payload(json_fixture_loader("health_metadata"))

assert health.max_sample_count == 4096
assert health.decoding_strategy == "parallel_dense"
assert health.data_generation == _capabilities()


Expand Down
2 changes: 2 additions & 0 deletions tests/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ def health(self) -> HealthMetadata:
checkpoint_path="/models/jointfm.pt",
device="cpu",
head="studentt",
decoding_strategy="parallel_dense",
supported_query_modes=("forecast",),
supported_return_modes=("mean", "samples", "quantiles", "log_prob"),
supported_time_index_modes=(
Expand Down Expand Up @@ -125,6 +126,7 @@ def test_health_command_prints_non_secret_metadata(monkeypatch, capsys) -> None:
payload = json.loads(output)
assert exit_code == 0
assert payload["service"]["status"] == "ok"
assert payload["service"]["decoding_strategy"] == "parallel_dense"
assert payload["deployment"]["deployment_id"] == "deployment-id"
assert "secret-token" not in output

Expand Down
1 change: 1 addition & 0 deletions tests/test_configuration.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ class _HealthTransport:
"checkpoint_path": "/models/jointfm.pt",
"device": "cpu",
"head": "studentt",
"decoding_strategy": "parallel_dense",
"supported_query_modes": ["forecast"],
"supported_return_modes": ["mean", "samples", "quantiles", "log_prob"],
"supported_time_index_modes": [
Expand Down
44 changes: 44 additions & 0 deletions tests/test_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@
SUPPORTED_COLUMN_MODALITIES,
SUPPORTED_COLUMN_ROLES,
STRUCTURED_ERROR_CODES,
SUPPORTED_DECODING_STRATEGIES,
SUPPORTED_QUERY_MODES,
SUPPORTED_RETURN_MODES,
SUPPORTED_TIME_INDEX_MODES,
Expand Down Expand Up @@ -78,6 +79,7 @@ def _health_metadata() -> dict[str, object]:
"checkpoint_path": "/models/jointfm.pt",
"device": "cpu",
"head": "dummy",
"decoding_strategy": "parallel_dense",
"supported_query_modes": ["forecast"],
"supported_return_modes": ["mean", "quantiles", "samples", "log_prob"],
"supported_time_index_modes": [
Expand Down Expand Up @@ -127,6 +129,11 @@ def test_mode_and_error_contract() -> None:
"continuous_float",
"absolute_datetime",
)
assert SUPPORTED_DECODING_STRATEGIES == (
"parallel_dense",
"parallel_scalable",
"autoregressive",
)
assert SUPPORTED_COLUMN_MODALITIES == (
"numeric",
"categorical",
Expand Down Expand Up @@ -778,6 +785,7 @@ def test_health_and_response_models_parse_current_payloads() -> None:
)

assert health.model_version == "jointfm-inference:0.2.0+ckpt.smoke-1"
assert health.decoding_strategy == "parallel_dense"
assert isinstance(response, MeanForecastResult)
assert response.requested_columns == ("target",)
assert response.mean == ((100.0,),)
Expand Down Expand Up @@ -1013,6 +1021,42 @@ def test_validate_service_metadata_rejects_unknown_advertised_mode() -> None:
validate_service_metadata(metadata)


def test_validate_service_metadata_rejects_unknown_decoding_strategy() -> None:
"""Validate service metadata rejects an unknown decoding strategy."""
metadata = _health_metadata()
metadata["decoding_strategy"] = "diffusion"

with pytest.raises(UnsupportedServiceContractError, match="decoding_strategy"):
validate_service_metadata(metadata)


def test_validate_service_metadata_rejects_missing_decoding_strategy() -> None:
"""Validate service metadata rejects a missing decoding strategy."""
metadata = _health_metadata()
del metadata["decoding_strategy"]

with pytest.raises(UnsupportedServiceContractError, match="decoding_strategy"):
validate_service_metadata(metadata)


def test_validate_service_metadata_rejects_empty_decoding_strategy() -> None:
"""Validate service metadata rejects an empty decoding strategy."""
metadata = _health_metadata()
metadata["decoding_strategy"] = ""

with pytest.raises(UnsupportedServiceContractError, match="decoding_strategy"):
validate_service_metadata(metadata)


def test_health_metadata_parses_each_supported_decoding_strategy() -> None:
"""Health metadata accepts every supported decoding strategy."""
for strategy in SUPPORTED_DECODING_STRATEGIES:
payload = _health_metadata()
payload["decoding_strategy"] = strategy
health = HealthMetadata.from_payload(payload)
assert health.decoding_strategy == strategy


def test_validate_service_metadata_rejects_malformed_capabilities() -> None:
"""Validate service metadata rejects malformed capabilities."""
metadata = _health_metadata()
Expand Down
1 change: 1 addition & 0 deletions tests/test_fixture_compatibility.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ def test_health_fixture_matches_current_v1_service_contract(
metadata = HealthMetadata.from_payload(payload)

assert metadata.schema_version == "v1"
assert metadata.decoding_strategy == "parallel_dense"
assert metadata.supported_return_modes == (
"mean",
"samples",
Expand Down
1 change: 1 addition & 0 deletions tests/test_transport.py
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ def _health_payload(
"checkpoint_path": "/models/jointfm.pt",
"device": "cpu",
"head": "studentt",
"decoding_strategy": "parallel_dense",
"supported_query_modes": ["forecast"],
"supported_return_modes": ["mean", "samples", "quantiles", "log_prob"],
"supported_time_index_modes": [
Expand Down
Loading