Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 29 additions & 2 deletions docs/user_guides/fs/feature_view/feature_logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,19 +215,46 @@ materialization_result = feature_view.materialize_log(wait=True)
## Deleting Logs

When log data is no longer needed, you might want to delete it to free up space and maintain data hygiene.
This operation deletes the feature groups and recreates new ones.
This operation deletes the logging feature group and recreates a new one.
Scheduled materialization job and log timeline are reset as well.

### Delete Logs

Remove all log entries.
The `transformed` selector applies only to older feature views with separate logging groups.
On a feature view that still has the pre-4.6 pair of logging feature groups, `delete_log()` deletes both and recreates the log in the combined layout; passing `transformed=True` or `transformed=False` does the same, because the pair can only be replaced as a whole.
On the combined layout, `delete_log(transformed=True)` has nothing to delete and does nothing.

```python
# Delete all log entries
feature_view.delete_log()
```

## Upgrade Compatibility with Pre-4.6 Feature Logging

Hopsworks 4.6 changed the feature logging layout: transformed and untransformed features are logged into one combined feature group instead of a separate pair, labels are logged as `predicted_<label>` columns, and the model identity is stored in `model_name` and `model_version` columns instead of a single `hsml_model` column.

Feature views that enabled logging before the upgrade keep their original pair of logging feature groups unchanged.
Model deployments and batch jobs that still run a pre-4.6 client keep logging to those feature views without any code change or downtime.
Clients from 4.6 onwards also keep working against them: predictions and the model identity are written into the original columns, and `feature_view.read_log(model_name=..., model_version=...)` filters on the original `hsml_model` column.
Calling `feature_view.delete_log()` on such a feature view deletes the original pair and recreates the logs in the combined layout, because the deleted logs are recreated with the current schema.
A deployment that still runs a pre-4.6 client cannot write to the recreated group and its logs are lost, so update the deployment's environment to a 4.6 or later client before deleting the log.

Enabling logging on a feature view created after the upgrade requires a 4.6 or later client.
A pre-4.6 client cannot produce the combined layout, so its `feature_view.log(...)` calls against such feature views fail instead of writing incomplete rows.

The first positional parameter of `feature_view.log()` changed in 4.6 from `untransformed_features` to `logging_data`.
On feature views with the pre-4.6 pair of logging feature groups, positional calls written for the old signature are detected and keep working.

```python
# Positional call style written for pre-4.6 clients, still working on
# feature views that predate the upgrade:
feature_view.log(features, predictions)

# Equivalent call that works on every feature view; use this form when
# migrating code to a 4.6 or later client:
feature_view.log(features, predictions=predictions)
```

## Summary

Feature logging is a crucial part of maintaining and monitoring your machine learning workflows.
Expand Down
Loading