Skip to content

build(deps): make mlflow and wandb optional extras - #3785

Open
piyushumate wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
piyushumate:piyushumate/build/mlflow-wandb-optional-extras
Open

build(deps): make mlflow and wandb optional extras#3785
piyushumate wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
piyushumate:piyushumate/build/mlflow-wandb-optional-extras

Conversation

@piyushumate

Copy link
Copy Markdown

What does this PR do?

Make mlflow and wandb optional extras instead of hard core dependencies. Closes #3783.

Changelog

  • pyproject.toml: remove mlflow and wandb>=0.28.0 from dependencies[]; add [mlflow] (→ mlflow-skinny), [mlflow-full] (→ full mlflow), [wandb], and [tracking] (meta-extra) in [project.optional-dependencies]; wire [tracking] into [all]; fix all composite extras to use canonical distribution name nemo-automodel (hyphens)
  • nemo_automodel/shared/import_utils.py (existing): safe_import / safe_import_from now used for all optional-dep guards per CLAUDE.md coding style rule
  • components/loggers/loggers.py: module-level safe_import for wandb / mlflow; remove inline try/except ImportError from WandbConfig.build() and MLflowConfig.build()
  • components/loggers/wandb_utils.py: remove inline try/except in init_wandb_run; defer error to safe_import placeholder at call time; uv-based install hint
  • 8 recipe files (llm/train_ft, llm/kd, llm/train_seq_cls, dllm/train_ft, diffusion/train, vlm/finetune, vlm/kd, multimodal/finetune): replace top-level import wandb / import mlflow with module-level safe_import calls (returns boolean flag + module-or-placeholder); prefix every wandb.run is not None guard with _HAS_WANDB and and every mlflow.active_run() is not None with _HAS_MLFLOW and to short-circuit safely without installing the extras
  • tests/unit_tests/loggers/test_optional_logger_extras.py [NEW]: unit tests covering instantiation without extras, UnavailableError on build() with absent extras, pyproject.toml extras schema, and recipe module _HAS_* sentinel presence
  • ruff format + ruff check --fix applied to all changed files

Install patterns

uv add nemo-automodel                   # lean — no logger overhead
uv add nemo-automodel[mlflow]           # MLflow skinny (no mlflow vs mlflow-skinny conflict)
uv add nemo-automodel[mlflow-full]      # full MLflow stack (UI, SQL, pyarrow)
uv add nemo-automodel[wandb]            # W&B only
uv add nemo-automodel[tracking]         # both backends
uv add nemo-automodel[all]              # everything (includes tracking)

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests? — yes: tests/unit_tests/loggers/test_optional_logger_extras.py
  • Did you add or update any necessary documentation?

Additional Information

@piyushumate
piyushumate requested review from a team as code owners September 2, 2026 02:56
@copy-pr-bot

copy-pr-bot Bot commented Sep 2, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@piyushumate
piyushumate force-pushed the piyushumate/build/mlflow-wandb-optional-extras branch from 8cb3f0d to d10f565 Compare September 2, 2026 02:58
@piyushumate

piyushumate commented Sep 2, 2026

Copy link
Copy Markdown
Author

Hi @akoumpa @HuiyingLi — could you approve the CI workflows for this community PR when you get a chance?

This removes mlflow and wandb from hard core dependencies and moves them to optional extras ([mlflow], [mlflow-full], [wandb], [tracking]), fixing the mlflow vs mlflow-skinny mutual-exclusion conflict that breaks installs in SageMaker environments.

Closes #3783. Happy to address any review feedback. Thanks!

@akoumpa

akoumpa commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

/ok to test d10f565

@piyushumate

piyushumate commented Sep 2, 2026

Copy link
Copy Markdown
Author

@akoumpa Updated both lockfiles (uv.lock), resolved ty type checking rules, and verified that all linters, formatting, and unit tests pass locally. Just curious, Is there a way i can trigger or CI has to go through maintainers?

@piyushumate
piyushumate force-pushed the piyushumate/build/mlflow-wandb-optional-extras branch from 5ac9562 to feeb7ab Compare September 2, 2026 06:22
@piyushumate
piyushumate force-pushed the piyushumate/build/mlflow-wandb-optional-extras branch from e6eab03 to a2f9e11 Compare September 2, 2026 06:41
@akoumpa

akoumpa commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

/ok to test a2f9e11

@piyushumate

Copy link
Copy Markdown
Author

@akoumpa is the CI expected to take this much time? It seems to be stuck in queue. Do lemme know if I am missing anything

@akoumpa

akoumpa commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Hi @piyushumate apologies for the delay; the CI workers are oversubscribed, thus the delay :( I'll mention this to our automation folks to explore any potential solution -- in the meantime, i'll open a separate PR to run CI to leverage separate CI worker queue, if that passes, i'll FM your PR.

@piyushumate

piyushumate commented Sep 2, 2026

Copy link
Copy Markdown
Author

Hi @akoumpa — updated commit 8ae1f40 to disable W&B interactive prompts (resolving the CI CPU/GPU timeouts) and use mlflow-skinny compatible file storage in unit tests

@piyushumate

Copy link
Copy Markdown
Author

@akoumpa CI passed, please lemme know if anything else is needed. Thanks!

@akoumpa

akoumpa commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Hi @piyushumate can you take a look at the conflicts? i can rerun ci then

mlflow and wandb are experiment-tracking backends that are opt-in via YAML
config. Listing them as hard core dependencies causes two problems:

1. mlflow (full) conflicts with mlflow-skinny — mutually exclusive PyPI
   packages. Environments shipping mlflow-skinny (sagemaker-mlflow,
   kedro-mlflow, managed ML platforms) hard-block on install.
2. Both packages pull in heavy transitive deps unconditionally even for users
   who never configure a logger.

Changes:
- pyproject.toml: remove wandb and mlflow from dependencies[]; add:
    [mlflow]      = mlflow-skinny  (no conflict, full tracking API)
    [mlflow-full] = mlflow          (UI, SQL backend, pyarrow, etc.)
    [wandb]       = wandb>=0.28.0
    [tracking]    = nemo-automodel[mlflow] + nemo-automodel[wandb]
  Wire [tracking] into [all]. Fix composite extras to use canonical
  distribution name nemo-automodel (hyphens, not nemo_automodel underscores).

- loggers.py / wandb_utils.py: use safe_import() / safe_import_from() from
  nemo_automodel.shared.import_utils per CLAUDE.md coding-style rule.
  Remove inline try/except ImportError guards.

- 8 recipe files (llm/train_ft, llm/kd, llm/train_seq_cls, dllm/train_ft,
  diffusion/train, vlm/finetune, vlm/kd, multimodal/finetune): replace
  top-level import wandb / import mlflow with module-level safe_import calls
  (returns _HAS_WANDB/_HAS_MLFLOW boolean + module-or-placeholder).
  Prefix every wandb.run is not None guard with _HAS_WANDB and, and every
  mlflow.active_run() is not None with _HAS_MLFLOW and.

- tests/unit_tests/loggers/test_optional_logger_extras.py [NEW]: 13 unit
  tests covering instantiation without extras, UnavailableError on build()
  with absent extras, pyproject.toml extras schema, and recipe module
  _HAS_* sentinel presence.

- ruff format + ruff check --fix applied to all changed files.

Closes NVIDIA-NeMo#3783

Precedent: sagemaker-mlflow PR NVIDIA-NeMo#22 applied the identical fix.

Signed-off-by: piyushumate <piyushumate@users.noreply.github.com>
@piyushumate

piyushumate commented Sep 5, 2026

Copy link
Copy Markdown
Author

@akoumpa, conflicts have been resolved and the branch is rebased onto latest main. Both lockfiles have been regenerated and verified locally, and all linters and tests pass. Ready for CI. Thanks!

@akoumpa

akoumpa commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

/ok to test 080ad40

@piyushumate

piyushumate commented Sep 5, 2026

Copy link
Copy Markdown
Author

@akoumpa seems like CI oversubscribed? (nvm , it passed)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[packaging] Make mlflow and wandb optional extras (not hard deps); switch to mlflow-skinny by default

3 participants