diff --git a/.ci/cspell_dict.txt b/.ci/cspell_dict.txt index c8f5d99180b..52fcf62333a 100644 --- a/.ci/cspell_dict.txt +++ b/.ci/cspell_dict.txt @@ -134,6 +134,7 @@ eprinttype errstate evals execu +executorch extendleft facecolors fakequantizer @@ -314,6 +315,7 @@ nvfp objwalk odict oimp +onednn onnxhw onnxp onnxqdq @@ -403,6 +405,7 @@ reducel reducemax reducemean reducesum +regen relu reportviews requant @@ -449,6 +452,7 @@ stabilityai_stablelm stdv stfpm stridedslice +styleguide subarray subdict submodel @@ -476,6 +480,7 @@ timestep tinyyolov tobytes togethercomputer +torchao torchdynamo truncnorm tupled diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index ba8813f0668..2f636c36ca1 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -3,57 +3,8 @@ Act as a Senior Staff Engineer reviewing code for the Neural Network Compression Framework (NNCF). Be concise, critical, and provide code examples for fixes. -## Python Code Style Guidelines - -### Language Rules - -- Use nested functions/classes only when they make code more readable and simpler, or when closing over local variables. -- Code must be annotated with type hints (PEP-484) -- Use `pathlib.Path` instead of `os.*` methods -- Avoid using `assert` statements in `src/nncf`, raise proper exceptions with informative messages instead - -### Style Rules - -**Format of docstrings**: - -```python -def foo(arg1: int, arg2: bool = False) -> int: - """ - Brief description of what the function does. - - :param arg1: Description - :param arg2: Description - :return: Description - """ -``` - -- All externally visible functions need docstrings unless very short, obvious, and not externally visible - -**Naming convention** - -- Modules/Packages: `lower_with_under` -- Classes: `CapWords` -- Functions/Methods: `lower_with_under()` -- Constants: `CAPS_WITH_UNDER` -- Variables: `lower_with_under` -- Internal/Private: prefix with `_` -- Avoid single char names (except `i j k v` for loops, `e` for exceptions, `f` for files, `a b x y z` for tensors) -- Avoid dashes in package/module names -- Avoid type suffixes in names (e.g., `id_to_name_dict`) - -**File naming** - -- Use `.py` extension, never dashes -- Avoid generic names like `utils.py` or `helpers.py` -- Group related code in dedicated, explicitly-named modules - -### Test Suite Rules - -**Basic Style** - -- Use pytest framework -- Type-hint all parameters and fixtures -- Use fixtures, not xunit-style setup/teardown +Code style, architecture, and test rules: [AGENTS.md](../AGENTS.md) and +[docs/styleguide/PyGuide.md](../docs/styleguide/PyGuide.md). ### Review Focus Areas diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000000..80f72e4949b --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,110 @@ +# AGENTS.md + +Guidance for AI coding agents working in this repository. + +## Project Overview + +Neural Network Compression Framework (NNCF) provides post-training and training-time compression algorithms +(quantization, weight compression, pruning, QAT) for OpenVINO, PyTorch, TorchFX, and ONNX models. +Public API entry points: `nncf.quantize`, `nncf.compress_weights`, `nncf.Dataset`. + +## Architecture + +Backend-agnostic core built around `NNCFGraph`. Details: [docs/NNCFArchitecture.md](docs/NNCFArchitecture.md). + +```text +src/nncf/ +├── common/ # Backend-agnostic logic: graph, quantization, statistics +├── quantization/algorithms/ # Algorithm logic + _backend.py implementations +├── quantization/quantize_model.py # Public API, dispatches to backend *_impl functions +├── tensor/ # Backend-agnostic tensor and functions +├── openvino/ onnx/ torch/ # Backend-specific code (see AGENTS.md in each folder) +└── experimental/torch/fx/ # TorchFX backend (experimental) +``` + +Folder-specific guidance: +[Common](src/nncf/common/AGENTS.md), +[OpenVINO](src/nncf/openvino/AGENTS.md), +[PyTorch](src/nncf/torch/AGENTS.md), +[ONNX](src/nncf/onnx/AGENTS.md), +[TorchFX](src/nncf/experimental/torch/fx/AGENTS.md). + +Rules: + +- Common algorithm logic goes in `algorithm.py`; backend interface in `backend.py`; backend specifics in + `openvino_backend.py`, `onnx_backend.py`, `torch_backend.py`, `torch_fx_backend.py` + (see [min_max](src/nncf/quantization/algorithms/min_max) as a reference). +- Do not import backend frameworks (`openvino`, `torch`, `onnx`) at module level in `src/nncf/common` or in + algorithm common code; backend dispatch uses local imports (see [factory.py](src/nncf/common/factory.py)). +- Use `nncf.tensor` functions instead of raw numpy/torch calls in backend-agnostic code. +- A change in common code affects all backends: run tests for every supported backend. + +## Build and Test + +Each test target needs the matching install target first. `install-` installs the test requirements +and, for most targets, NNCF in editable mode (`uv pip install -e .`); see [Makefile](Makefile). + +```bash +make install- # common | openvino | torch | onnx | tools | docs | examples | conformance +make test- # common | openvino | torch | onnx | tools | docs | examples +make install-pre-commit +``` + +Backend-specific test targets: + +```bash +make test-openvino # runs with ONEDNN_MAX_CPU_ISA=AVX2 OV_CPU_MAX_ISA=AVX2 +make test-torch-cpu # excludes `cuda` and `long` markers +make test-torch-gpu # `cuda` marker only +make test-ptq # conformance, requires install-conformance +NUM_WORKERS=4 make test-torch-cpu ARGS="-k test_name" + +python -m pytest tests//path/test_file.py::test_name +``` + +- Regenerate reference data (graphs, JSON refs): `pytest tests//... --regen-ref-data`. Review the diff. +- Shared test logic lives in [tests/cross_fw/test_templates](tests/cross_fw/test_templates); + backend tests subclass these templates. +- Lint and format: `make pre-commit` (ruff, markdownlint, cspell, actionlint). +- Type check: `mypy` (strict, `src/nncf`; excluded files are listed in [pyproject.toml](pyproject.toml)). + +## Code Style + +Full guide: [docs/styleguide/PyGuide.md](docs/styleguide/PyGuide.md). + +- Type hints (PEP-484) for all functions, including tests and fixtures. +- No `assert` in `src/nncf`; raise exceptions with informative messages (see `nncf.errors`). +- Use `pathlib.Path` instead of `os.path`. +- Naming: `lower_with_under` for modules/functions/variables, `CapWords` for classes, `CAPS_WITH_UNDER` for + constants, `_` prefix for private. Avoid type suffixes in names (`id_to_name_dict`) and generic module names + (`utils.py`, `helpers.py`). Avoid single-char names except `i j k v` (loops), `e` (exceptions), `f` (files), + `a b x y z` (tensors). +- Use nested functions/classes only when they simplify code or close over local variables. +- Externally visible functions need docstrings unless very short and obvious. +- Every `.py` file starts with the Intel Apache-2.0 copyright header (ruff `CPY001`); copy it from any existing file. +- Docstring format: + +```python +def foo(arg1: int, arg2: bool = False) -> int: + """ + Brief description of what the function does. + + :param arg1: Description + :param arg2: Description + :return: Description + """ +``` + +## Tests + +- pytest only; use fixtures, not xunit-style setup/teardown. +- Place tests in `tests//` with `test_*` file names; test data in `tests//data/`. +- Markers are defined in `tests//pytest.ini` (e.g. `cuda`, `long` for torch). + +## Related Docs + +- [docs/usage](docs/usage) - algorithm documentation; [examples](examples) - per-backend examples +- [CONTRIBUTING.md](CONTRIBUTING.md) - contribution workflow +- [AI_POLICY.md](AI_POLICY.md) - rules for AI-assisted contributions +- [.github/copilot-instructions.md](.github/copilot-instructions.md) - code review guidelines +- [.github/skills](.github/skills) - task-specific agent skills (release notes, dependency checks, workflows) diff --git a/src/nncf/common/AGENTS.md b/src/nncf/common/AGENTS.md new file mode 100644 index 00000000000..5a34327bfd3 --- /dev/null +++ b/src/nncf/common/AGENTS.md @@ -0,0 +1,33 @@ +# AGENTS.md - Common (backend-agnostic) code + +Shared core used by all backends. General rules: [root AGENTS.md](../../../AGENTS.md). + +## Key Modules + +- `graph/graph.py` - `NNCFGraph`; `graph/layer_attributes.py` - `BaseLayerAttributes` and subclasses +- `graph/operator_metatypes.py` - `OperatorMetatype`, `OperatorMetatypeRegistry` (base for backend metatypes) +- `graph/model_transformer.py` - `ModelTransformer` base +- `graph/transformations/` - `TargetPoint`, `TargetType`, `TransformationCommand`, `TransformationLayout` +- `graph/patterns/` - fusing/ignored patterns, `PatternsManager` +- `factory.py` - backend dispatch: `build_graph`, `ModelTransformerFactory`, `EngineFactory`, + `CommandCreatorFactory`, `StatisticsAggregatorFactory` +- `utils/backend.py` - `BackendType`, `get_backend`, `is__model` predicates +- `tensor_statistics/` - `TensorCollector`, reducers/aggregators, `StatisticPoint`, `StatisticPointsContainer`, + `StatisticsAggregator` base +- `quantization/` - quantizer setup, propagation, configs (`structs.py`, `quantizer_propagation/`) +- `hardware/` - `HWConfig`, per-device setups in `setups/` (CPU, GPU, NPU) +- `logging/logger.py` - `nncf_logger`; `deprecation.py` - `deprecated`, `warning_deprecated` +- `utils/api_marker.py` - `@api` decorator for public API + +## Rules + +- No module-level imports of `torch`, `openvino`, `onnx`; import them locally inside backend-dispatch branches. +- Backend logic goes into backend folders; add a new backend branch to every factory in `factory.py`. +- Use `nncf.tensor` (`Tensor`, `nncf.tensor.functions`) for tensor math, not numpy/torch directly. +- Log via `nncf_logger`, not `print`; deprecate public API with `deprecation.py` helpers. +- Changes here affect every backend: run `make test-common` and the tests of all backends. + +## Tests + +- `tests/common/` (no backend frameworks required): `make install-common`, `make test-common`. +- Reference data: `tests/common/data/`; regenerate with `pytest tests/common/... --regen-ref-data`. diff --git a/src/nncf/experimental/torch/fx/AGENTS.md b/src/nncf/experimental/torch/fx/AGENTS.md new file mode 100644 index 00000000000..848c2c25d86 --- /dev/null +++ b/src/nncf/experimental/torch/fx/AGENTS.md @@ -0,0 +1,25 @@ +# AGENTS.md - TorchFX backend (experimental) + +Backend for `torch.fx.GraphModule` produced by `torch.export`. General rules: [root AGENTS.md](../../../../../AGENTS.md). + +## Key Modules + +- `nncf_graph_builder.py` - `GraphConverter`: `GraphModule` -> `NNCFGraph` +- `model_transformer.py` - `FXModelTransformer`; `commands.py` - `FXApplyTransformationCommand`, `FXModelExtractionCommand` +- `transformations.py` - graph-level FX transformations used by commands +- `quantization/quantize_model.py` - `quantize_impl`, `compress_weights_impl` (used by `nncf.quantize`) +- `quantization/quantize_pt2e.py` - `quantize_pt2e` for torchao/PT2E quantizers; `quantization/quantizer/` - quantizer adapters +- `constant_folding.py` - constant folding pass + +## Gotchas + +- Operator metatypes are shared with PyTorch: `src/nncf/torch/graph/operator_metatypes.py`. +- Input must be an exported `GraphModule` (`torch.export.export(...).module()`), not an eager `nn.Module`. +- `quantize_impl` deep-copies the model and rebuilds `GraphModule` after transformations; keep `model.meta` intact. +- Not all algorithms are supported; check the `*_impl` functions for rejected parameters before adding features. + +## Tests + +- `tests/torch/fx/` (part of `make test-torch-cpu`); reference data: `tests/torch/data/fx/`. +- ExecuTorch/quantizer integration: `tests/executorch/`. +- Regenerate references: `pytest tests/torch/fx/... --regen-ref-data`. diff --git a/src/nncf/onnx/AGENTS.md b/src/nncf/onnx/AGENTS.md new file mode 100644 index 00000000000..61b22ee2f74 --- /dev/null +++ b/src/nncf/onnx/AGENTS.md @@ -0,0 +1,23 @@ +# AGENTS.md - ONNX backend + +Backend for `onnx.ModelProto`. General rules: [root AGENTS.md](../../../AGENTS.md). + +## Key Modules + +- `graph/nncf_graph_builder.py` - `GraphConverter`: `onnx.ModelProto` -> `NNCFGraph`, `ONNXLayerAttributes` +- `graph/metatypes/onnx_metatypes.py` - `ONNX_OPERATION_METATYPES`, `ONNXOpMetatype`, `ONNXOpWithWeightsMetatype` +- `graph/model_transformer.py` - `ONNXModelTransformer` (inserts QuantizeLinear/DequantizeLinear pairs) +- `graph/transformations/commands.py` - `ONNXTargetPoint`, `ONNXInsertionCommand` and other commands +- `quantization/quantize_model.py` - `quantize_impl`, `quantize_with_accuracy_control_impl`, `compress_weights_impl` +- `engine.py` - `ONNXEngine` (onnxruntime) for inference during calibration + +## Gotchas + +- Models are protobuf objects; `ONNXModelTransformer` works on a `deepcopy` unless `inplace=True`. +- Weight/bias port detection depends on metatype definitions; check `graph/metatypes/groups.py` when adding ops. +- Models over 2GB need external data; see `check_model_protobuf_size` / `check_external_data_location`. + +## Tests + +- `tests/onnx/`, run with `make test-onnx`. +- Reference graphs: `tests/onnx/data/reference_graphs/`; regenerate with `pytest tests/onnx/... --regen-ref-data`. diff --git a/src/nncf/openvino/AGENTS.md b/src/nncf/openvino/AGENTS.md new file mode 100644 index 00000000000..ec7b8ac54fe --- /dev/null +++ b/src/nncf/openvino/AGENTS.md @@ -0,0 +1,26 @@ +# AGENTS.md - OpenVINO backend + +Backend for `openvino.Model`. General rules: [root AGENTS.md](../../../AGENTS.md). + +## Key Modules + +- `graph/nncf_graph_builder.py` - `GraphConverter`: `ov.Model` -> `NNCFGraph` +- `graph/metatypes/openvino_metatypes.py` - `OV_OPERATOR_METATYPES` registry, `OV*Metatype` classes +- `graph/model_transformer.py` - `OVModelTransformer` +- `graph/transformations/commands.py` - `OVTargetPoint`, `OVInsertionCommand` and other commands +- `quantization/quantize_model.py` - `quantize_impl`, `quantize_with_accuracy_control_impl`, `compress_weights_impl` +- `engine.py` - `OVNativeEngine` / `OVCompiledModelEngine` for inference during calibration +- `statistics/` - statistic collectors + +## Gotchas + +- `OVModelTransformer.transform` clones the model unless `inplace=True`; do not assume the input model is modified. +- Target points address explicit input/output ports (`port_id`), not just nodes. +- Preserve NNCF runtime info when creating new models (`copy_rt_info` in `graph/model_utils.py`). + +## Tests + +- `tests/openvino/native/`, run with `make test-openvino` (forces `OV_CPU_MAX_ISA=AVX2`, `ONEDNN_MAX_CPU_ISA=AVX2`). +- Reference data is versioned by OpenVINO release in `tests/openvino/native/data//`; + `get_actual_reference_for_current_openvino()` picks the newest version not above the installed one. +- Regenerate references: `pytest tests/openvino/native/... --regen-ref-data`. diff --git a/src/nncf/torch/AGENTS.md b/src/nncf/torch/AGENTS.md new file mode 100644 index 00000000000..573cfcf6b85 --- /dev/null +++ b/src/nncf/torch/AGENTS.md @@ -0,0 +1,39 @@ +# AGENTS.md - PyTorch backend + +Backend for eager `torch.nn.Module`. General rules: [root AGENTS.md](../../../AGENTS.md). +For `torch.fx.GraphModule` see [TorchFX AGENTS.md](../experimental/torch/fx/AGENTS.md). + +## Key Modules + +- `function_hook/` - current model-modification mechanism: + - `wrapper.py` - `wrap_model`, `ForwardWithHooks`, `register_pre/post_function_hook` + - `hook_storage.py` - `HookStorage` holding inserted compression modules + - `nncf_graph/nncf_graph_builder.py` - `build_nncf_graph`, `GraphModelWrapper` + - `model_transformer.py` - `PT2ModelTransformer` + - `quantization/quantize_model.py` - `quantize_impl`, `compress_weights_impl` + - `serialization.py` - `get_config` / `load_from_config` +- `graph/operator_metatypes.py` - `PT_OPERATOR_METATYPES` (also reused by TorchFX) +- `graph/transformations/commands.py` - `PTTargetPoint` and commands +- `strip.py` - `nncf.torch.strip` +- `extensions/` - C++/CUDA kernels + +## Hooks: Insertion and Execution + +- `wrap_model` replaces `model.forward` with `ForwardWithHooks` and adds an empty `HookStorage` to the model. +- Hooks are `nn.Module`s registered via `register_pre/post_function_hook(model, op_name, port_id, hook)`. +- Algorithms insert hooks via `PT2InsertionCommand` + `PT2ModelTransformer`; `op_name` is the `NNCFGraph` node name. +- On forward, `FunctionHookMode` (`TorchFunctionMode`) intercepts each torch function call and builds its `op_name`. +- It runs pre-hooks on inputs, calls the function, then runs post-hooks on outputs. + +## Gotchas + +- Compression ops are inserted as hooks into `HookStorage` of a wrapped model, not by replacing modules. +- `nncf.quantize` deep-copies the input model; `PT2ModelTransformer` then modifies the wrapped model in place. +- Graph is traced from a real forward pass, so an example input is required to build `NNCFGraph`. +- Serialization: compressed state is restored via `nncf.torch.load_from_config` + `load_state_dict`. + +## Tests + +- `tests/torch/` (markers `cuda`, `long`): `make test-torch-cpu`, `make test-torch-gpu`, `make test-torch-long`. +- Hook mechanism tests: `tests/torch/function_hook/`; reference data: `tests/torch/data/function_hook/`. +- Regenerate references: `pytest tests/torch/... --regen-ref-data`.