Skip to content

fix: keep plugins in other sys.path entries importable - #584

Open
cristianchiriac wants to merge 1 commit into
SigmaHQ:mainfrom
cristianchiriac:fix/sigma-namespace-extend-path
Open

cristianchiriac wants to merge 1 commit into
SigmaHQ:mainfrom
cristianchiriac:fix/sigma-namespace-extend-path

Conversation

@cristianchiriac

Copy link
Copy Markdown
Contributor

pySigma 2.0.0 added sigma/__init__.py (for sigma.default_policy). Before that, sigma was an implicit namespace package. Now it is a regular package, so Python no longer combines sigma/ directories from different sys.path entries. Backends, pipelines and validators that are installed outside pySigma's own site-packages directory can't be imported any more. The most common case is an editable install, which is what poetry install does in every backend checkout and CI job.

Reproduction in fresh venvs, using import sigma.backends.logpoint with pySigma-backend-logpoint:

pySigma plugin install result
1.5.1 pip install -e OK
2.0.0 pip install -e ModuleNotFoundError: No module named 'sigma.backends.logpoint'
2.0.0 pip install (non-editable) OK
this PR pip install -e OK

The same error shows up in CI for backends that move to pySigma 2.0. For example, logpoint/pySigma-backend-logpoint#40 fails at test collection: https://github.com/logpoint/pySigma-backend-logpoint/actions/runs/37338362378/job/111858805629

Fix: sigma/__init__.py calls pkgutil.extend_path so sigma.__path__ again covers sigma/ directories on all sys.path entries. sigma.backends and sigma.pipelines stay namespace packages, and sigma.default_policy and its documented override keep working unchanged.

The alternative would be to move default_policy into sigma.policy and delete sigma/__init__.py again. That would also fix the problem, but it changes the sigma.default_policy API documented in docs/guides/sigma_policy.rst. I went with the smaller change, but I'm happy to switch if you prefer that approach.

Testing

  • New tests/test_namespace.py imports a plugin module placed in a separate PYTHONPATH entry in a subprocess. It fails without the change (ModuleNotFoundError) and passes with it.
  • Full suite: 1807 passed, 2 skipped (-m "not online"). mypy sigma and black --check are clean.
  • With this branch installed and logpoint and sqlite installed as editable plugins, both import, and InstalledSigmaPlugins.autodiscover() finds them.

pySigma 2.0 added sigma/__init__.py to provide sigma.default_policy. That
turned sigma from an implicit namespace package into a regular package, so
Python stops looking for sigma.* modules in other sys.path entries. Plugins
installed in editable mode (e.g. poetry install in a backend checkout) or into
another site directory can no longer be imported:

    ModuleNotFoundError: No module named 'sigma.backends.logpoint'

Extend sigma.__path__ with pkgutil.extend_path so these portions are found
again, while keeping sigma.default_policy where it is.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant