Skip to content

Supprt Python 3.15 with breaking changes. - #945

Draft
junkmd wants to merge 11 commits into
enthought:mainfrom
junkmd:py315_support
Draft

junkmd wants to merge 11 commits into
enthought:mainfrom
junkmd:py315_support

Conversation

@junkmd

@junkmd junkmd commented Sep 21, 2026 •

Copy link
Copy Markdown
Collaborator

1. Summary

This PR adds official support for Python 3.15 to comtypes.
It resolves internal CPython ABI/struct layout incompatibilities and updates the code generator to handle Python 3.15's changes to enum.IntFlag.

Warning

Breaking Change:
Enumerations containing negative member values are now generated as enum.IntEnum instead of enum.IntFlag. This avoids Python 3.15 range-masking behavior and mathematically inconsistent bitflag definitions, but may affect user code performing bitwise operations or IntFlag type checks on those enums.

2. Motivation & Background

  1. Python 3.15 PyCArgObject Layout Change:

    • See Support Python 3.15 PyCArgObject layout changes in comtypes.util #938.
    • In Python 3.15, the internal layout of PyCArgObject in ctypes changed (tag changed from c_char to c_char_p, and size changed from c_int to c_ssize_t).
    • Without an adaptive bridge, low-level offset calculations in comtypes.util fail or cause memory access violations on Python 3.15.
  2. Python 3.15 enum.IntFlag Negative Member Masking:

    • See Adapting to Python 3.15+ IntFlag changes for negative members. #894.
    • In Python 3.15, IntFlag reinterprets negative member values by masking them into the positive bit domain rather than preserving their negative literal values (e.g., -1).
    • Bitwise flag semantics on negative integer constants are conceptually flawed.
    • Generated friendly modules previously cast all enums unconditionally to IntFlag, causing negative constants (such as MsiInstallState.msiInstallStateUnknown = -1) to corrupt their literal values on Python 3.15+.
  3. Minor Fixes & Code Cleanups
    Alongside the Python 3.15 updates, we have cleaned up several legacy inconsistencies and over-engineered utility functions to improve codebase health and long-term maintainability.

3. Breaking Changes & Migration Guide

enum.IntFlag → enum.IntEnum for Negative-Valued Enums

Description of the Incompatibility

  • Previous behavior: All enums generated in friendly modules (comtypes.gen.<mod>) inherited from enum.IntFlag.
  • New behavior:
    • Enums containing at least one negative member value inherit from enum.IntEnum.
    • Enums containing only non-negative values continue to inherit from enum.IntFlag.

Impact on Your Code

  • Bitwise Operations: Bitwise operations (|, &, ^, ~) on negative-valued enums (e.g., MsiInstallState) will no longer return an enum instance or may raise a TypeError depending on usage.
  • Type Checking: Code relying on isinstance(val, enum.IntFlag) or issubclass(EnumType, enum.IntFlag) will return False for enums with negative values.

Migration Action

  • If your code uses bitwise operations on enums, ensure the enum does not represent distinct negative status/sentinel codes (e.g., check against IntEnum values using equality == rather than bitwise masking &).
  • Update type annotations or assertions that explicitly assumed enum.IntFlag.

Removal of setup_logging from comtypes.logutil

Description of the Incompatibility

  • Previous behavior: comtypes.logutil provided a setup_logging(*pathnames) helper that configured the root logger from INI-style config files. It had been deprecated (with a DeprecationWarning) since Proposal: Fix NTDebugHandler and Deprecate setup_logging in logutil.py. #920.
  • New behavior: setup_logging and the internal deprecated decorator have been removed entirely from comtypes.logutil. The module now only exposes NTDebugHandler.

Impact on Your Code

  • Any call to comtypes.logutil.setup_logging(...) will raise AttributeError at runtime.

Migration Action

  • Set up logging yourself using the standard logging module. If you were using setup_logging to route debug output to the Windows debug console (e.g., DebugView), configure NTDebugHandler directly:

    import logging
    from comtypes.logutil import NTDebugHandler
    
    handler = NTDebugHandler()
    handler.setFormatter(logging.Formatter("%(levelname)s:%(name)s:%(message)s"))
    logging.root.addHandler(handler)
    logging.root.setLevel(logging.DEBUG)
  • For INI-based configuration, use logging.config.fileConfig from the standard library instead.


VARIANT.missing Now Uses Named Constant

Description of the Change

  • Previous behavior: VARIANT.missing was initialized by assigning the raw integer literal 0x80020004 to v._.VT_I4.
  • New behavior: The value is now assigned via the named constant hresult.DISP_E_PARAMNOTFOUND.

Note

There is a subtle nuance here that is easy to overlook.

In Python, 0x80020004 is a positive integer (2147549188), whereas hresult.DISP_E_PARAMNOTFOUND is -2147352572 (the signed 32-bit interpretation of the same bit pattern). They are not the same Python integer.

In practice, however, neither value has caused any observed errors or behavioral differences.
Fundamentally, the semantically significant part of constructing a "missing parameter" VARIANT is setting vt = VT_ERROR. The exact integer stored in VT_I4 plays a secondary role in how COM callee code identifies the argument as missing.

This area likely contains a long-standing mistake, and we are taking this opportunity to fix both the literal value and improve readability.
Given the vast ecosystem of COM type libraries, there is a slight possibility this could affect something, but we are proceeding with this change with the clear recognition that this is a bug fix.

No migration action is required.

Important

If you encounter a regression where code that previously worked with VARIANT.missing now behaves differently, please report it to the community (open a GitHub issue). Such a regression would most likely surface a pre-existing latent inconsistency that was hidden by the opaque literal, and sharing it will help clarify the correct semantics for future maintainers.

4. Key Changes

Core & Code Generator

  • comtypes.tools.codegenerator.namespaces.EnumerationNamespaces:
    • Inspect member values to decide whether to derive from IntEnum (if negative values exist) or IntFlag.
    • Dynamically import IntEnum and/or IntFlag based on whether each is used.
  • comtypes.util:
    • Dynamically configure PyCArgObject fields based on sys.version_info >= (3, 15) (_TAG_TYPE = c_char_p, _SIZE_TYPE = c_ssize_t).

Logging (comtypes.logutil)

  • comtypes.logutil:
    • Removed setup_logging function and its associated deprecated decorator (see Proposal: Fix NTDebugHandler and Deprecate setup_logging in logutil.py. #920).
    • Added type annotations to NTDebugHandler.emit (record: logging.LogRecord, writeW: Callable[[str], None], return type -> None).
    • logging.NTDebugHandler = NTDebugHandler assignment now carries a # type: ignore comment to suppress mypy's complaint about the monkey-patch.
  • comtypes.test.test_logutil:
    • Removed Test_deprecated test class and import of deprecated.

Automation (comtypes.automation)

  • VARIANT.missing:
    • Replaced the raw integer literal 0x80020004 with the named constant hresult.DISP_E_PARAMNOTFOUND for clarity. No behavioral change.

Tests & Documentation

  • comtypes.test.test_client: Added test_enum_base_classes to assert correct base class assignment (IntEnum vs IntFlag) based on member sign.
  • comtypes.test.test_util: Removed temporary skip logic for Python 3.15 alpha/beta.
  • docs/source/client.rst: Updated documentation with a Changed in version 1.5.0 callout detailing the IntEnum / IntFlag distinction and rationale.

CI & Build Infrastructure

  • Updated .github/workflows/autotest.yml to include Python 3.15 in the test matrix.

5. Verification & Testing

  • Test suite executed cleanly with cache invalidation (comtypes.clear_cache).
  • Verified PyCArgObject offset calculation on Python 3.15.
  • Verified code generation outputs expected IntEnum for negative members and IntFlag for positive members.
  • CI workflow runs successfully across Python 3.9 through 3.15.

@junkmd junkmd added this to the 1.5.0 / Support Python 3.15 milestone Sep 21, 2026
@codecov-commenter

codecov-commenter commented Sep 21, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.20%. Comparing base (824b7a1) to head (a5f8b9a).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #945      +/-   ##
==========================================
+ Coverage   88.98%   89.20%   +0.21%     
==========================================
  Files         140      140              
  Lines       13679    13642      -37     
==========================================
- Hits        12172    12169       -3     
+ Misses       1507     1473      -34     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@junkmd junkmd linked an issue Sep 21, 2026 that may be closed by this pull request
@junkmd
junkmd force-pushed the py315_support branch 3 times, most recently from 3563782 to 26bd33b Compare September 26, 2026 03:17
@junkmd
junkmd force-pushed the py315_support branch 3 times, most recently from 437ea3b to bd7e5b9 Compare October 4, 2026 09:04
@junkmd
junkmd force-pushed the py315_support branch 2 times, most recently from b9fecb7 to c9fbbb4 Compare October 4, 2026 10:49
junkmd added 11 commits October 4, 2026 20:20
…gative values. This commit enhances the code generator to correctly determine the base class for enumerations.

* `comtypes/tools/codegenerator/namespaces.py`:
  - Introduce `to_enums` method to generate Python `enum` classes.
  - If an enumeration contains negative values, it will be generated as
    `IntEnum` (e.g., `MsiInstallState`).
  - If an enumeration contains only non-negative values, it will be
    generated as `IntFlag` (e.g., `OLE_TRISTATE`).

* `comtypes/tools/codegenerator/codegenerator.py`:
  - Update the import statement to dynamically import `IntEnum` and `IntFlag`
    based on their usage.
  - Utilize the new `to_enums` method for enum generation.

* `comtypes/test/test_client.py`:
  - Add `test_enum_base_classes` to verify the correct generation of
    `IntEnum` and `IntFlag` for enums based on their value ranges.

This addresses issue for Python 3.15+ compatibility where `IntFlag` might
truncate negative values.
Remove the temporary logic that skipped tests on Python 3.15 alpha/beta
versions. This logic was previously added to avoid `RuntimeError` during
import due to `PyCArgObject` layout changes.
Update `util` to accommodate changes in the internal `PyCArgObject` structure
introduced in Python 3.15.

The `tag` field was changed from `c_char` to `c_char_p`, and the `size` field
was changed from `c_int` to `c_ssize_t`. This change uses a version-based
bridge to maintain backward compatibility with older Python versions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants