Skip to content

CPF Merge after <Mapping> applied hangs #1219

Description

@isc-dchui

Describe the bug

A module that declares both a <Mapping> and a <CPF> resource deadlocks during install if the CPF merge runs after the mapping is applied. The install hangs indefinitely at the merge and must be killed.

The cause is an interaction between IPM's install transaction and the external merge process:

  1. %IPM.Utils.Module:LoadNewModule opens a transaction (src/cls/IPM/Utils/Module.cls:1161) and holds it open across the entire lifecycle chain, committing only after ExecutePhases returns (src/cls/IPM/Utils/Module.cls:1323).
  2. Applying a <Mapping> calls Config.MapPackages.Create(..., $$$CPFSave) via %IPM.Utils.Module:AddPackageMapping, which locks the CPF/config globals. Because $tlevel > 0, IRIS defers the lock release until the transaction ends, so the process keeps holding it.
  3. %IPM.ResourceProcessor.CPF:MergeCPF then shells out to iris merge through $zf(-100) (src/cls/IPM/ResourceProcessor/CPF.cls:119-135). That external process needs the same lock and cannot see or wait out an uncommitted transaction belonging to another process.

The child process waits on a lock that cannot be released until the install commits; the install cannot commit until the child returns.

Order decides whether this happens. Both directions were tested:

Order <CPF> attribute Result
Mapping, then merge Phase="Activate" When="After" Hangs
Merge, then mapping default Phase="Initialize" Passes

Merge-first never overlaps, because the lock is taken only after the external process has exited.

This has gone unnoticed because no existing module or test combines the two resource types. The default <CPF> phase is Initialize, which precedes the Reload phase where mappings are applied, so the safe order happens to be the default.

To Reproduce

Steps to reproduce the behavior:

  1. Create a module directory with this module.xml. The only thing that matters is Phase="Activate", which puts the merge after the mapping:
<?xml version="1.0" encoding="UTF-8"?>
<Export generator="Cache" version="25">
  <Document name="MappingCPFOrder.ZPM">
    <Module>
      <Name>MappingCPFOrder</Name>
      <Version>0.0.1</Version>
      <Packaging>module</Packaging>
      <SourcesRoot>src</SourcesRoot>
      <Mapping Name="MappingCPFOrder.Foo.PKG" Source="USER" />
      <CPF Name="order.cpf" Phase="Activate" When="After" />
      <Resource Name="MappingCPFOrder.PKG" />
    </Module>
  </Document>
</Export>
  1. Add src/cpf/order.cpf:
[config]
globals=0,0,160000,0,0,0
  1. Add any class under src/cls/MappingCPFOrder/, e.g. Sample.cls containing Class MappingCPFOrder.Sample Extends %RegisteredObject { }.

  2. Run zpm "load /path/to/MappingCPFOrder".

  3. The install prints the merge banner and then hangs forever:

[USER|MappingCPFOrder]   Compile SUCCESS
[USER|MappingCPFOrder]   Activate START
[USER|MappingCPFOrder]   Configure START
[USER|MappingCPFOrder]   Configure SUCCESS
Seeding update steps for module = mappingcpforder, for version = 0.0.1

Evaluating system expressions using temporary file at /tmp/fil6Kz83e.cpf

Merging CPF file: /path/to/MappingCPFOrder/src/cpf/order.cpf
[config]
        globals=0,0,160000,0,0,0

^%SS shows the process sitting in %IPM.ResourceProcessor.CPF. The process must be killed; Ctrl-C does not interrupt it, as MergeCPF runs with interrupts disabled.

To confirm the ordering is the trigger, change the <CPF> line to the default phase:

<CPF Name="order.cpf" />

The same module then installs cleanly, with both the mapping and the merged setting applied.

Expected behavior

load completes. The <Mapping> is applied and the <CPF> file is merged, in whatever order the declared phases imply. Neither resource type should be sensitive to the other's phase, and no combination of standard resources should be able to deadlock an install.

System information (please complete the following information):

  • IPM version: 0.10.10-SNAPSHOT (reproduced on main; not specific to any recent change)
  • IRIS version: containers.intersystems.com/intersystems/iris-community:2026.1
  • OS: Linux (container), host Windows 11
  • Docker or local: Docker

Additional context

Possible directions, roughly in order of how well they address the cause:

  1. Make the merge in-process — use Config.CPF.Merge() instead of the $zf(-100) callout, removing the cross-process lock entirely. There is already a TODO at src/cls/IPM/ResourceProcessor/CPF.cls:120-122 wanting this, noting that Config.CPF.Merge() "doesn't work" and referencing the discussion at Feat: CPF Merge #703 (comment). Resolving that would fix every ordering, for every resource combination.
  2. Defer config-global writes until after the install transaction commits — collect mapping changes during the install and apply the CPF write plus activation at the end. Related prior commit 589ce2b9, which introduced %IPM.Utils.Module:PerformBatchActivation to batch activation for unmap and enable commands. Note that batching alone does not fix this: the write still occurs inside the same open transaction, so the lock is still held when the merge spawns. This was tested and the deadlock persisted.
  3. Avoid holding a transaction across phases that spawn external processes. This weakens the rollback guarantee the transaction exists to provide.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions