[johnbowen42]

Currently if we have LLVM IR shapes like

// Assume %0 is a closure pointer
store int32 %1, %0
store int32 %2, %0
call registered_lambda(%0)

our analysis of the lambda callsite cannot automatically tell which store to trust as the definitive source of truth for the first slot of the lambda at the callsite. Fortunately, LLVM has O(N) (N is size of the function) built-in analysis called MemorySSA, which can answer, at the callsite call registered_lambda(%0), which store is definitive. In the parlance of MemorySSA, we say the instruction store int32 %2, %0 "clobbers", where clobbering basically means is the source of truth for that memory value and the callsite.

Fortunately, this LLVM IR shape is relatively exotic for lambda closure pointer. Nonetheless, we want Proteus to conservatively and correctly deal with simple cases of memory clobbering, explicitly reject complex LLVM IR shapes, and above all not trust the def-use graph traversal ordering as a source of ground truth for memory locations. This PR introduces a MemorySSA oracle model, that the def-use walk uses as a guide when analyzing instructions that write to lambda closure pointers. The oracle allows the analysis to ask, "which instruction clobbers the lambda closure pointer at a given lambda callsite??" Of course, an adversarial user can cook up examples where lambda closures pass through phi nodes, or a lambda is re-assembled piecemeal from disparate memory locations. These are all cases we currently conservatively reject, and update the clang frontend to do help users understand why.

Here are some of the edge cases where we fail to specialize
(1) What if the analysis finds that different slots in a lambda are stored from different pointers? -- fail to specialize
(2) What if we find that multiple user-defined function calls determine the value of different slots of the lambda closure -- fail to specialize
(3) What if we find multiple memcpys that copy different lambda slots from different source pointers? -- fail to specialize

Here are some cases where we do specialize
✅ the lambda closure pointer is clobbered at the callsite by a collection of stores, all from the same base ptr
✅ the lambda closure pointer is stored to multiple times, only one store clobbers
✅ the lambda closure pointer is clobbered at the callsite by a memory intrinsic
✅ the lambda closure pointer is clobbered at the callsite by a function call, which writes to the closure pointer using one of the above patterns

We try to handle all unsupported LLVM IR patterns from the Clang frontend which is also updated here. I also have added a flag to weaken the Clang frontend so that we can test that the LLVM IR analysis also knows about the bad shapes I describe above.

[johnbowen42]

marking as draft temporarily while I push up several commits

[davidbeckingsale]

I went through the full diff and spent some time running the pass offline against device IR for the shapes I was unsure about (libProteusPass built from this branch with LLVM 19, HIP device-only compile for gfx90a, reading the lambda manifest). Overall the design holds together well: the forward walk only collects candidates and acceptance requires MemorySSA's own selection to be among them, so most mistakes degrade to a decline. The findings below are the places where that does not hold, plus a few smaller things.

I pushed three regression tests to this branch (fe9f34d) that pin the confirmed issues. They assert the correct behavior, so they fail today and CI will go red until the fixes land; I verified each one reproduces by compiling its device side through the pass and reading the manifest, but I have not run them on a GPU.

Confirmed (reproduced against the built pass)

1. Direct-argument clobber guard is bypassed by any write that is not a bare StoreInst (miscompile, lambda_pointee_overwrite).
2. memcpy offset correction is frame-relative, so a displaced destination in a callee resolves to the wrong byte (miscompile, lambda_displaced_helper_memcpy).
3. Oracle reports live-on-entry for a multi-predecessor block with no MemoryPhi, declining straight-line memory histories through branchy callees (spurious decline, lambda_callee_join_no_phi).

Not reproduced, from reading
4. One MemorySSA oracle per callsite.
5. Virtual bases crash getWrapperIntervalInType.
6. The erased-write diagnostic only sees reinterpret_cast.
7. LambdaArgVisitor::visitMemIntrinsic is unreachable.
8. Comments cite a review doc that is not in the tree; one test comment is stale.
9. The CUDA integration script reverts main's absolute miniforge prefix.

Details inline. Happy to pair on 1 and 2 if useful; the offline harness makes the loop a couple of minutes.

[davidbeckingsale]

The MemorySSA query here only runs when HasDirectPriorStore finds a same-block StoreInst whose pointer operand is the argument itself. Any other write skips it: a memcpy, a constructor or helper call that writes through the argument, a store through a GEP of the argument, a store in a dominating block, or any write at all when an addrspacecast sits between the argument and the call (that path passes &ASC as the boundary, so MemoryAnalysisPtrUse == LambdaCB is false). The walk then falls through to the caller and accepts the original object.

Reproduced with tests/gpu/lambda_pointee_overwrite.cpp: a helper does __builtin_memcpy(Target, Replacement, sizeof(F)); (*Target)();. The manifest records kernel arg 0 (Initial) while the device executes Replacement. The whole-object memcpy is legal under the new checker contract, so the frontend does not catch it either.

I see the pre-filter came in with "Fix cuda unit test", which suggests the unfiltered query produced spurious declines (probably unrelated calls before the lambda call showing up as clobbers on a non-alloca location). I think the query needs to run unconditionally on this shape, with the non-aliasing-call case handled the way callCannotModifyTrackedLocal does for allocas, rather than being gated on a syntactic store.

[davidbeckingsale]

OffsetCorrection = DstOff - SrcOff is computed from GetPointerBaseWithConstantOffset on the raw destination, which is relative to the current frame's base (the formal, in a callee). The store path at line 1153 derives its correction relative to the visitor root instead: ValueRangeMap[TrackedBase].Offset - StoreRange.Offset. When a transfer is resolved in a callee whose formal was passed a displaced pointer (&env.Body), the two disagree by that displacement.

Reproduced with tests/gpu/lambda_displaced_helper_memcpy.cpp: closure at offset 4 of an envelope, helper memcpy's over it through a field pointer. The pass log shows the callee memcpy resolving with offset-correction=0, and the manifest records kernel arg 1 at offset 4 instead of 0. With a 16-byte Pair source that stays in bounds and would fold Body.Y and Spare.X as the two slots. With an 8-byte source the same error happens to trip the full-extent check and declines, which is probably why nothing in the existing suite caught it.

Using the same root-relative displacement as the store path, (ValueRangeMap[TrackedBase].Offset - Range.Offset) - SrcOff, gives the right answer in both the root frame and callees; DstOff should drop out.

[davidbeckingsale]

When the boundary block has no prior MemoryDef, no MemoryPhi, and more than one predecessor, this returns live-on-entry. The absence of a MemoryPhi means every predecessor carries the same reaching access, so the walk should continue through the immediate dominator (or any predecessor) rather than give up. This matters because every deferred callee resolution queries at the callee's ReturnInst, which has no memory access of its own.

Reproduced with tests/gpu/lambda_callee_join_no_phi.cpp: a helper memcpy's the closure, then if (Flag) V = pureScale(Flag); with pureScale marked const, then returns V. The return block has two predecessors and an SSA phi but no MemoryPhi; the analysis logs "MemorySSA did not identify a clobbering instruction" and declines. The straight-line twin specializes. At -O2 any helper with a register-only diamond before its return hits this.

[davidbeckingsale]

analyzeLambdaUses builds a new LambdaArgVisitor per callsite and the constructor (line 227) makes a fresh MemorySSAClobberOracle, so DominatorTree, AssumptionCache, AAResults and MemorySSA are rebuilt for the same enclosing functions once per callsite, and again per functor operator in emitLambdaDeviceCallsiteManifest. The IR is not mutated during analysis, so one oracle could be shared across both loops.

[davidbeckingsale]

Record->bases() includes virtual bases, and ASTRecordLayout::getBaseClassOffset requires a direct non-virtual base (it asserts in debug builds and reads a missing map entry otherwise). A struct Derived : virtual Base where Base holds a wrapper, memcpy'd as a whole, would hit this. Checking Base.isVirtual() and either using getVBaseClassOffset or skipping conservatively would close it.

[davidbeckingsale]

The erased-write diagnostic records only CXXReinterpretCastExpr and only diagnoses BinaryOperator assignments. auto *W = (uint32_t *)&Copy; W[1] = 5; compiles clean, as do static_cast<void*> chains, ++W[1], and passing the erased pointer to a function that writes through it. The IR analysis does still decline these shapes (I checked the C-style cast case), so this is diagnostic coverage rather than soundness, but the design doc leans on the checker as the contract boundary, so worth tightening or documenting.

[davidbeckingsale]

I believe this visitor is unreachable: LambdaArgVisitor only ever pushes SSA values (store value operands, memcpy raw sources, call operands, pointer operands), never a void-typed intrinsic, so visitMemIntrinsic and visitIntrinsicInst here cannot execute and the FunctorSize coverage check added in this PR is dead. The real check is the one in LambdaInstUseVisitor::visitMemIntrinsic. Suggest deleting these or asserting unreachable so there is one source of truth.

[davidbeckingsale]

This and KernelArgPtrUseVisitor.h:1010 and the design doc point readers to docs/dev/reviews/clobbering-aware-use-analysis.md and "gate 8", but no docs/dev/reviews directory exists on the branch. Either add the doc or inline the rationale. Relatedly, tests/gpu/lambda_size_test.cpp:27 still says the envelope case is EXPECTED TO FAIL while its CHECK lines require success.

[davidbeckingsale]

This changes install_miniforge back to the relative miniforge3, undoing the ${PWD}/miniforge3 absolute prefix that #549 introduced and #554 kept. If that was deliberate (the commit says "Try to fix cuda integration tests"), a note on why would help; otherwise it looks like a rebase artifact.