Preserve enum type in custom attribute argument of type obj (#995)

[edgarfgp]

Problem

Assigning an enum to a custom attribute argument of type obj loses the enum type. The value is stored in metadata as the underlying int32, so reading the attribute back gives a plain int rather than the enum. C# stores the same thing as the enum, so this is also a cross-language inconsistency.

Fixes #995.

Before

open System

type MyAttribute() =
    inherit Attribute()
    let mutable prop : obj = null
    member _.Prop with get () : obj = prop and set (v: obj) = prop <- v

type MyEnum = A = 1 | B = 2

[<My(Prop = MyEnum.B)>]
type C = class end

let v = (typeof<C>.GetCustomAttributes(false)[0] :?> MyAttribute).Prop
v.GetType()                // System.Int32
Convert.ToString v         // "2"

After

v.GetType()                // MyEnum
Convert.ToString v         // "B"

The enum type is now written into the attribute blob using the ECMA-335 enum tag (0x55 + type name), the same encoding C# produces, so it round-trips as the enum. Reading that encoding back (which previously threw) is also handled, and decode/encode are kept symmetric so static linking preserves it.

Quotations

The original triage worried this needed quotation/TAST changes. It doesn't: the fix only reads the enum type (already on the Const node) when emitting the attribute. Quotations are untouched and already keep the enum type added a test to confirm.

[github-actions]

❗ Release notes required

You can open this PR in browser to add release notes: open in github.dev

---

✅ Found changes and release notes in following paths:

Change path	Release notes path	Description
src/Compiler	docs/release-notes/.FSharp.Compiler.Service/11.0.100.md

[github-actions]

🔍 Tooling Safety Check — Affects-Compiler-Output
Affects-Compiler-Output: changes custom attribute blob encoding (enum 0x55 tag in il.fs/IlxGen.fs)

Generated by PR Tooling Safety Check · opus46 3.2M · ◷

[T-Gro]

🤖 This review was generated by AI (@expert-reviewer agent). Findings may contain inaccuracies — please verify independently.

Nice, targeted fix for #995 — the ILAttribElem.Enum carrier plus the symmetric encode tag (0x55 + type name) are the right model, and consolidating the 0x55 decode into decodeCustomAttrElemType removes the duplicated named-arg branch. Two correctness concerns about the decode side, both in decodeILAttribData/parseVal in src/Compiler/AbstractIL/il.fs — see inline comments.

Summary:

1. The encode path now writes the enum's true underlying width, but the decode fallback still reads a fixed int32, so boxed enums whose underlying type isn't int32 (e.g. the LongEnum/int64 case this PR adds) don't round-trip through decodeILAttribData and can desync the blob pointer for any following named args. The runtime test passes only because the CLR — not decodeILAttribData — reads the metadata.
2. The new ILType.Value _ -> ILAttribElem.Enum(ty, v) wrap in the System.Object branch also fires for boxed primitives (int/bool/char/byte…), not just enums, mislabeling them as Enum on decode.

[T-Gro]

Encode/decode width is asymmetric for non-int32 boxed enums. The new encode path (GenAttribArg → ILAttribElem.Enum(enumIlTy, underlyingElem) → encodeCustomAttrPrimValue) emits the underlying value at its true width — 8 bytes for an int64 enum like the LongEnum added in EnumValueAsObjectArg01.fs. But when that blob is read back through decodeILAttribData, a boxed enum decodes to ILType.Value <enumName> (via the 0x55 case) and lands in this fallback, which unconditionally reads int32 (4 bytes).

Consequences for any non-int32 boxed enum:

• the decoded value is truncated/wrong, and
• sigptr advances 4 bytes instead of 8, desyncing the parse of any following named argument (silent blob corruption).

The LongEnum test only exercises the CLR's metadata reader at runtime, so it doesn't cover this path. The comment here acknowledges the int32 assumption but understates the impact — the PR description claims decode/encode are "kept symmetric so static linking preserves it," which doesn't hold for int64/byte/int16/… enums. Consider reading the value at the enum's underlying width (resolving it from the 0x55 type) or, at minimum, not silently advancing by the wrong amount.

[edgarfgp]

I think this is a pre-existing, documented assumption of decodeILAttribData

/// Not all custom attribute data can be decoded without binding types.  In particular
/// enums must be bound in order to discover the size of the underlying integer.
/// The following assumes enums have size int32.

[T-Gro]

This wrap also matches boxed primitives, not just enums. decodeCustomAttrElemType returns ILType.Value for the primitive tags too (et_I4 → typ_Int32, et_BOOLEAN → typ_Bool, et_CHAR, et_U1, …, all ILType.Value). So when a plain int/bool/char constant is boxed into an obj-typed attribute arg, this branch wraps it as ILAttribElem.Enum(System.Int32, ILAttribElem.Int32 n) rather than leaving it as ILAttribElem.Int32 n.

The re-encoded bytes happen to be identical (the type tag for System.Int32 is et_I4, not 0x55), so round-tripping survives — but the encoder emits a bare primitive for these while the decoder now emits an Enum, an asymmetry in the in-memory ILAttribElem shape that can surprise consumers pattern-matching on the decoded element. Consider restricting the wrap to genuine enums (e.g. exclude the well-known System.* primitive value types) so only real enums become ILAttribElem.Enum.

[edgarfgp]

Fixed in 7c088ce