diff --git a/docfx/docs/3rdPartyMetadata.md b/docfx/docs/3rdPartyMetadata.md index 3fa335e4..62bc22e0 100644 --- a/docfx/docs/3rdPartyMetadata.md +++ b/docfx/docs/3rdPartyMetadata.md @@ -8,6 +8,9 @@ But the general transformation from metadata to C# code may be applied to other Constructing metadata is outside the scope of this document. However you may find [the win32metadata architecture](https://github.com/microsoft/win32metadata/blob/main/docs/architecture.md) document instructive. +Metadata can keep native primitive and pointer signatures while supplying logical enum and ownership information through annotations. +See [metadata annotations and compatibility](metadata-annotations.md) for projection rules, unsupported shapes, and the pinned consumer compatibility harness. + ## Hooking metadata into CsWin32 Metadata is fed into CsWin32 through MSBuild items. diff --git a/docfx/docs/features.md b/docfx/docs/features.md index f912c88d..a6eac5ba 100644 --- a/docfx/docs/features.md +++ b/docfx/docs/features.md @@ -5,5 +5,6 @@ - Generates xml documentation based on and links back to learn.microsoft.com - Ships no bulky assemblies alongside your application. - [Layered composition](composition.md): multiple assemblies can extend a single shared `PInvoke` static class so callers reach every native API through one symbol. +- [Metadata annotations](metadata-annotations.md): project logical enums and contextual ownership while preserving native storage. ![Animation demonstrating p/invoke code generation](../images/demo.gif) diff --git a/docfx/docs/metadata-annotations.md b/docfx/docs/metadata-annotations.md new file mode 100644 index 00000000..71d0e4a4 --- /dev/null +++ b/docfx/docs/metadata-annotations.md @@ -0,0 +1,190 @@ +# Metadata annotations and compatibility + +Native metadata signatures need not replace native integers or pointers with synthetic enum or handle types. +CsWin32 consumes annotations in `Windows.Win32.Foundation.Metadata` to recover logical APIs without inventing native representation or ownership facts. + +## Associated enums + +`AssociatedEnumAttribute(string Name)` on a field, parameter, or return value identifies a logical enum. +An unqualified name first prefers the declaring namespace; otherwise it must be unambiguous in the metadata's loaded assembly-reference closure. +A namespace-qualified name can identify an enum in a referenced metadata assembly. +Loading an unrelated metadata file does not make its enum eligible. +Missing, ambiguous, non-enum, or platform-incompatible targets produce generation errors. + +For a logical enum, **the native use site's primitive type determines the generated enum's underlying width and signedness**, not the logical enum's metadata `value__` type. +CsWin32 then references that enum directly in public signatures and fields. +This recovers the OLD synthetic-enum projection while the winmd keeps faithful native primitive/pointer signatures. +It includes scalars, pointer elements and depth, fixed/flexible buffers, callback signatures, and source-generated signatures. +Fields remain real, ref-returnable fields; they do not become conversion properties just because the logical metadata enum declared different storage. +Member names and `[Flags]` semantics still come from the enum definition. + +Storage discovery considers compatible association sites in the actual loaded reference closure, including when the enum is requested before its consumers. +Conflicting native widths or signedness for one logical enum fail explicitly instead of depending on generation order. +Member values must fit the native width without truncation; representable signed/unsigned bit patterns, including high-bit flags, are preserved. +An already available C# enum must also have compatible storage. +Nonintegral storage, unsupported native shapes, and combined `AssociatedEnum`/`NativeBitfield` backing fields produce generation errors, not silent native-only fallbacks. + +A distinct pre-existing case is an enum used **directly in native metadata signatures**, not just named by annotations. +Those signatures already constrain its native ABI, so annotations elsewhere cannot retype it. +An existing scalar field with conflicting storage retains the established native-backing-field/value-property pattern. +Extern methods and unmanaged COM structs retain raw scalar entry points and add explicit converting enum overloads; a return-only view uses an `_Enum` suffix. +Indirect or unsupported contexts with that genuine storage conflict fail explicitly. +These bounded compatibility paths do not change the direct-field/signature contract for migrated logical enums, add a legacy-property option, or infer caller interfaces. + +## Contextual ownership + +`RAIIFreeAttribute(string Name)` on a producing return value or output parameter identifies cleanup. +Return annotations occupy metadata parameter row sequence zero. +`InvalidHandleValueAttribute(long Value)` may specify multiple invalid values on that return/output row. +Explicit contextual invalid values override legacy typedef invalid values. + +Generated owner identity includes the cleanup function, native value representation, and invalid-value set. +Different invalid-value contracts cannot share an owner with an incorrect `IsInvalid`. +A nondefault contract adds a deterministic `Invalid...` suffix to the owner type name. +BCL `SafeFileHandle`/`SafeRegistryHandle` reuse requires a compatible explicit contract: both zero and minus one are invalid. +Absent contextual values retain existing behavior. + +Typed borrowing leases and protected dangerous-value access retain their lifetime safeguards. +CsWin32 does not infer a fabricated handle identity or cleanup contract merely from a `void*` signature. +In particular, the supplied WinHttp pilot has no HINTERNET ownership annotations: its raw `void*` APIs remain raw, not automatically owned. +Wider or otherwise unsupported ownership representations require explicit review rather than guessed handle metadata. + +## Pinned consumer review + +The baseline is the in-progress ownership implementation at +[`f3f5a69aa4823466ef7b55bbdc8e5ba41ec021e6`](https://github.com/microsoft/CsWin32/commit/f3f5a69aa4823466ef7b55bbdc8e5ba41ec021e6) +from [#1844](https://github.com/microsoft/CsWin32/pull/1844), **not a released CsWin32 baseline**. +No release-compatibility claim follows from these checks. + +`HeaderAnnotationCompatibilityTests` runs OLD metadata + baseline CsWin32, OLD metadata + proposed CsWin32, and NEW metadata + proposed CsWin32. +All legs use the same requested API list, reference compilation, generation options, and unchanged caller source. +The pinned context is `net8.0-windows`, x64, unsafe, C# 12, Roslyn 5.9.0, public single-file generation, SafeHandles enabled, and separate `allowMarshaling=true`/`false` legs. +The matrix uses runtime marshaling, not COM source generators. + +The harness compares public symbols as well as compiling unchanged callers. +Reports grade results as `identical-generation`, `source-compatible-difference`, or `residual-source-break`, and retain symbol additions/removals and exact caller diagnostics. +Mutation controls remove associations or change pointer depth and require the unchanged canonical caller to fail. +The native-storage matrix deliberately gives the NEW logical enum a different metadata underlying type, across all eight integral native types and both enum-first/consumer-first request orders. +Public symbols and the same unchanged callers must still match the OLD synthetic-enum projection. +Independent existing-annotation regression reports are labeled separately from `synthetic-enum-migration` acceptance. + +### Recorded checkpoint evidence + +| Metadata / generator | Marshaling / no-marshaling public symbols | Result | +|--|--|--| +| Canonical OLD / baseline | 19 / 18 | Unchanged callers compile | +| Canonical OLD / proposed | 19 / 18 | Identical generation and public symbols | +| Canonical NEW / proposed | 19 / 18 | Identical generation and public symbols | +| Native-storage OLD / baseline and proposed, eight integral types | 19 / 18 | Identical generation; unchanged callers compile | +| Native-storage NEW / proposed, eight integral types, both request orders | 19 / 18 | Source-compatible text differences; identical public symbols and unchanged callers | +| WinHttp OLD / baseline | 187 / 186 | Unchanged callers compile | +| WinHttp OLD / proposed | 187 / 186 | Identical generation and public symbols | +| WinHttp NEW / proposed | 187 / 186 | Residual source break: four scheme-member renames; no other public-symbol or generated-text changes | + +The actual WinHttp image runs pin proposed generator commit `41ffdb9f4cf445d224d4477ec674a4039308c724`. +All 41 compatibility cases ran without skips, followed by both canonical caller-mutation controls. +The runner now includes those controls in its 43-case selection. +OLD + proposed remains identical to OLD + baseline; NEW + proposed is not graded source-compatible merely because its explicitly adjusted caller compiles. + +### Observed WinHttp member-name breaks + +Both actual NEW marshaling legs report exactly four `CS0117` errors in the unchanged `EnumFields` caller. +The enum remains `WINHTTP_INTERNET_SCHEME : int`; `URL_COMPONENTS.nScheme` and `WINHTTP_PROXY_RESULT_ENTRY.ProxyScheme` remain direct enum fields, including ref-field access. +All other public symbols, method signatures, callback/buffer projections, raw `void*` ownership contracts, and generated text match OLD. + +Each row identifies a public enum constant in `Windows.Win32.Networking.WinHttp.WINHTTP_INTERNET_SCHEME`, its unchanged caller, and the smallest caller adjustment: + +| Removed member | Added member / same value | Failing unchanged call site | +|--|--|--| +| `WINHTTP_INTERNET_SCHEME_HTTP` | `INTERNET_SCHEME_HTTP = 1` | `UnchangedCaller.cs(16,47)`: `url.nScheme = WINHTTP_INTERNET_SCHEME.WINHTTP_INTERNET_SCHEME_HTTP;` | +| `WINHTTP_INTERNET_SCHEME_HTTPS` | `INTERNET_SCHEME_HTTPS = 2` | `UnchangedCaller.cs(17,47)`: `url.nScheme = WINHTTP_INTERNET_SCHEME.WINHTTP_INTERNET_SCHEME_HTTPS;` | +| `WINHTTP_INTERNET_SCHEME_FTP` | `INTERNET_SCHEME_FTP = 3` | `UnchangedCaller.cs(18,53)`: `entry.ProxyScheme = WINHTTP_INTERNET_SCHEME.WINHTTP_INTERNET_SCHEME_FTP;` | +| `WINHTTP_INTERNET_SCHEME_SOCKS` | `INTERNET_SCHEME_SOCKS = 4` | `UnchangedCaller.cs(19,53)`: `entry.ProxyScheme = WINHTTP_INTERNET_SCHEME.WINHTTP_INTERNET_SCHEME_SOCKS;` | + +The supplied candidate uses native macro spellings instead of the OLD synthetic member spellings and contains no compatibility aliases. +Changing only these four member references restores compilation in both marshaling modes. +That adjustment is tested and reported separately; the three primary legs keep their caller sources unchanged and NEW keeps its `residual-source-break` grade. +Exact added/removed symbols and a member-name-only generated-text comparison reject any additional surface or output changes. +These four observed source breaks remain explicit maintainer-review items, not skipped NEW coverage. +These measurements do not establish that the renames are unavoidable, decide whether OLD logical member spellings can be preserved, or imply pilot acceptance. + +### Independent existing-annotation regression + +`ExistingAnnotatedFieldRegression` separately records the member-kind change from +`public MODE Windows.Win32.Test.RECORD.Mode { get; set; }` to a `public MODE` field. +The same ordinary reads and assignments compile with both generators. +An unchanged caller extension `public partial struct RECORD : IModeProperty`, where `IModeProperty` requires a `MODE Mode { get; set; }` property, instead fails with `CS0535` at `UnchangedCaller.cs(21,36)`. +Reflection-based callers looking up a property likewise need to account for the field representation; reflection behavior is not covered by the compile matrix. +This is a separately demonstrated existing-consumer regression, **not the OLD synthetic-enum field contract**, a pilot policy decision, or a prerequisite for WinHttp acceptance. +No caller shim, interface inference, or new generation option is required by the migration contract. +The two-generator evidence is retained in `existing-annotation-*.json` with `CompatibilityScope=independent-existing-annotation-regression`; it is not mislabeled as a migrated NEW leg. + +### Snapshot provenance and physical limitations + +Canonical metadata is emitted deterministically from the checked-in `AssociatedEnumTests` declarations: + +| Snapshot | SHA256 | +|--|--| +| Canonical OLD | `155C94BA94A497FE616BD9096799F3F86AB99B26EA82D3D91CB6608FC45522A0` | +| Canonical NEW | `6255F8A82C1FFED6DD6BF62BAB3162B7481EC2C49E5C4915899A6BC6A03A20F9` | +| WinHttp OLD full image | `1361407C2888102CB5161D83503235C09C36F917837C7107CA0A80E5CF4FAC43` | +| WinHttp migrated NEW full image | `C7AEC280B0CBB19CDAD6B94FFED8473B2B22E4042BF886FCF33DCBDD92274258` | +| Matching unmigrated frozen-input tooling control | `274A288E622B840552934AE74FE2A33FEC4E7D4E47E3D3454B92A147A13B0CDA` | + +The frozen OLD WinHttp image is from producer commit `b5b7b26ed09d648766f07449af48c64ef0e99d13`, native/header mirror `77817f12308e29ec8bf77dc6fe56786e65ff9e08`, SDK `10.0.26100.7705`, and matched ClangSharp `21.1.8.4`. +This frozen OLD image has a confirmed pre-existing physical defect: 39 local metadata-attribute constructor MethodDefs lack `HASTHIS`, while instance MemberRefs are present. +For example, the `AssociatedEnumAttribute` and `RAIIFreeAttribute` constructor signatures are `0001010E`, and `InvalidHandleValueAttribute` is `0001010A`, without the instance bit. +The OLD generated-C# and unchanged-caller results remain valid **consumer projection checks**, but do not establish physically valid constructor binding for the OLD SDK image. +CsWin32's name/payload interpretation and compilation of generated source are not a full metadata binding validator. +This known tooling-baseline defect is not attributed to the header migration, and the frozen OLD snapshot is not replaced. + +The actual migrated NEW image is from producer commit `9faf8a24831e1ef0a44d4ccfe4cd972f380452b3`, with tooling ancestor `00ade0afc94c965522ccdbac54a06fc1bcbd01a1`, the same SDK CPP, and the unchanged matched ClangSharp stack. +The tooling ancestor includes `e065daa65c51769815e4a37c229ec9191a0578f4` (SAL/MIDL `RetVal` deduplication), `461d84f61c6c9611985a1152addf4b4636c5afc4` (local instance-constructor `HASTHIS`), and native-association/discovery-cache corrections. +The matching complete **unmigrated frozen-input tooling control** is pinned separately in the table above. +Our limited constructor-header audit finds no missing instance bits in NEW or that control. +It is not migrated WinHttp NEW, is not substituted into any NEW acceptance leg, and is not a claim of complete physical metadata validation. +Matrix reports retain `OldMetadataPhysicalLimitations`/`NewMetadataPhysicalLimitations`; `unmigrated-tooling-control.json` separately records the frozen/control hashes and audit findings. + +The requested WinHttp corpus is checked into `HeaderAnnotationCompatibilityTests.WinHttpThreeWayMatrix`. +It covers request/access/authentication/scheme/URL flags, ref fields, raw and friendly entry points, callbacks, read buffers, interior-pointer header buffers, raw creation, and cleanup. +Canonical ownership tests separately execute contextual invalid values, output ownership, disposal while borrowed, and lease release, and run the lifetime analyzer. +The existing-native-enum compatibility checks also execute a converting unmanaged vtable call, including checked callers and native truncation, and compile both raw and enum scalar callers. + +Both actual NEW WinHttp legs have now run; the four member-name source breaks above are observed, not predictions. +The harness records exact diagnostics at the unchanged `EnumFields` call sites and checks that changing only those member names restores compilation. +Other NEW diagnostics or public/generated-text deltas fail the checks rather than being waived. +Canonical ownership evidence remains separate: the WinHttp images provide no in-scope RAIIFree/invalid-handle facts and establish no WinHttp-specific automatic ownership or runtime SafeHandle proof. + +All eight legal enum storage types are additionally checked on x86, x64, and ARM64 with deliberately different logical metadata storage. +Source-generated scalar/indirect tests verify direct enum signatures without conversion-marshaler attributes; they do not run the platform LibraryImport generator or native marshaling. +This checkpoint therefore does not establish runtime acceptance of LibraryImport generation. + +### Reproducing the matrix + +From a restored checkout, run: + +```powershell +.\tools\Test-HeaderAnnotationCompatibility.ps1 +``` + +The runner archives and builds the exact baseline commit under this worktree's ignored `obj` directory. +It sets `NBGV_GitEngine=Disabled` for its .NET commands and does not change package feeds or versions. +No other checkout or branch is modified. + +Supply immutable SDK images and their SHA256s for the pilot: + +```powershell +.\tools\Test-HeaderAnnotationCompatibility.ps1 ` + -OldMetadata 'C:\snapshots\old\Windows.Win32.winmd' -OldSHA256 '' ` + -NewMetadata 'C:\snapshots\new\Windows.Win32.winmd' -NewSHA256 '' +``` + +OLD-only input checks and reports the baseline/proposed OLD pair, then explicitly skips NEW acceptance. +Optionally add `-ToolingControlMetadata 'C:\snapshots\unmigrated-control\Windows.Win32.winmd' -ToolingControlSHA256 ''` to audit the corrected-tooling control separately. +The control does not populate `NewMetadata` or change the frozen OLD matrix. +Reports and generated sources are written to `obj\HeaderAnnotationCompatibility`. +JSON records the exact baseline/proposed commits, built generator assembly hashes, metadata hashes, options, requested APIs, unchanged callers, public symbols, and diagnostics. +The original clean `41ffdb9` actual-image reports and generated sources are preserved locally under `obj\HeaderAnnotationCompatibility\checkpoint-41ffdb9-actual-new`. +Each `winhttp-{marshaling,no-marshaling}-scheme-member-adjustment.json` separately records the explicit four replacements, adjusted caller source and diagnostics; it does not overwrite an unchanged-caller grade. +`compatibility.trx` includes the canonical association-removal and pointer-depth mutation controls. +A dirty proposed checkout is explicitly marked `+working-tree`. diff --git a/docfx/docs/toc.yml b/docfx/docs/toc.yml index 184674de..da27c699 100644 --- a/docfx/docs/toc.yml +++ b/docfx/docs/toc.yml @@ -5,3 +5,4 @@ items: - href: composition.md - href: ArchSpecificAPIs.md - href: 3rdPartyMetadata.md +- href: metadata-annotations.md diff --git a/src/Microsoft.Windows.CsWin32/ArrayTypeHandleInfo.cs b/src/Microsoft.Windows.CsWin32/ArrayTypeHandleInfo.cs index 4517da61..15d6d455 100644 --- a/src/Microsoft.Windows.CsWin32/ArrayTypeHandleInfo.cs +++ b/src/Microsoft.Windows.CsWin32/ArrayTypeHandleInfo.cs @@ -17,6 +17,11 @@ internal override TypeSyntaxAndMarshaling ToTypeSyntax(TypeSyntaxSettings inputs inputs = inputs with { AllowMarshaling = false }; } + if (customAttributes is { } attributes) + { + attributes.Generator.ValidateAssociatedEnumElementType(this.ElementType, attributes.Collection); + } + TypeSyntaxAndMarshaling element = this.ElementType.ToTypeSyntax(inputs, forElement, customAttributes); if (inputs.AllowMarshaling || inputs.IsField) { diff --git a/src/Microsoft.Windows.CsWin32/Generator.AssociatedEnum.cs b/src/Microsoft.Windows.CsWin32/Generator.AssociatedEnum.cs new file mode 100644 index 00000000..eaa7538e --- /dev/null +++ b/src/Microsoft.Windows.CsWin32/Generator.AssociatedEnum.cs @@ -0,0 +1,489 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +namespace Microsoft.Windows.CsWin32; + +public partial class Generator +{ + private readonly Dictionary<(string Name, string Namespace), HandleTypeHandleInfo> associatedEnums = new(); + private readonly Dictionary associatedEnumStorage = new(); + private Dictionary? associatedEnumParameterOwners; + + /// + /// Gets an enum's projected storage from its native association sites. + /// + /// The resolved enum definition. + /// The enum's underlying primitive type. + internal static PrimitiveTypeCode GetEnumUnderlyingType(HandleTypeHandleInfo enumType) => + enumType.Generator.GetProjectedEnumUnderlyingType((TypeDefinitionHandle)enumType.Handle); + + /// + /// Gets the integral storage type declared in the logical enum metadata. + /// + /// The resolved enum definition. + /// The metadata's enum storage type. + internal static PrimitiveTypeCode GetDeclaredEnumUnderlyingType(HandleTypeHandleInfo enumType) + { + TypeDefinition definition = enumType.Reader.GetTypeDefinition((TypeDefinitionHandle)enumType.Handle); + enumType.Generator.GetBaseTypeInfo(definition, out StringHandle baseName, out StringHandle baseNamespace); + if (!enumType.Reader.StringComparer.Equals(baseName, nameof(Enum)) || !enumType.Reader.StringComparer.Equals(baseNamespace, nameof(System))) + { + throw new GenerationFailedException($"AssociatedEnum \"{enumType}\" does not refer to an enum."); + } + + FieldDefinition storage = definition.GetFields().Select(enumType.Reader.GetFieldDefinition).Single(field => !field.Attributes.HasFlag(FieldAttributes.Static)); + if (storage.DecodeSignature(enumType.Generator.SignatureHandleProvider, null) is not PrimitiveTypeHandleInfo underlying + || underlying.PrimitiveTypeCode is not (PrimitiveTypeCode.SByte or PrimitiveTypeCode.Byte or PrimitiveTypeCode.Int16 or PrimitiveTypeCode.UInt16 + or PrimitiveTypeCode.Int32 or PrimitiveTypeCode.UInt32 or PrimitiveTypeCode.Int64 or PrimitiveTypeCode.UInt64)) + { + throw new GenerationFailedException($"AssociatedEnum \"{enumType}\" has unsupported storage."); + } + + return underlying.PrimitiveTypeCode; + } + + /// + /// Gets the associated enum syntax without treating its logical declaration as a native use site. + /// + /// The associated enum. + /// The projection settings. + /// The generated element. + /// The projected enum type. + internal static TypeSyntax GetAssociatedEnumTypeSyntax(HandleTypeHandleInfo enumType, TypeSyntaxSettings settings, GeneratingElement forElement) => + (enumType with { IsAssociatedEnumProjection = true }).ToTypeSyntax(settings, forElement, null).Type; + + /// + /// Resolves an enum association within the declaring namespace and the metadata reference closure. + /// + /// The attributes on the native value. + /// The associated enum, or null when no association is present. + internal HandleTypeHandleInfo? FindAssociatedEnum(CustomAttributeHandleCollection? customAttributeHandles) + { + if (this.FindAttribute(customAttributeHandles, InteropDecorationNamespace, AssociatedEnumAttribute) is not CustomAttribute attribute) + { + return null; + } + + if (attribute.Parent.Kind == HandleKind.FieldDefinition + && this.FindAttribute(customAttributeHandles, InteropDecorationNamespace, NativeBitfieldAttribute) is not null) + { + throw new GenerationFailedException("AssociatedEnum on a NativeBitfield backing field is not supported."); + } + + return this.ResolveAssociatedEnum(GetAssociatedEnumName(attribute), this.GetAssociatedEnumNamespace(attribute.Parent)); + } + + /// + /// Validates an associated enum's native element storage before projection. + /// + /// The native element type. + /// The attributes describing the association. + internal void ValidateAssociatedEnumElementType(TypeHandleInfo elementType, CustomAttributeHandleCollection? attributes) + { + if (this.FindAssociatedEnum(attributes) is not HandleTypeHandleInfo enumType) + { + return; + } + + while (elementType is ITypeHandleContainer container) + { + elementType = container.ElementType; + } + + if (elementType is PrimitiveTypeHandleInfo primitive) + { + this.ValidateAssociatedEnumStorage(enumType, primitive.PrimitiveTypeCode); + } + } + + /// + /// Requires an enum projection to match the native use site's width and signedness. + /// + /// The associated enum. + /// The native use site's primitive type. + internal void ValidateAssociatedEnumStorage(HandleTypeHandleInfo enumType, PrimitiveTypeCode nativeType) + { + if (this.RequiresAssociatedEnumConversion(enumType, nativeType)) + { + throw new GenerationFailedException($"AssociatedEnum \"{enumType}\" has conflicting native storage at this {nativeType} use site."); + } + } + + /// + /// Determines whether real native enum signatures require a scalar adapter instead of a direct projection. + /// + /// The associated enum. + /// The native use site's primitive type. + /// Whether the existing native enum and annotated scalar have different storage. + internal bool RequiresAssociatedEnumConversion(HandleTypeHandleInfo enumType, PrimitiveTypeCode nativeType) + { + if (!IsEnumStorageType(nativeType)) + { + throw new GenerationFailedException($"AssociatedEnum \"{enumType}\" requires integral native storage, not {nativeType}."); + } + + return GetEnumUnderlyingType(enumType) != nativeType; + } + + private static bool IsEnumStorageType(PrimitiveTypeCode type) => + type is PrimitiveTypeCode.SByte or PrimitiveTypeCode.Byte or PrimitiveTypeCode.Int16 or PrimitiveTypeCode.UInt16 + or PrimitiveTypeCode.Int32 or PrimitiveTypeCode.UInt32 or PrimitiveTypeCode.Int64 or PrimitiveTypeCode.UInt64; + + private static string GetAssociatedEnumName(CustomAttribute attribute) + { + CustomAttributeValue arguments = attribute.DecodeValue(CustomAttributeTypeProvider.Instance); + if (arguments.FixedArguments.Length != 1 || arguments.FixedArguments[0].Value is not string name || string.IsNullOrWhiteSpace(name)) + { + throw new GenerationFailedException("AssociatedEnum requires a nonempty enum name."); + } + + return name; + } + + private HandleTypeHandleInfo ResolveAssociatedEnum(string name, string scope) + { + if (this.associatedEnums.TryGetValue((name, scope), out HandleTypeHandleInfo? cached)) + { + return cached; + } + + TrySplitPossiblyQualifiedName(name, out string? enumNamespace, out string enumName); + List matches = new(); + bool incompatiblePlatform = false; + foreach (Generator generator in this.GetAssociatedEnumReferenceClosure()) + { + foreach (NamespaceMetadata ns in generator.GetNamespacesToSearch(enumNamespace)) + { + if (ns.Types.TryGetValue(enumName, out TypeDefinitionHandle handle)) + { + matches.Add(new HandleTypeHandleInfo(generator, generator.Reader, handle)); + } + + incompatiblePlatform |= ns.TypesForOtherPlatform.Contains(enumName); + } + } + + if (enumNamespace is null) + { + List localMatches = matches.Where(match => + match.Reader.StringComparer.Equals(match.Reader.GetTypeDefinition((TypeDefinitionHandle)match.Handle).Namespace, scope)).ToList(); + if (localMatches.Count > 0) + { + matches = localMatches; + } + } + + if (matches.Count != 1) + { + if (matches.Count == 0 && incompatiblePlatform) + { + throw new PlatformIncompatibleException($"AssociatedEnum \"{name}\" is not available for the target platform."); + } + + throw new GenerationFailedException(matches.Count == 0 + ? $"AssociatedEnum \"{name}\" was not found in the metadata reference closure." + : $"AssociatedEnum \"{name}\" is ambiguous. Use a namespace-qualified enum name."); + } + + HandleTypeHandleInfo result = matches[0]; + GetDeclaredEnumUnderlyingType(result); + this.associatedEnums.Add((name, scope), result); + return result; + } + + private HandleTypeHandleInfo? FindStoragePreservingEnumProperty(TypeHandleInfo nativeType, CustomAttributeHandleCollection attributes) => + nativeType is PrimitiveTypeHandleInfo primitive && IsEnumStorageType(primitive.PrimitiveTypeCode) + && this.FindAssociatedEnum(attributes) is HandleTypeHandleInfo enumType && GetEnumUnderlyingType(enumType) != primitive.PrimitiveTypeCode + ? enumType : null; + + private PrimitiveTypeCode GetProjectedEnumUnderlyingType(TypeDefinitionHandle enumHandle) + { + if (this.associatedEnumStorage.TryGetValue(enumHandle, out PrimitiveTypeCode cached)) + { + return cached; + } + + HandleTypeHandleInfo enumType = new(this, this.Reader, enumHandle); + PrimitiveTypeCode declaredStorage = GetDeclaredEnumUnderlyingType(enumType); + string name = this.Reader.GetString(this.Reader.GetTypeDefinition(enumHandle).Name); + PrimitiveTypeCode? nativeStorage = null; + string? firstSite = null; + IEnumerable generators = this.SuperGenerator is SuperGenerator super ? super.Generators.Values : [this]; + Generator[] consumers = generators.Where(consumer => consumer.GetAssociatedEnumReferenceClosure().Contains(this)).ToArray(); + if (consumers.Any(consumer => consumer.MetadataIndex.GetAssociatedEnumAttributes(name).Length > 0) + && this.HasNativeEnumSignatures(enumHandle, consumers)) + { + nativeStorage = declaredStorage; + } + else + { + foreach (Generator consumer in consumers) + { + foreach (CustomAttributeHandle handle in consumer.MetadataIndex.GetAssociatedEnumAttributes(name)) + { + CustomAttribute attribute = consumer.Reader.GetCustomAttribute(handle); + if (consumer.GetAssociatedEnumUseSiteAttributes(attribute) is not { } attributes) + { + continue; + } + + HandleTypeHandleInfo target = consumer.ResolveAssociatedEnum(GetAssociatedEnumName(attribute), consumer.GetAssociatedEnumNamespace(attribute.Parent)); + if (target.Generator != this || target.Handle != enumHandle) + { + continue; + } + + consumer.FindAssociatedEnum(attributes); + (TypeHandleInfo Type, string Name) site = consumer.GetAssociatedEnumUseSite(attribute); + TypeHandleInfo element = site.Type; + while (element is ITypeHandleContainer container) + { + element = container.ElementType; + } + + if (element is not PrimitiveTypeHandleInfo primitive || !IsEnumStorageType(primitive.PrimitiveTypeCode)) + { + throw new GenerationFailedException($"AssociatedEnum \"{target}\" requires integral native storage at {site.Name}."); + } + + PrimitiveTypeCode storage = primitive.PrimitiveTypeCode; + if (nativeStorage is PrimitiveTypeCode previous && previous != storage) + { + throw new GenerationFailedException($"AssociatedEnum \"{target}\" has conflicting native storage: {previous} at {firstSite}, and {storage} at {site.Name}. One projected enum cannot represent both widths/signedness."); + } + + nativeStorage = storage; + firstSite = site.Name; + } + } + } + + PrimitiveTypeCode result = nativeStorage ?? declaredStorage; + TypeDefinition enumDefinition = this.Reader.GetTypeDefinition(enumHandle); + string qualifiedName = $"{this.Reader.GetString(enumDefinition.Namespace)}.{name}"; + SpecialType expectedType = result switch + { + PrimitiveTypeCode.SByte => SpecialType.System_SByte, PrimitiveTypeCode.Byte => SpecialType.System_Byte, + PrimitiveTypeCode.Int16 => SpecialType.System_Int16, PrimitiveTypeCode.UInt16 => SpecialType.System_UInt16, + PrimitiveTypeCode.Int32 => SpecialType.System_Int32, PrimitiveTypeCode.UInt32 => SpecialType.System_UInt32, + PrimitiveTypeCode.Int64 => SpecialType.System_Int64, PrimitiveTypeCode.UInt64 => SpecialType.System_UInt64, + _ => throw new GenerationFailedException($"AssociatedEnum \"{qualifiedName}\" has unsupported native storage {result}."), + }; + if (this.FindTypeSymbolIfAlreadyAvailable(qualifiedName) is INamedTypeSymbol { EnumUnderlyingType: { } underlying } + && underlying.SpecialType != expectedType) + { + throw new GenerationFailedException($"AssociatedEnum \"{qualifiedName}\" requires {result} storage, but the referenced C# enum uses {underlying}."); + } + + this.associatedEnumStorage.Add(enumHandle, result); + return result; + } + + private bool HasNativeEnumSignatures(TypeDefinitionHandle enumHandle, IEnumerable consumers) + { + TypeDefinition definition = this.Reader.GetTypeDefinition(enumHandle); + string name = this.Reader.GetString(definition.Name); + string ns = this.Reader.GetString(definition.Namespace); + foreach (Generator consumer in consumers) + { + foreach (EntityHandle handle in consumer.MetadataIndex.GetNativeSignatureTypes()) + { + if (consumer == this && handle == enumHandle) + { + return true; + } + + if (handle.Kind == HandleKind.TypeReference) + { + TypeReference reference = consumer.Reader.GetTypeReference((TypeReferenceHandle)handle); + if (consumer.Reader.StringComparer.Equals(reference.Name, name) && consumer.Reader.StringComparer.Equals(reference.Namespace, ns) + && consumer.TryGetTypeDefHandle(handle, out QualifiedTypeDefinitionHandle target) + && target.Generator == this && target.DefinitionHandle == enumHandle) + { + return true; + } + } + } + } + + return false; + } + + private CustomAttributeHandleCollection? GetAssociatedEnumUseSiteAttributes(CustomAttribute attribute) + { + if (attribute.Parent.Kind == HandleKind.FieldDefinition) + { + FieldDefinition field = this.Reader.GetFieldDefinition((FieldDefinitionHandle)attribute.Parent); + TypeDefinition type = this.Reader.GetTypeDefinition(field.GetDeclaringType()); + CustomAttributeHandleCollection attributes = field.GetCustomAttributes(); + if (!this.IsCompatibleWithPlatform(type.GetCustomAttributes()) || !this.IsCompatibleWithPlatform(attributes)) + { + return null; + } + + return attributes; + } + + ParameterHandle parameterHandle = (ParameterHandle)attribute.Parent; + Parameter parameter = this.Reader.GetParameter(parameterHandle); + MethodDefinition method = this.Reader.GetMethodDefinition(this.GetAssociatedEnumParameterOwner(parameterHandle)); + TypeDefinition declaringType = this.Reader.GetTypeDefinition(method.GetDeclaringType()); + return this.IsCompatibleWithPlatform(declaringType.GetCustomAttributes()) && this.IsCompatibleWithPlatform(method.GetCustomAttributes()) + && this.IsCompatibleWithPlatform(parameter.GetCustomAttributes()) ? parameter.GetCustomAttributes() : null; + } + + private (TypeHandleInfo Type, string Name) GetAssociatedEnumUseSite(CustomAttribute attribute) + { + if (attribute.Parent.Kind == HandleKind.FieldDefinition) + { + FieldDefinition field = this.Reader.GetFieldDefinition((FieldDefinitionHandle)attribute.Parent); + TypeDefinition type = this.Reader.GetTypeDefinition(field.GetDeclaringType()); + CustomAttributeHandleCollection attributes = field.GetCustomAttributes(); + TypeHandleInfo nativeType; + if (this.FindAttribute(attributes, SystemRuntimeCompilerServices, nameof(FixedBufferAttribute)) is CustomAttribute fixedBuffer) + { + string elementName = fixedBuffer.DecodeValue(CustomAttributeTypeProvider.Instance).FixedArguments[0].Value!.ToString()!; + PrimitiveTypeCode code = elementName switch + { + "System.SByte" => PrimitiveTypeCode.SByte, "System.Byte" => PrimitiveTypeCode.Byte, + "System.Int16" => PrimitiveTypeCode.Int16, "System.UInt16" => PrimitiveTypeCode.UInt16, + "System.Int32" => PrimitiveTypeCode.Int32, "System.UInt32" => PrimitiveTypeCode.UInt32, + "System.Int64" => PrimitiveTypeCode.Int64, "System.UInt64" => PrimitiveTypeCode.UInt64, + _ => throw new GenerationFailedException($"AssociatedEnum cannot project fixed-buffer element type {elementName}."), + }; + nativeType = new PrimitiveTypeHandleInfo(code); + } + else + { + nativeType = field.DecodeSignature(this.SignatureHandleProvider, null); + } + + return (nativeType, $"{this.GetNamespaceForPossiblyNestedType(type)}.{this.Reader.GetString(type.Name)}.{this.Reader.GetString(field.Name)}"); + } + + ParameterHandle parameterHandle = (ParameterHandle)attribute.Parent; + Parameter parameter = this.Reader.GetParameter(parameterHandle); + MethodDefinition method = this.Reader.GetMethodDefinition(this.GetAssociatedEnumParameterOwner(parameterHandle)); + TypeDefinition declaringType = this.Reader.GetTypeDefinition(method.GetDeclaringType()); + MethodSignature signature = method.DecodeSignature(this.SignatureHandleProvider, null); + return ( + parameter.SequenceNumber == 0 ? signature.ReturnType : signature.ParameterTypes[parameter.SequenceNumber - 1], + $"{this.GetNamespaceForPossiblyNestedType(declaringType)}.{this.Reader.GetString(declaringType.Name)}.{this.Reader.GetString(method.Name)}::{(parameter.SequenceNumber == 0 ? "return" : this.Reader.GetString(parameter.Name))}"); + } + + private MethodDeclarationSyntax? DeclareStoragePreservingEnumOverload(MethodDefinition method, MethodDeclarationSyntax nativeMethod, bool avoidWinmdRootAlias) + { + MethodSignature signature = method.DecodeSignature(this.SignatureHandleProvider, null); + ParameterListSyntax parameters = nativeMethod.ParameterList.WithParameters([.. nativeMethod.ParameterList.Parameters.Select(StripAttributes)]); + ArgumentSyntax[] arguments = parameters.Parameters.Select(parameter => Argument(IdentifierName(parameter.Identifier)) + .WithRefKindKeyword(parameter.Modifiers.FirstOrDefault(modifier => modifier.Kind() is SyntaxKind.InKeyword or SyntaxKind.OutKeyword or SyntaxKind.RefKeyword))).ToArray(); + bool changed = false; + TypeSyntaxSettings settings = this.extensionMethodSignatureTypeSettings with { AvoidWinmdRootAlias = avoidWinmdRootAlias }; + foreach (ParameterHandle handle in method.GetParameters()) + { + Parameter parameter = this.Reader.GetParameter(handle); + int index = parameter.SequenceNumber - 1; + if (index < 0 || signature.ParameterTypes[index] is not PrimitiveTypeHandleInfo primitive + || nativeMethod.ParameterList.Parameters[index].Type is not PredefinedTypeSyntax nativeType + || this.FindAssociatedEnum(parameter.GetCustomAttributes()) is not HandleTypeHandleInfo enumType + || !this.RequiresAssociatedEnumConversion(enumType, primitive.PrimitiveTypeCode)) + { + continue; + } + + TypeSyntax projected = GetAssociatedEnumTypeSyntax(enumType, settings, GeneratingElement.FriendlyOverload); + parameters = parameters.ReplaceNode(parameters.Parameters[index], parameters.Parameters[index].WithType(projected.WithTrailingTrivia(Space))); + arguments[index] = arguments[index].WithExpression(UncheckedExpression(CastExpression(nativeType.WithoutTrivia(), arguments[index].Expression))); + changed = true; + } + + TypeSyntax returnType = nativeMethod.ReturnType; + ExpressionSyntax invocation = InvocationExpression(IdentifierName(nativeMethod.Identifier), [.. arguments]); + if (signature.ReturnType is PrimitiveTypeHandleInfo returnPrimitive && returnType is PredefinedTypeSyntax + && this.FindAssociatedEnum(this.GetReturnTypeCustomAttributes(method)) is HandleTypeHandleInfo returnEnum + && this.RequiresAssociatedEnumConversion(returnEnum, returnPrimitive.PrimitiveTypeCode)) + { + returnType = GetAssociatedEnumTypeSyntax(returnEnum, settings, GeneratingElement.FriendlyOverload); + invocation = UncheckedExpression(CastExpression(returnType, invocation)); + } + else if (!changed) + { + return null; + } + + return nativeMethod.WithAttributeLists(default) + .WithModifiers([.. nativeMethod.Modifiers.Where(modifier => !modifier.IsKind(SyntaxKind.ExternKeyword) && !modifier.IsKind(SyntaxKind.PartialKeyword))]) + .WithIdentifier(changed ? nativeMethod.Identifier : Identifier(nativeMethod.Identifier.ValueText + "_Enum")) + .WithParameterList(parameters) + .WithReturnType(returnType.WithTrailingTrivia(Space)) + .WithBody(Block(IsVoid(returnType) ? ExpressionStatement(invocation) : ReturnStatement(invocation))) + .WithExpressionBody(null) + .WithSemicolonToken(default); + } + + private IEnumerable GetAssociatedEnumReferenceClosure() + { + HashSet visited = new(); + Queue pending = new(); + pending.Enqueue(this); + while (pending.Count > 0) + { + Generator generator = pending.Dequeue(); + if (!visited.Add(generator)) + { + continue; + } + + yield return generator; + if (this.SuperGenerator is not SuperGenerator super) + { + continue; + } + + foreach (AssemblyReferenceHandle referenceHandle in generator.Reader.AssemblyReferences) + { + AssemblyReference reference = generator.Reader.GetAssemblyReference(referenceHandle); + if (super.TryGetGenerator(generator.Reader.GetString(reference.Name), out Generator? referencedGenerator)) + { + pending.Enqueue(referencedGenerator); + } + } + } + } + + private string GetAssociatedEnumNamespace(EntityHandle owner) + { + TypeDefinitionHandle declaringType; + if (owner.Kind == HandleKind.FieldDefinition) + { + declaringType = this.Reader.GetFieldDefinition((FieldDefinitionHandle)owner).GetDeclaringType(); + } + else if (owner.Kind == HandleKind.Parameter) + { + declaringType = this.Reader.GetMethodDefinition(this.GetAssociatedEnumParameterOwner((ParameterHandle)owner)).GetDeclaringType(); + } + else + { + throw new GenerationFailedException("AssociatedEnum is only supported on fields, parameters and return values."); + } + + return this.GetNamespaceForPossiblyNestedType(this.Reader.GetTypeDefinition(declaringType)); + } + + private MethodDefinitionHandle GetAssociatedEnumParameterOwner(ParameterHandle parameter) + { + if (this.associatedEnumParameterOwners is null) + { + this.associatedEnumParameterOwners = new(); + foreach (MethodDefinitionHandle methodHandle in this.Reader.MethodDefinitions) + { + foreach (ParameterHandle handle in this.Reader.GetMethodDefinition(methodHandle).GetParameters()) + { + this.associatedEnumParameterOwners.Add(handle, methodHandle); + } + } + } + + return this.associatedEnumParameterOwners[parameter]; + } +} diff --git a/src/Microsoft.Windows.CsWin32/Generator.Enum.cs b/src/Microsoft.Windows.CsWin32/Generator.Enum.cs index 09ce8e92..9f4ac7d6 100644 --- a/src/Microsoft.Windows.CsWin32/Generator.Enum.cs +++ b/src/Microsoft.Windows.CsWin32/Generator.Enum.cs @@ -8,12 +8,14 @@ public partial class Generator /// public bool TryGetEnumName(string enumValueName, [NotNullWhen(true)] out string? declaringEnum) => this.MetadataIndex.TryGetEnumName(this.Reader, enumValueName, out declaringEnum); - private EnumDeclarationSyntax DeclareEnum(TypeDefinition typeDef) + private EnumDeclarationSyntax DeclareEnum(TypeDefinitionHandle typeHandle, TypeDefinition typeDef) { bool flagsEnum = this.FindAttribute(typeDef.GetCustomAttributes(), nameof(System), nameof(FlagsAttribute)) is not null; + PrimitiveTypeCode storage = this.GetProjectedEnumUnderlyingType(typeHandle); + bool changedStorage = storage != GetDeclaredEnumUnderlyingType(new(this, this.Reader, typeHandle)); var enumValues = new List(); - TypeSyntax? enumBaseType = null; + TypeSyntax enumBaseType = PrimitiveTypeHandleInfo.ToTypeSyntax(storage, this.enumTypeSettings.PreferNativeInt); foreach (FieldDefinitionHandle fieldDefHandle in typeDef.GetFields()) { AddEnumValue(fieldDefHandle); @@ -32,11 +34,6 @@ private EnumDeclarationSyntax DeclareEnum(TypeDefinition typeDef) } } - if (enumBaseType is null) - { - throw new NotSupportedException("Unknown enum type."); - } - string? name = this.Reader.GetString(typeDef.Name); EnumDeclarationSyntax result = EnumDeclaration(Identifier(name), SeparatedList(enumValues)) .WithModifiers([TokenWithSpace(this.Visibility)]); @@ -64,12 +61,30 @@ void AddEnumValue(FieldDefinitionHandle fieldDefHandle) ConstantHandle valueHandle = fieldDef.GetDefaultValue(); if (valueHandle.IsNil) { - enumBaseType = fieldDef.DecodeSignature(this.SignatureHandleProvider, null).ToTypeSyntax(this.enumTypeSettings, GeneratingElement.EnumValue, null).Type; return; } bool enumBaseTypeIsSigned = enumBaseType is PredefinedTypeSyntax { Keyword: { RawKind: (int)SyntaxKind.LongKeyword or (int)SyntaxKind.IntKeyword or (int)SyntaxKind.ShortKeyword or (int)SyntaxKind.SByteKeyword } }; ExpressionSyntax enumValue = flagsEnum ? ToHexExpressionSyntax(this.Reader, valueHandle, enumBaseTypeIsSigned) : ToExpressionSyntax(this.Reader, valueHandle); + if (changedStorage) + { + Constant constant = this.Reader.GetConstant(valueHandle); + decimal value = Convert.ToDecimal(this.Reader.GetBlobReader(constant.Value).ReadConstant(constant.TypeCode), CultureInfo.InvariantCulture); + (decimal minimum, decimal maximum) = storage switch + { + PrimitiveTypeCode.SByte or PrimitiveTypeCode.Byte => (sbyte.MinValue, byte.MaxValue), + PrimitiveTypeCode.Int16 or PrimitiveTypeCode.UInt16 => (short.MinValue, ushort.MaxValue), + PrimitiveTypeCode.Int32 or PrimitiveTypeCode.UInt32 => (int.MinValue, uint.MaxValue), + _ => (long.MinValue, (decimal)ulong.MaxValue), + }; + if (value < minimum || value > maximum) + { + throw new GenerationFailedException($"AssociatedEnum member {enumValueName}={value} does not fit native {storage} storage without truncation."); + } + + enumValue = UncheckedExpression(CastExpression(enumBaseType, enumValue)); + } + EnumMemberDeclarationSyntax enumMember = EnumMemberDeclaration(SafeIdentifier(enumValueName), EqualsValueClause(enumValue)); enumValues.Add(enumMember); enumValues.Add(TokenWithLineFeed(SyntaxKind.CommaToken)); diff --git a/src/Microsoft.Windows.CsWin32/Generator.Extern.cs b/src/Microsoft.Windows.CsWin32/Generator.Extern.cs index ff3004f9..10cde2ab 100644 --- a/src/Microsoft.Windows.CsWin32/Generator.Extern.cs +++ b/src/Microsoft.Windows.CsWin32/Generator.Extern.cs @@ -220,26 +220,6 @@ private void DeclareExternMethod(MethodDefinitionHandle methodDefinitionHandle) CustomAttributeHandleCollection? returnTypeAttributes = this.GetReturnTypeCustomAttributes(methodDefinition); TypeSyntaxAndMarshaling returnType = signature.ReturnType.ToTypeSyntax(typeSettings with { IsReturnValue = true }, GeneratingElement.ExternMethod, returnTypeAttributes?.QualifyWith(this), ParameterAttributes.Out); - bool customMarshaling = this.options.AllowMarshaling && this.useSourceGenerators; - - // Search for any enum substitutions. - TypeSyntax? returnTypeEnumName = customMarshaling ? null : this.FindAssociatedEnum(returnTypeAttributes); - TypeSyntax?[]? parameterEnumType = null; - foreach (ParameterHandle parameterHandle in methodDefinition.GetParameters()) - { - Parameter parameter = this.Reader.GetParameter(parameterHandle); - if (parameter.SequenceNumber == 0) - { - continue; - } - - if (this.FindAssociatedEnum(parameter.GetCustomAttributes()) is IdentifierNameSyntax parameterEnumName && !customMarshaling) - { - parameterEnumType ??= new TypeSyntax?[signature.ParameterTypes.Length]; - parameterEnumType[parameter.SequenceNumber - 1] = parameterEnumName; - } - } - bool setLastError = (import.Attributes & MethodImportAttributes.SetLastError) == MethodImportAttributes.SetLastError; bool setLastErrorViaMarshaling = setLastError && (this.Options.AllowMarshaling || !this.canUseSetLastPInvokeError); bool setLastErrorManually = setLastError && !setLastErrorViaMarshaling; @@ -293,34 +273,19 @@ AttributeListSyntax CreateDllImportAttributeList() } MethodDeclarationSyntax exposedMethod; - if (returnTypeEnumName is null && parameterEnumType is null && !setLastErrorManually) + if (!setLastErrorManually) { // No need for wrapping the extern method, so just expose it directly. exposedMethod = externDeclaration.WithModifiers(externDeclaration.Modifiers.Insert(0, TokenWithSpace(this.Visibility))); } else { - string ns = this.GetMethodNamespace(methodDefinition); - NameSyntax nsSyntax = ParseName(ReplaceCommonNamespaceWithAlias(this, ns)); ParameterListSyntax exposedParameterList = this.CreateParameterList(methodDefinition, signature, typeSettings, GeneratingElement.ExternMethod); static SyntaxToken RefInOutKeyword(ParameterSyntax p) => p.Modifiers.Any(SyntaxKind.OutKeyword) ? TokenWithSpace(SyntaxKind.OutKeyword) : p.Modifiers.Any(SyntaxKind.RefKeyword) ? TokenWithSpace(SyntaxKind.RefKeyword) : default; ArgumentListSyntax argumentList = ArgumentList([.. exposedParameterList.Parameters.Select(p => Argument(IdentifierName(p.Identifier.ValueText)).WithRefKindKeyword(RefInOutKeyword(p)))]); - if (parameterEnumType is not null) - { - for (int i = 0; i < parameterEnumType.Length; i++) - { - if (parameterEnumType[i] is TypeSyntax parameterType) - { - NameSyntax qualifiedParameterType = QualifiedName(nsSyntax, (SimpleNameSyntax)parameterType); - exposedParameterList = exposedParameterList.ReplaceNode(exposedParameterList.Parameters[i], exposedParameterList.Parameters[i].WithType(qualifiedParameterType.WithTrailingTrivia(Space))); - this.RequestInteropType(ns, parameterEnumType[i]!.ToString(), this.DefaultContext); - argumentList = argumentList.ReplaceNode(argumentList.Arguments[i], argumentList.Arguments[i].WithExpression(CastExpression(externDeclaration.ParameterList.Parameters[i].Type!.WithTrailingTrivia(default(SyntaxTriviaList)), argumentList.Arguments[i].Expression))); - } - } - } // We need to specify Entrypoint because our local function will have a different name. // It must have a unique name because some functions will have the same signature as our exposed method except for the return type. @@ -328,13 +293,6 @@ static SyntaxToken RefInOutKeyword(ParameterSyntax p) => IdentifierNameSyntax localExternFunctionName = IdentifierName("LocalExternFunction"); ExpressionSyntax invocation = InvocationExpression(localExternFunctionName, argumentList); - if (returnTypeEnumName is not null) - { - this.RequestInteropType(ns, returnTypeEnumName.ToString(), this.DefaultContext); - returnTypeEnumName = QualifiedName(nsSyntax, (SimpleNameSyntax)returnTypeEnumName); - invocation = CastExpression(returnTypeEnumName, invocation); - } - BlockSyntax body = Block(); IdentifierNameSyntax? retValLocalName = returnType.Type is PredefinedTypeSyntax { Keyword.RawKind: (int)SyntaxKind.VoidKeyword } ? null : IdentifierName("__retVal"); @@ -352,7 +310,7 @@ static SyntaxToken RefInOutKeyword(ParameterSyntax p) => // var __retVal = LocalExternFunction(...); body = body.AddStatements( LocalDeclarationStatement( - VariableDeclaration(returnTypeEnumName ?? returnType.Type, [VariableDeclarator(retValLocalName.Identifier, EqualsValueClause(invocation))]))); + VariableDeclaration(returnType.Type, [VariableDeclarator(retValLocalName.Identifier, EqualsValueClause(invocation))]))); } else { @@ -381,7 +339,7 @@ static SyntaxToken RefInOutKeyword(ParameterSyntax p) => .WithSemicolonToken(SemicolonWithLineFeed); body = body.AddStatements(externFunction); - exposedMethod = MethodDeclaration(returnTypeEnumName ?? returnType.Type, externDeclaration.Identifier) + exposedMethod = MethodDeclaration(returnType.Type, externDeclaration.Identifier) .AddModifiers(TokenWithSpace(this.Visibility), TokenWithSpace(SyntaxKind.StaticKeyword)) .WithParameterList(exposedParameterList) .WithBody(body); diff --git a/src/Microsoft.Windows.CsWin32/Generator.FriendlyOverloads.cs b/src/Microsoft.Windows.CsWin32/Generator.FriendlyOverloads.cs index 7a6c3551..828918c5 100644 --- a/src/Microsoft.Windows.CsWin32/Generator.FriendlyOverloads.cs +++ b/src/Microsoft.Windows.CsWin32/Generator.FriendlyOverloads.cs @@ -78,6 +78,13 @@ private ExpressionSyntax GetIntPtrFromTypeDef(ExpressionSyntax typedefValue, Typ private IEnumerable DeclareFriendlyOverloads(MethodDefinition methodDefinition, MethodDeclarationSyntax externMethodDeclaration, NameSyntax declaringTypeName, FriendlyOverloadOf overloadOf, HashSet helperMethodsAdded, bool avoidWinmdRootAlias) { + if (overloadOf is FriendlyOverloadOf.ExternMethod or FriendlyOverloadOf.StructMethod + && this.DeclareStoragePreservingEnumOverload(methodDefinition, externMethodDeclaration, avoidWinmdRootAlias) is MethodDeclarationSyntax enumOverload) + { + yield return enumOverload; + externMethodDeclaration = enumOverload; + } + if (!this.options.FriendlyOverloads.Enabled) { yield break; @@ -578,7 +585,7 @@ private IEnumerable DeclareFriendlyOverload( IdentifierNameSyntax localName = IdentifierName(externParam.Identifier.ValueText + "Local"); // NOTE: We don't handle scenarios where the parameter is [MemorySize] annotated (e.g. EnumProcessModules) or [NativeArrayInfo] (e.g. ITypeInfo.GetNames) - if (this.RequestSafeHandle(outReleaseMethod, pointedElementInfo) is TypeSyntax safeHandleType) + if (this.RequestSafeHandle(outReleaseMethod, pointedElementInfo, valueAttributes: paramAttributes) is TypeSyntax safeHandleType) { // out SafeHandle parameters[paramIndex] = externParam @@ -1404,7 +1411,7 @@ bool TryHandleCountParam(TypeSyntax elementType, bool nullableSource) } TypeSyntax? returnSafeHandleType = this.TryGetResourceReleaseMethod(originalSignature.ReturnType, returnTypeAttributes, out string? returnReleaseMethod) - ? this.RequestSafeHandle(returnReleaseMethod, originalSignature.ReturnType) : null; + ? this.RequestSafeHandle(returnReleaseMethod, originalSignature.ReturnType, valueAttributes: returnTypeAttributes) : null; IdentifierNameSyntax resultLocal = IdentifierName("__result"); diff --git a/src/Microsoft.Windows.CsWin32/Generator.Handle.cs b/src/Microsoft.Windows.CsWin32/Generator.Handle.cs index aee1316d..b138d1ce 100644 --- a/src/Microsoft.Windows.CsWin32/Generator.Handle.cs +++ b/src/Microsoft.Windows.CsWin32/Generator.Handle.cs @@ -11,8 +11,9 @@ public partial class Generator /// The cleanup function. /// The original native value type, or null to use the cleanup parameter type. /// Whether caller-defined cleanup helpers may be returned. + /// Contextual annotations on the producing return value or output parameter. /// The owner type, or null when automatic ownership cannot be generated. - internal TypeSyntax? RequestSafeHandle(string releaseMethod, TypeHandleInfo? valueType = null, bool allowAbstract = false) + internal TypeSyntax? RequestSafeHandle(string releaseMethod, TypeHandleInfo? valueType = null, bool allowAbstract = false, CustomAttributeHandleCollection? valueAttributes = null) { if (!this.options.UseSafeHandles) { @@ -21,7 +22,11 @@ public partial class Generator try { + HashSet contextualInvalidValues = this.GetInvalidHandleValues(valueAttributes); + bool bclInvalidValuesCompatible = contextualInvalidValues.Count == 0 + || contextualInvalidValues.SetEquals([IntPtr.Zero, new IntPtr(-1)]); if (BclInteropSafeHandles.TryGetValue(releaseMethod, out TypeSyntax? knownOwner) + && bclInvalidValuesCompatible && (valueType is null || (valueType is HandleTypeHandleInfo knownHandle && knownHandle.Generator.GetQualifiedTypeDefinition(knownHandle.Handle) is QualifiedTypeDefinition definition && definition.Reader.GetString(definition.Definition.Namespace) == (releaseMethod == "CloseHandle" ? "Windows.Win32.Foundation" : "Windows.Win32.System.Registry") @@ -79,12 +84,24 @@ public partial class Generator return null; } - if (canonicalValueType && BclInteropSafeHandles.TryGetValue(releaseMethod, out TypeSyntax? bclType)) + if (canonicalValueType && bclInvalidValuesCompatible && BclInteropSafeHandles.TryGetValue(releaseMethod, out TypeSyntax? bclType)) { return bclType; } - string cacheKey = releaseMethod + ":" + nativeValueType; + HashSet defaultInvalidValues = valueType is HandleTypeHandleInfo nativeHandle + ? nativeHandle.Generator.GetInvalidHandleValues(nativeHandle.Handle) + : new() { IntPtr.Zero }; + if (defaultInvalidValues.Count == 0 && releaseMethodParameterTypeHandleInfo is PointerTypeHandleInfo) + { + defaultInvalidValues.Add(IntPtr.Zero); + } + + HashSet invalidHandleValues = contextualInvalidValues.Count > 0 ? contextualInvalidValues : defaultInvalidValues; + bool contextualInvalidProfile = !invalidHandleValues.SetEquals(defaultInvalidValues); + string invalidProfile = string.Join("_", invalidHandleValues.OrderBy(value => value.ToInt64()) + .Select(value => unchecked((ulong)value.ToInt64()).ToString("X", CultureInfo.InvariantCulture))); + string cacheKey = releaseMethod + ":" + nativeValueType + ":" + invalidProfile; if (this.volatileCode.TryGetSafeHandleForReleaseMethod(cacheKey, out TypeSyntax? safeHandleType, out bool cachedCanConstruct)) { return allowAbstract || cachedCanConstruct ? safeHandleType : null; @@ -100,6 +117,11 @@ public partial class Generator string safeHandleClassName = canonicalValueType ? $"{releaseMethod}SafeHandle" : $"{releaseMethod}{this.GetSafeHandleValueName(valueType, nativeValueType)}SafeHandle"; + if (contextualInvalidProfile) + { + safeHandleClassName = safeHandleClassName.Substring(0, safeHandleClassName.Length - "SafeHandle".Length) + $"Invalid{invalidProfile}SafeHandle"; + } + IdentifierNameSyntax safeHandleTypeIdentifier = IdentifierName(safeHandleClassName); safeHandleType = QualifiedName(ParseName($"global::{this.Namespace}"), safeHandleTypeIdentifier); CustomAttributeHandleCollection? atts = this.GetReturnTypeCustomAttributes(releaseMethodDef); @@ -130,16 +152,6 @@ public partial class Generator this.RequestExternMethod(releaseMethodHandle.Value); }); - // Collect all the known invalid values for this handle. - // If no invalid values are given (e.g. BSTR), we'll just assume 0 is invalid. - HashSet invalidHandleValues = valueType is HandleTypeHandleInfo nativeHandle - ? nativeHandle.Generator.GetInvalidHandleValues(nativeHandle.Handle) - : new() { IntPtr.Zero }; - if (invalidHandleValues.Count == 0 && releaseMethodParameterTypeHandleInfo is PointerTypeHandleInfo) - { - invalidHandleValues.Add(IntPtr.Zero); - } - IntPtr preferredInvalidValue = GetPreferredInvalidHandleValue(invalidHandleValues, new IntPtr(-1)); this.TryGetRenamedMethod(releaseMethod, out string? renamedReleaseMethod); @@ -456,11 +468,21 @@ private QualifiedTypeDefinition GetQualifiedTypeDefinition(EntityHandle handle) private HashSet GetInvalidHandleValues(EntityHandle handle) { QualifiedTypeDefinition td = this.GetQualifiedTypeDefinition(handle); + return td.Generator.GetInvalidHandleValues(td.Definition.GetCustomAttributes()); + } + + private HashSet GetInvalidHandleValues(CustomAttributeHandleCollection? attributes) + { HashSet invalidHandleValues = new(); - foreach (CustomAttributeHandle ah in td.Definition.GetCustomAttributes()) + if (attributes is not { } collection) { - CustomAttribute a = td.Reader.GetCustomAttribute(ah); - if (MetadataUtilities.IsAttribute(td.Reader, a, InteropDecorationNamespace, InvalidHandleValueAttribute)) + return invalidHandleValues; + } + + foreach (CustomAttributeHandle ah in collection) + { + CustomAttribute a = this.Reader.GetCustomAttribute(ah); + if (MetadataUtilities.IsAttribute(this.Reader, a, InteropDecorationNamespace, InvalidHandleValueAttribute)) { CustomAttributeValue attributeData = a.DecodeValue(CustomAttributeTypeProvider.Instance); long invalidValue = (long)(attributeData.FixedArguments[0].Value ?? throw new GenerationFailedException("Missing invalid value attribute.")); diff --git a/src/Microsoft.Windows.CsWin32/Generator.MetadataHelpers.cs b/src/Microsoft.Windows.CsWin32/Generator.MetadataHelpers.cs index 49987978..4d9aa35c 100644 --- a/src/Microsoft.Windows.CsWin32/Generator.MetadataHelpers.cs +++ b/src/Microsoft.Windows.CsWin32/Generator.MetadataHelpers.cs @@ -18,17 +18,6 @@ public partial class Generator internal CustomAttribute? FindAttribute(CustomAttributeHandleCollection? customAttributeHandles, string attributeNamespace, string attributeName) => MetadataUtilities.FindAttribute(this.Reader, customAttributeHandles, attributeNamespace, attributeName); - internal IdentifierNameSyntax? FindAssociatedEnum(CustomAttributeHandleCollection? customAttributeHandles) - { - if (this.FindAttribute(customAttributeHandles, InteropDecorationNamespace, AssociatedEnumAttribute) is CustomAttribute att) - { - CustomAttributeValue args = att.DecodeValue(CustomAttributeTypeProvider.Instance); - return IdentifierName((string)args.FixedArguments[0].Value!); - } - - return null; - } - internal bool TryGetTypeDefHandle(TypeReferenceHandle typeRefHandle, out QualifiedTypeDefinitionHandle typeDefHandle) { if (this.SuperGenerator is object) diff --git a/src/Microsoft.Windows.CsWin32/Generator.Struct.cs b/src/Microsoft.Windows.CsWin32/Generator.Struct.cs index 74712676..f6a8b5ee 100644 --- a/src/Microsoft.Windows.CsWin32/Generator.Struct.cs +++ b/src/Microsoft.Windows.CsWin32/Generator.Struct.cs @@ -118,9 +118,26 @@ private StructDeclarationSyntax DeclareStruct(TypeDefinitionHandle typeDefHandle CustomAttributeValue attributeArgs = fixedBufferAttribute.Value.DecodeValue(CustomAttributeTypeProvider.Instance); var fieldType = (TypeSyntax)attributeArgs.FixedArguments[0].Value!; ExpressionSyntax size = LiteralExpression(SyntaxKind.NumericLiteralExpression, Literal((int)attributeArgs.FixedArguments[1].Value!)); - field = FieldDeclaration( - [TokenWithSpace(this.Visibility), TokenWithSpace(SyntaxKind.UnsafeKeyword), Token(SyntaxKind.FixedKeyword)], - VariableDeclaration(fieldType, [fieldDeclarator.WithArgumentList(BracketedArgumentList(Argument(size)))])); + if (this.FindAssociatedEnum(fieldDef.GetCustomAttributes()) is HandleTypeHandleInfo enumType) + { + PrimitiveTypeCode underlyingType = GetEnumUnderlyingType(enumType); + if (fieldType.ToString() != $"System.{underlyingType}" + && !fieldType.IsEquivalentTo(PrimitiveTypeHandleInfo.ToTypeSyntax(underlyingType, typeSettings.PreferNativeInt))) + { + throw new GenerationFailedException($"AssociatedEnum \"{enumType}\" cannot reinterpret fixed-buffer element type {fieldType}."); + } + + TypeSyntax elementType = GetAssociatedEnumTypeSyntax(enumType, typeSettings, GeneratingElement.StructMember); + var fieldInfo = this.ReinterpretFieldType(fieldDef, ArrayType(elementType, [ArrayRankSpecifier([size])]), fieldDef.GetCustomAttributes(), context); + additionalMembers = additionalMembers.AddRange(fieldInfo.AdditionalMembers); + field = FieldDeclaration([TokenWithSpace(this.Visibility)], VariableDeclaration(fieldInfo.FieldType, [fieldDeclarator])); + } + else + { + field = FieldDeclaration( + [TokenWithSpace(this.Visibility), TokenWithSpace(SyntaxKind.UnsafeKeyword), Token(SyntaxKind.FixedKeyword)], + VariableDeclaration(fieldType, [fieldDeclarator.WithArgumentList(BracketedArgumentList(Argument(size)))])); + } } else if (fieldDefHandle == flexibleArrayFieldHandle) { @@ -186,24 +203,19 @@ private StructDeclarationSyntax DeclareStruct(TypeDefinitionHandle typeDefHandle } } - TypeSyntaxAndMarshaling fieldTypeSyntax = fieldTypeInfo.ToTypeSyntax(thisFieldTypeSettings, GeneratingElement.StructMember, fieldAttributes.QualifyWith(this)); + HandleTypeHandleInfo? propertyEnum = this.FindStoragePreservingEnumProperty(fieldTypeInfo, fieldAttributes); + TypeSyntaxAndMarshaling fieldTypeSyntax = fieldTypeInfo.ToTypeSyntax(thisFieldTypeSettings, GeneratingElement.StructMember, propertyEnum is null ? fieldAttributes.QualifyWith(this) : null); (TypeSyntax FieldType, SyntaxList AdditionalMembers, AttributeSyntax? MarshalAsAttribute) fieldInfo = this.ReinterpretFieldType(fieldDef, fieldTypeSyntax.Type, fieldAttributes, context); additionalMembers = additionalMembers.AddRange(fieldInfo.AdditionalMembers); - PropertyDeclarationSyntax? property = null; - if (this.FindAssociatedEnum(fieldAttributes) is { } propertyType) + if (propertyEnum is not null) { - // Keep the field with its original type, but then add a property that returns the enum type. + // A real native enum signature fixes this enum's storage; retain the existing scalar property without changing either ABI. + TypeSyntax propertyType = GetAssociatedEnumTypeSyntax(propertyEnum, thisFieldTypeSettings, GeneratingElement.StructMember); fieldDeclarator = VariableDeclarator(SafeIdentifier($"_{fieldName}")); field = FieldDeclaration([TokenWithSpace(SyntaxKind.PrivateKeyword)], VariableDeclaration(fieldInfo.FieldType, [fieldDeclarator])); - - // internal EnumType FieldName { - // get => (EnumType)this._fieldName; - // set => this._fieldName = (UnderlyingType)value; - // } - this.RequestInteropType(this.GetNamespaceForPossiblyNestedType(typeDef), propertyType.Identifier.ValueText, context); ExpressionSyntax fieldAccess = MemberAccessExpression(SyntaxKind.SimpleMemberAccessExpression, ThisExpression(), IdentifierName(fieldDeclarator.Identifier)); - property = PropertyDeclaration(propertyType.WithTrailingTrivia(Space), Identifier(fieldName).WithTrailingTrivia(LineFeed)) + PropertyDeclarationSyntax property = PropertyDeclaration(propertyType.WithTrailingTrivia(Space), Identifier(fieldName).WithTrailingTrivia(LineFeed)) .AddModifiers(TokenWithSpace(this.Visibility)) .WithAccessorList(AccessorList( AccessorDeclaration(SyntaxKind.GetAccessorDeclaration).WithExpressionBody(ArrowExpressionClause(CastExpression(propertyType, fieldAccess))).WithSemicolonToken(Semicolon), @@ -223,7 +235,6 @@ private StructDeclarationSyntax DeclareStruct(TypeDefinitionHandle typeDefHandle if (this.HasObsoleteAttribute(fieldDef.GetCustomAttributes())) { field = field.AddAttributeLists(AttributeList(ObsoleteAttributeSyntax)); - property = property?.AddAttributeLists(AttributeList(ObsoleteAttributeSyntax)); } if (RequiresUnsafe(fieldInfo.FieldType)) @@ -813,13 +824,6 @@ private void GatherFlattenedAnonymousAccessors( accessors.Add(this.CreateFlattenedBitfieldAccessor(bitfieldPropName, bitfieldPropType, accessPrefix, crefTypePrefix, fieldObsolete)); } - // Skip fields whose generated representation is not a plain, ref-returnable field of the decoded type. - if (this.FindAssociatedEnum(fieldAttributes) is not null) - { - // Surfaced as a by-value property rather than a field. - continue; - } - if (this.FindAttribute(fieldAttributes, SystemRuntimeCompilerServices, nameof(FixedBufferAttribute)).HasValue) { continue; @@ -831,6 +835,10 @@ private void GatherFlattenedAnonymousAccessors( } TypeHandleInfo fieldTypeInfo = fieldDef.DecodeSignature(this.SignatureHandleProvider, null); + if (this.FindStoragePreservingEnumProperty(fieldTypeInfo, fieldAttributes) is not null) + { + continue; + } // Skip field types that are reinterpreted during struct generation (delegates become function pointers, // COM interface pointers become arrays, marshaled managed types become unmanaged pointers); the generated diff --git a/src/Microsoft.Windows.CsWin32/Generator.cs b/src/Microsoft.Windows.CsWin32/Generator.cs index 7292b64a..467f3cec 100644 --- a/src/Microsoft.Windows.CsWin32/Generator.cs +++ b/src/Microsoft.Windows.CsWin32/Generator.cs @@ -1959,7 +1959,7 @@ private IEnumerable EnumerateAccessibleAssemblies() else if (this.Reader.StringComparer.Equals(baseTypeName, nameof(Enum)) && this.Reader.StringComparer.Equals(baseTypeNamespace, nameof(System))) { // Consider reusing .NET types like FILE_SHARE_FLAGS -> System.IO.FileShare - typeDeclaration = this.DeclareEnum(typeDef); + typeDeclaration = this.DeclareEnum(typeDefHandle, typeDef); } else if (this.Reader.StringComparer.Equals(baseTypeName, nameof(MulticastDelegate)) && this.Reader.StringComparer.Equals(baseTypeNamespace, nameof(System))) { diff --git a/src/Microsoft.Windows.CsWin32/HandleTypeHandleInfo.cs b/src/Microsoft.Windows.CsWin32/HandleTypeHandleInfo.cs index a7c5316e..8ed9d5ad 100644 --- a/src/Microsoft.Windows.CsWin32/HandleTypeHandleInfo.cs +++ b/src/Microsoft.Windows.CsWin32/HandleTypeHandleInfo.cs @@ -29,6 +29,11 @@ internal HandleTypeHandleInfo(Generator generator, MetadataReader reader, Entity internal byte? RawTypeKind { get; } + /// + /// Gets a value indicating whether this type references a logical enum rather than a native enum signature. + /// + internal bool IsAssociatedEnumProjection { get; init; } + public override string ToString() => this.ToTypeSyntaxForDisplay().ToString(); internal static NameSyntax GetNestingQualifiedName(Generator? generator, MetadataReader reader, TypeDefinition td, bool hasUnmanagedSuffix, bool isInterfaceNestedInStruct) @@ -78,6 +83,28 @@ internal bool IsType(string leafName) internal override TypeSyntaxAndMarshaling ToTypeSyntax(TypeSyntaxSettings inputs, Generator.GeneratingElement forElement, QualifiedCustomAttributeHandleCollection? customAttributes, ParameterAttributes parameterAttributes = default) { + if (customAttributes?.Generator.FindAssociatedEnum(customAttributes.Value.Collection) is HandleTypeHandleInfo enumType + && (!this.generator.TryGetTypeDefHandle(this.Handle, out QualifiedTypeDefinitionHandle definition) + || definition.Generator != enumType.Generator || definition.DefinitionHandle != enumType.Handle)) + { + throw new GenerationFailedException($"AssociatedEnum \"{enumType}\" cannot be projected over native type \"{this}\"."); + } + + if (!this.IsAssociatedEnumProjection && inputs.Generator is not null + && this.generator.TryGetTypeDefHandle(this.Handle, out QualifiedTypeDefinitionHandle nativeDefinition)) + { + TypeDefinition nativeType = nativeDefinition.Resolve().Definition; + nativeDefinition.Generator.GetBaseTypeInfo(nativeType, out StringHandle baseName, out StringHandle baseNamespace); + if (nativeDefinition.Reader.StringComparer.Equals(baseName, nameof(Enum)) && nativeDefinition.Reader.StringComparer.Equals(baseNamespace, nameof(System))) + { + HandleTypeHandleInfo nativeEnum = new(nativeDefinition.Generator, nativeDefinition.Reader, nativeDefinition.DefinitionHandle); + if (Generator.GetEnumUnderlyingType(nativeEnum) != Generator.GetDeclaredEnumUnderlyingType(nativeEnum)) + { + throw new GenerationFailedException($"Native enum {nativeDefinition.Reader.GetString(nativeType.Name)} retains its metadata storage, which conflicts with its AssociatedEnum use sites."); + } + } + } + NameSyntax? nameSyntax; bool isInterface; bool isNonCOMConformingInterface; diff --git a/src/Microsoft.Windows.CsWin32/MetadataIndex.cs b/src/Microsoft.Windows.CsWin32/MetadataIndex.cs index bf6a039e..e6e21cc8 100644 --- a/src/Microsoft.Windows.CsWin32/MetadataIndex.cs +++ b/src/Microsoft.Windows.CsWin32/MetadataIndex.cs @@ -51,6 +51,9 @@ internal class MetadataIndex /// private readonly ConcurrentDictionary enumValueLookupCache = new(StringComparer.Ordinal); + private readonly Lazy>> associatedEnumAttributes; + private readonly Lazy> nativeSignatureTypes; + /// /// A lazily computed reference to System.Enum, as defined by this metadata. /// Should be retrieved by . @@ -66,6 +69,44 @@ internal MetadataIndex(MetadataFile metadataFile, Platform? platform) { this.metadataFile = metadataFile; this.platform = platform; + this.associatedEnumAttributes = new(() => + { + using MetadataFile.Rental rental = metadataFile.GetMetadataReader(); + MetadataReader reader = rental.Value; + Dictionary> attributes = new(StringComparer.Ordinal); + Dictionary constructors = new(); + foreach (CustomAttributeHandle handle in reader.CustomAttributes) + { + CustomAttribute attribute = reader.GetCustomAttribute(handle); + if (attribute.Parent.Kind is not (HandleKind.FieldDefinition or HandleKind.Parameter)) + { + continue; + } + + if (!constructors.TryGetValue(attribute.Constructor, out bool associatedEnum)) + { + associatedEnum = MetadataUtilities.IsAttribute(reader, attribute, Generator.InteropDecorationNamespace, Generator.AssociatedEnumAttribute); + constructors.Add(attribute.Constructor, associatedEnum); + } + + if (!associatedEnum + || attribute.DecodeValue(CustomAttributeTypeProvider.Instance).FixedArguments is not [{ Value: string name }]) + { + continue; + } + + string leafName = name.Substring(name.LastIndexOf('.') + 1); + if (!attributes.TryGetValue(leafName, out List? matches)) + { + attributes.Add(leafName, matches = new()); + } + + matches.Add(handle); + } + + return attributes.ToDictionary(pair => pair.Key, pair => pair.Value.ToImmutableArray(), StringComparer.Ordinal); + }); + this.nativeSignatureTypes = new(this.CollectNativeSignatureTypes); using MetadataFile.Rental mrRental = metadataFile.GetMetadataReader(); MetadataReader mr = mrRental.Value; @@ -253,6 +294,20 @@ void PopulateNamespace(NamespaceDefinition ns, string? parentNamespace) private string DebuggerDisplay => $"{this.metadataFile.Path} ({this.platform})"; + /// + /// Gets enum associations indexed by the target's unqualified name without retaining a metadata reader. + /// + /// The unqualified enum name. + /// The matching annotation handles. + internal ImmutableArray GetAssociatedEnumAttributes(string name) => + this.associatedEnumAttributes.Value.TryGetValue(name, out ImmutableArray attributes) ? attributes : []; + + /// + /// Gets the types explicitly referenced by compatible native fields, methods, and callbacks. + /// + /// Reader-independent type definition and reference handles. + internal IReadOnlyCollection GetNativeSignatureTypes() => this.nativeSignatureTypes.Value; + /// /// Attempts to translate a to a . /// @@ -444,4 +499,113 @@ private static string CommonPrefix(IReadOnlyList ss) // Return null if the value was determined to be missing. return this.enumTypeReference.HasValue && !this.enumTypeReference.Value.IsNil ? this.enumTypeReference.Value : null; } + + private HashSet CollectNativeSignatureTypes() + { + using MetadataFile.Rental rental = this.metadataFile.GetMetadataReader(); + MetadataReader reader = rental.Value; + NativeSignatureTypeProvider provider = new(); + foreach (TypeDefinitionHandle handle in reader.TypeDefinitions) + { + TypeDefinition type = reader.GetTypeDefinition(handle); + if (!MetadataUtilities.IsCompatibleWithPlatform(reader, this, this.platform, type.GetCustomAttributes())) + { + continue; + } + + string? baseName = null; + if (type.BaseType.Kind == HandleKind.TypeReference) + { + TypeReference baseType = reader.GetTypeReference((TypeReferenceHandle)type.BaseType); + if (reader.StringComparer.Equals(baseType.Namespace, nameof(System))) + { + baseName = reader.GetString(baseType.Name); + } + } + + bool nativeMethods = (type.Attributes & TypeAttributes.Interface) != 0 || baseName == nameof(MulticastDelegate); + if (baseName == nameof(ValueType)) + { + foreach (FieldDefinitionHandle fieldHandle in type.GetFields()) + { + FieldDefinition field = reader.GetFieldDefinition(fieldHandle); + if ((field.Attributes & FieldAttributes.Static) == 0 && MetadataUtilities.IsCompatibleWithPlatform(reader, this, this.platform, field.GetCustomAttributes())) + { + field.DecodeSignature(provider, null); + } + } + } + + foreach (MethodDefinitionHandle methodHandle in type.GetMethods()) + { + MethodDefinition method = reader.GetMethodDefinition(methodHandle); + if ((nativeMethods || (method.Attributes & MethodAttributes.PinvokeImpl) != 0) + && MetadataUtilities.IsCompatibleWithPlatform(reader, this, this.platform, method.GetCustomAttributes())) + { + method.DecodeSignature(provider, null); + } + } + } + + return provider.Types; + } + + private sealed class NativeSignatureTypeProvider : ISignatureTypeProvider + { + /// + /// Gets the type handles referenced in the decoded signatures. + /// + internal HashSet Types { get; } = new(); + + /// + public object? GetTypeFromDefinition(MetadataReader reader, TypeDefinitionHandle handle, byte rawTypeKind) + { + this.Types.Add(handle); + return null; + } + + /// + public object? GetTypeFromReference(MetadataReader reader, TypeReferenceHandle handle, byte rawTypeKind) + { + this.Types.Add(handle); + return null; + } + + /// + public object? GetTypeFromSpecification(MetadataReader reader, object? genericContext, TypeSpecificationHandle handle, byte rawTypeKind) => + reader.GetTypeSpecification(handle).DecodeSignature(this, genericContext); + + /// + public object? GetArrayType(object? elementType, ArrayShape shape) => null; + + /// + public object? GetSZArrayType(object? elementType) => null; + + /// + public object? GetPointerType(object? elementType) => null; + + /// + public object? GetPrimitiveType(PrimitiveTypeCode typeCode) => null; + + /// + public object? GetByReferenceType(object? elementType) => null; + + /// + public object? GetFunctionPointerType(MethodSignature signature) => null; + + /// + public object? GetGenericInstantiation(object? genericType, ImmutableArray typeArguments) => null; + + /// + public object? GetGenericMethodParameter(object? genericContext, int index) => null; + + /// + public object? GetGenericTypeParameter(object? genericContext, int index) => null; + + /// + public object? GetModifiedType(object? modifier, object? unmodifiedType, bool isRequired) => null; + + /// + public object? GetPinnedType(object? elementType) => null; + } } diff --git a/src/Microsoft.Windows.CsWin32/PointerTypeHandleInfo.cs b/src/Microsoft.Windows.CsWin32/PointerTypeHandleInfo.cs index b2795747..24322e57 100644 --- a/src/Microsoft.Windows.CsWin32/PointerTypeHandleInfo.cs +++ b/src/Microsoft.Windows.CsWin32/PointerTypeHandleInfo.cs @@ -31,6 +31,13 @@ internal override TypeSyntaxAndMarshaling ToTypeSyntax(TypeSyntaxSettings inputs mustUsePointers |= inputs.IsReturnValue; mustUsePointers |= customAttributes is { } attributes && attributes.Generator.FindInteropDecorativeAttribute(attributes.Collection, Generator.ContainsInteriorPointersAttribute) is not null; + if (customAttributes is { } enumAttributes) + { + enumAttributes.Generator.ValidateAssociatedEnumElementType( + this.ElementType, + enumAttributes.Collection); + } + if (mustUsePointers) { // These pointers require caller-controlled storage rather than a temporary managed reference or marshaled allocation. @@ -48,7 +55,9 @@ inputs with TypeSyntaxAndMarshaling elementTypeDetails = this.ElementType.ToTypeSyntax(inputs with { PreferInOutRef = false }, forElement, customAttributes, parameterAttributes); if (elementTypeDetails.MarshalAsAttribute is object || elementTypeDetails.MarshalUsingType is string || - (inputs.Generator?.IsManagedType(this.ElementType) is true) || (inputs.PreferInOutRef && !xOptional && this.ElementType is PrimitiveTypeHandleInfo { PrimitiveTypeCode: not PrimitiveTypeCode.Void })) + (inputs.Generator?.IsManagedType(this.ElementType) is true) || (inputs.PreferInOutRef && !xOptional + && this.ElementType is PrimitiveTypeHandleInfo { PrimitiveTypeCode: not PrimitiveTypeCode.Void } + && customAttributes?.Generator.FindAssociatedEnum(customAttributes.Value.Collection) is null)) { bool xIn = (parameterAttributes & ParameterAttributes.In) == ParameterAttributes.In; bool xOut = (parameterAttributes & ParameterAttributes.Out) == ParameterAttributes.Out; diff --git a/src/Microsoft.Windows.CsWin32/PrimitiveTypeHandleInfo.cs b/src/Microsoft.Windows.CsWin32/PrimitiveTypeHandleInfo.cs index 55f04773..aba75e1d 100644 --- a/src/Microsoft.Windows.CsWin32/PrimitiveTypeHandleInfo.cs +++ b/src/Microsoft.Windows.CsWin32/PrimitiveTypeHandleInfo.cs @@ -9,26 +9,16 @@ internal record PrimitiveTypeHandleInfo(PrimitiveTypeCode PrimitiveTypeCode) : T internal override TypeSyntaxAndMarshaling ToTypeSyntax(TypeSyntaxSettings inputs, Generator.GeneratingElement forElement, QualifiedCustomAttributeHandleCollection? customAttributes, ParameterAttributes parameterAttributes) { - // We want to expose the enum type when there is one, but doing it *properly* requires marshaling to the underlying type. - // The length of the enum may differ from the length of the primitive type, which means we can't just use the enum type directly. - // Even if the lengths match, if the enum's underlying base type conflicts with the primitive's signed type (e.g. int != uint), - // certain CPU architectures can fail as well. - // So we just have to use the primitive type when marshaling is not allowed. - if (inputs.AllowMarshaling && customAttributes?.Generator.FindAssociatedEnum(customAttributes.Value.Collection) is NameSyntax enumTypeName && inputs.Generator!.TryGenerateType(enumTypeName.ToString(), out IReadOnlyCollection preciseMatch)) + if (customAttributes?.Generator.FindAssociatedEnum(customAttributes.Value.Collection) is HandleTypeHandleInfo enumType) { - // Use the qualified name. - enumTypeName = ParseName(Generator.ReplaceCommonNamespaceWithAlias(inputs.Generator, preciseMatch.First())); - UnmanagedType unmanagedType = GetUnmanagedType(this.PrimitiveTypeCode); - - // If marshaling using source generators, we need to generate a custom marshaler and a MarshalUsing(...) attribute. - if (inputs.AllowMarshaling && inputs.Generator?.UseSourceGenerators == true) + if (customAttributes.Value.Generator.RequiresAssociatedEnumConversion(enumType, this.PrimitiveTypeCode) + && forElement is Generator.GeneratingElement.ExternMethod or Generator.GeneratingElement.InterfaceAsStructMember) { - string marshalerTypeName = inputs.Generator!.RequestCustomEnumMarshaller(preciseMatch.First(), unmanagedType); - return new TypeSyntaxAndMarshaling(enumTypeName) { MarshalUsingType = marshalerTypeName }; + return new TypeSyntaxAndMarshaling(ToTypeSyntax(this.PrimitiveTypeCode, inputs.PreferNativeInt)); } - MarshalAsAttribute marshalAs = new(unmanagedType); - return new TypeSyntaxAndMarshaling(enumTypeName, marshalAs, nativeArrayInfo: null); + customAttributes.Value.Generator.ValidateAssociatedEnumStorage(enumType, this.PrimitiveTypeCode); + return new TypeSyntaxAndMarshaling(Generator.GetAssociatedEnumTypeSyntax(enumType, inputs, forElement)); } else { @@ -71,20 +61,4 @@ internal static TypeSyntax ToTypeSyntax(PrimitiveTypeCode typeCode, bool preferN _ => throw new NotSupportedException("Unsupported type code: " + typeCode), }; } - - private static UnmanagedType GetUnmanagedType(PrimitiveTypeCode typeCode) - { - return typeCode switch - { - PrimitiveTypeCode.SByte => UnmanagedType.I1, - PrimitiveTypeCode.Byte => UnmanagedType.U1, - PrimitiveTypeCode.Int16 => UnmanagedType.I2, - PrimitiveTypeCode.UInt16 => UnmanagedType.U2, - PrimitiveTypeCode.Int32 => UnmanagedType.I4, - PrimitiveTypeCode.UInt32 => UnmanagedType.U4, - PrimitiveTypeCode.Int64 => UnmanagedType.I8, - PrimitiveTypeCode.UInt64 => UnmanagedType.U8, - _ => throw new NotSupportedException("Unsupported type code: " + typeCode), - }; - } } diff --git a/test/Microsoft.Windows.CsWin32.Tests/AssociatedEnumTests.cs b/test/Microsoft.Windows.CsWin32.Tests/AssociatedEnumTests.cs new file mode 100644 index 00000000..e86744d7 --- /dev/null +++ b/test/Microsoft.Windows.CsWin32.Tests/AssociatedEnumTests.cs @@ -0,0 +1,761 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +namespace Microsoft.Windows.CsWin32.Tests; + +/// +/// Verifies logical enums without substituting them into native metadata signatures. +/// +public class AssociatedEnumTests : GeneratorTestBase +{ + /// + /// The canonical metadata declarations, using native primitives with logical enum associations. + /// + internal const string MetadataSource = """ + using System; + using System.Runtime.InteropServices; + using Windows.Win32.Foundation.Metadata; + using MODE_STORAGE = System.UInt32; + + namespace Windows.Win32.Foundation.Metadata + { + public sealed class AssociatedEnumAttribute : Attribute + { + public AssociatedEnumAttribute(string name) { } + } + public sealed class NativeArrayInfoAttribute : Attribute + { + public short CountParamIndex; + public int CountConst; + } + } + namespace Windows.Win32.Test + { + [Flags] + public enum MODE : uint { First = 1, Second = 2, High = 0x80000000 } + [Flags] + public enum NARROW_MODE : uint { First = 1, Second = 2, High = 0x8000 } + public unsafe struct RECORD + { + [AssociatedEnum("MODE")] public MODE_STORAGE Mode; + [AssociatedEnum("Windows.Win32.Test.MODE")] public MODE_STORAGE* Modes; + } + public unsafe struct FIXED_RECORD + { + [AssociatedEnum("MODE")] public fixed uint Modes[2]; + } + public struct NARROW_RECORD + { + [AssociatedEnum("NARROW_MODE")] public ushort Mode; + public ushort Tail; + } + [UnmanagedFunctionPointer(CallingConvention.StdCall)] + [return: AssociatedEnum("MODE")] + public unsafe delegate MODE_STORAGE CALLBACK([AssociatedEnum("MODE")] MODE_STORAGE mode, [AssociatedEnum("MODE")] MODE_STORAGE* data); + public static unsafe class Apis + { + [DllImport("fixture.dll", ExactSpelling = true)] + [return: AssociatedEnum("MODE")] + public static extern MODE_STORAGE RoundTrip([AssociatedEnum("MODE")] MODE_STORAGE mode); + [DllImport("fixture.dll", ExactSpelling = true)] + [return: AssociatedEnum("Windows.Win32.Test.MODE")] + public static extern MODE_STORAGE* PointerReturn(); + [DllImport("fixture.dll", ExactSpelling = true)] + public static extern void Buffer([In, AssociatedEnum("MODE"), NativeArrayInfo(CountParamIndex = 1)] MODE_STORAGE* modes, uint count); + [DllImport("fixture.dll", ExactSpelling = true)] + public static extern void Output([Out, AssociatedEnum("MODE")] MODE_STORAGE* mode); + [DllImport("fixture.dll", ExactSpelling = true)] + public static extern void DoublePointer([Out, AssociatedEnum("MODE")] MODE_STORAGE** modes); + [DllImport("fixture.dll", ExactSpelling = true)] + public static extern void UseCallback(CALLBACK callback); + [DllImport("fixture.dll", ExactSpelling = true)] + public static extern void Narrow([AssociatedEnum("NARROW_MODE")] ushort mode); + [DllImport("fixture.dll", ExactSpelling = true)] + [return: AssociatedEnum("NARROW_MODE")] + public static extern ushort NarrowReturn(); + [DllImport("fixture.dll", ExactSpelling = true)] + public static extern void NarrowPointer([AssociatedEnum("NARROW_MODE")] ushort* mode); + [DllImport("fixture.dll", ExactSpelling = true)] + public static extern void Unannotated(uint mode); + } + } + """; + + /// + /// The unchanged caller shared by the canonical compatibility legs and mutation controls. + /// + internal const string Caller = """ + using System; + using System.Runtime.InteropServices; + using Windows.Win32; + using Windows.Win32.Test; + static unsafe class Caller + { + static void Use(MODE* modes, MODE** indirect, RECORD record) + { + MODE flags = MODE.First | MODE.Second | MODE.High; + MODE result = PInvoke.RoundTrip(flags); + record.Mode = flags; + ref MODE field = ref record.Mode; + MODE* fields = record.Modes; + PInvoke.Buffer(modes, 2); + PInvoke.Buffer(new ReadOnlySpan(modes, 2)); + PInvoke.Output(modes); + PInvoke.Output(out MODE output); + PInvoke.DoublePointer(indirect); + MODE* returned = PInvoke.PointerReturn(); + PInvoke.Unannotated(42); + } + } + """; + + /// + /// Initializes a new instance of the class. + /// + /// The test output. + public AssociatedEnumTests(ITestOutputHelper logger) + : base(logger) + { + } + + /// + /// Compiles unchanged enum, flags, ref-field, buffer and pointer callers against old and new metadata. + /// + /// Whether enums occur directly in the native metadata signatures. + /// Whether runtime marshaling is enabled. + [Theory, CombinatorialData] + public void NativeSignaturesRecoverLogicalEnumCallers(bool legacy, bool allowMarshaling) + { + this.GenerateFixture(legacy, allowMarshaling, "RECORD", "RoundTrip", "PointerReturn", "Buffer", "Output", "DoublePointer", "UseCallback", "Unannotated"); + this.compilation = this.AddCode(Caller); + string callback = allowMarshaling + ? "static void Register() => PInvoke.UseCallback(Callback); static MODE Callback(MODE mode, MODE* data) => mode;" + : "static void Register() => PInvoke.UseCallback(&Callback); [UnmanagedCallersOnly(CallConvs = new[] { typeof(System.Runtime.CompilerServices.CallConvStdcall) })] static MODE Callback(MODE mode, MODE* data) => mode;"; + this.compilation = this.AddCode("using System.Runtime.InteropServices; using Windows.Win32; using Windows.Win32.Test; static unsafe class CallbackCaller { " + callback + " }"); + this.AssertNoDiagnostics(logAllGeneratedCode: false); + INamedTypeSymbol record = this.compilation.GetTypeByMetadataName("Windows.Win32.Test.RECORD")!; + Assert.IsAssignableFrom(Assert.Single(record.GetMembers("Mode"))); + Assert.Equal("Windows.Win32.Test.MODE", this.compilation.GetTypeByMetadataName("Windows.Win32.Test.MODE")!.ToDisplayString()); + Assert.All(this.FindGeneratedMethod("DoublePointer"), method => Assert.Contains("MODE*", method.ParameterList.ToString(), StringComparison.Ordinal)); + } + + /// + /// Verifies exact storage for all legal enum underlying types, independent of marshaling and CPU. + /// + /// The native use site's storage type. + /// Whether runtime marshaling is enabled. + /// The target CPU. + [Theory, CombinatorialData] + public void ExactIntegralStorage( + [CombinatorialValues("sbyte", "byte", "short", "ushort", "int", "uint", "long", "ulong")] string storage, + bool allowMarshaling, + [CombinatorialValues(Platform.X86, Platform.X64, Platform.Arm64)] Platform platform) + { + string source = GetNativeStorageMetadata(storage, legacy: false); + this.GenerateSource(source, allowMarshaling, platform, "RECORD", "RoundTrip", "PointerReturn", "Output", "DoublePointer", "Buffer", "Unannotated"); + this.compilation = this.AddCode(Caller); + this.AssertNoDiagnostics(logAllGeneratedCode: false); + IMethodSymbol roundTrip = Assert.Single(this.compilation.GetTypeByMetadataName("Windows.Win32.PInvoke")!.GetMembers("RoundTrip").OfType()); + Assert.Equal(storage, ((INamedTypeSymbol)roundTrip.ReturnType).EnumUnderlyingType!.ToDisplayString()); + } + + /// + /// Verifies inline buffers and direct enum fields whose storage comes from their native use sites. + /// + /// Whether runtime marshaling is enabled. + [Theory] + [InlineData(true)] + [InlineData(false)] + public void FieldStorageAndFixedBuffers(bool allowMarshaling) + { + this.GenerateFixture(false, allowMarshaling, "FIXED_RECORD", "NARROW_RECORD", "Narrow"); + this.compilation = this.AddCode(""" + using Windows.Win32; + using Windows.Win32.Test; + /// Exercises native field storage. + public static unsafe class Layout + { + /// Returns both layout sizes. + public static int Run() + { + FIXED_RECORD record = default; + record.Modes.AsSpan()[0] = MODE.First | MODE.High; + NARROW_RECORD narrow = default; + checked { narrow.Mode = NARROW_MODE.High | NARROW_MODE.Second; } + ref NARROW_MODE mode = ref narrow.Mode; + if (record.Modes.AsSpan()[0] != (MODE.First | MODE.High) || mode != (NARROW_MODE.High | NARROW_MODE.Second)) throw new System.Exception(); + return sizeof(FIXED_RECORD) * 100 + sizeof(NARROW_RECORD); + } + static void Compile() => PInvoke.Narrow(NARROW_MODE.Second); + } + """); + this.AssertNoDiagnostics(logAllGeneratedCode: false); + using MemoryStream stream = new(); + var emitted = this.compilation.Emit(stream, cancellationToken: TestContext.Current.CancellationToken); + Assert.True(emitted.Success, string.Join(Environment.NewLine, emitted.Diagnostics)); + Assembly assembly = Assembly.Load(stream.ToArray()); + Assert.Equal(804, assembly.GetType("Layout")!.GetMethod("Run")!.Invoke(null, null)); + } + + /// + /// Verifies that unsupported associations fail instead of producing native-only or invalid projections. + /// + /// The invalid association or native shape. + /// The requested API. + /// The expected diagnostic detail. + [Theory] + [InlineData("Missing", "RoundTrip", "was not found")] + [InlineData("RECORD", "RoundTrip", "does not refer to an enum")] + [InlineData("", "RoundTrip", "nonempty enum name")] + public void InvalidAssociationsFailExplicitly(string replacement, string api, string message) + { + string source = MetadataSource.Replace("AssociatedEnum(\"MODE\")", $"AssociatedEnum(\"{replacement}\")", StringComparison.Ordinal); + GenerationFailedException exception = Assert.Throws(() => this.GenerateSource(source, false, Platform.X64, api)); + Assert.Contains(message, exception.ToString(), StringComparison.Ordinal); + } + + /// + /// Rejects incompatible widths or signedness for one enum independently of generation order. + /// + /// The first requested enum or consumer. + [Theory] + [InlineData("MODE")] + [InlineData("RoundTrip")] + [InlineData("RECORD")] + public void ConflictingNativeStorageFailsDeterministically(string firstRequest) + { + const string ConflictMethod = """ + public static extern void Unannotated(uint mode); + [DllImport("fixture.dll", ExactSpelling = true)] + public static extern void Conflict([AssociatedEnum("MODE")] int mode); + """; + string source = MetadataSource.Replace("public static extern void Unannotated(uint mode);", ConflictMethod, StringComparison.Ordinal); + GenerationFailedException exception = Assert.Throws(() => this.GenerateSource(source, false, Platform.X64, firstRequest)); + Assert.Contains("conflicting native storage", exception.ToString(), StringComparison.Ordinal); + Assert.Contains("UInt32", exception.ToString(), StringComparison.Ordinal); + Assert.Contains("Int32", exception.ToString(), StringComparison.Ordinal); + Assert.Contains("Apis.Conflict::mode", exception.ToString(), StringComparison.Ordinal); + } + + /// + /// Does not silently truncate logical members when narrowing the generated enum's native storage. + /// + [Fact] + public void OutOfRangeMemberFailsExplicitly() + { + string source = MetadataSource.Replace("High = 0x8000 }", "High = 0x10000 }", StringComparison.Ordinal); + GenerationFailedException exception = Assert.Throws(() => this.GenerateSource(source, false, Platform.X64, "Narrow")); + Assert.Contains("High=65536 does not fit native UInt16 storage without truncation", exception.ToString(), StringComparison.Ordinal); + } + + /// + /// Rejects nonintegral native use sites instead of selecting a logical enum's arbitrary metadata storage. + /// + [Fact] + public void NonintegralNativeStorageFailsExplicitly() + { + string source = MetadataSource.Replace("public static extern MODE_STORAGE RoundTrip", "public static extern float RoundTrip", StringComparison.Ordinal); + GenerationFailedException exception = Assert.Throws(() => this.GenerateSource(source, false, Platform.X64, "RoundTrip")); + Assert.Contains("integral native storage", exception.ToString(), StringComparison.Ordinal); + } + + /// + /// Retains real native enum ABI and raw scalar callers while offering explicit enum overloads. + /// + /// Whether runtime marshaling is enabled. + [Theory] + [InlineData(true)] + [InlineData(false)] + public void NativeEnumSignatureRetainsItsStorage(bool allowMarshaling) + { + const string NativeEnumMethod = """ + public static extern void Unannotated(uint mode); + [DllImport("fixture.dll", ExactSpelling = true)] + public static extern void NativeEnum(NARROW_MODE mode); + """; + string source = MetadataSource.Replace("public static extern void Unannotated(uint mode);", NativeEnumMethod, StringComparison.Ordinal); + this.GenerateSource(source, allowMarshaling, Platform.X64, "NARROW_MODE", "NativeEnum", "Narrow", "NarrowReturn"); + this.AssertNoDiagnostics(); + INamedTypeSymbol mode = this.compilation.GetTypeByMetadataName("Windows.Win32.Test.NARROW_MODE")!; + Assert.Equal(SpecialType.System_UInt32, mode.EnumUnderlyingType!.SpecialType); + this.compilation = this.AddCode("using Windows.Win32; using Windows.Win32.Test; static class ExistingScalarCaller { static void Raw(ushort value) => PInvoke.Narrow(value); static void Enum(NARROW_MODE value) => PInvoke.Narrow(value); static ushort RawReturn() => PInvoke.NarrowReturn(); static NARROW_MODE EnumReturn() => PInvoke.NarrowReturn_Enum(); }"); + this.AssertNoDiagnostics(); + } + + /// + /// Rejects reinterpreting an indirect association whose enum storage is fixed by real native enum signatures. + /// + [Fact] + public void NativeEnumIndirectConflictFailsExplicitly() + { + const string NativeEnumField = "namespace Windows.Win32.Test { public struct NATIVE_RECORD { public NARROW_MODE Value; } }"; + GenerationFailedException exception = Assert.Throws(() => this.GenerateSource( + MetadataSource + Environment.NewLine + NativeEnumField, false, Platform.X64, "NarrowPointer")); + Assert.Contains("conflicting native storage", exception.ToString(), StringComparison.Ordinal); + } + + /// + /// Preserves the existing scalar property when a real native enum signature fixes different storage. + /// + /// Whether runtime marshaling is enabled. + [Theory] + [InlineData(true)] + [InlineData(false)] + public void NativeEnumFieldRetainsExistingScalarProperty(bool allowMarshaling) + { + const string NativeEnumField = "namespace Windows.Win32.Test { public struct NATIVE_RECORD { public NARROW_MODE Value; } }"; + this.GenerateSource(MetadataSource + Environment.NewLine + NativeEnumField, allowMarshaling, Platform.X64, "NARROW_RECORD", "NATIVE_RECORD"); + INamedTypeSymbol mode = this.compilation.GetTypeByMetadataName("Windows.Win32.Test.NARROW_MODE")!; + Assert.Equal(SpecialType.System_UInt32, mode.EnumUnderlyingType!.SpecialType); + INamedTypeSymbol record = this.compilation.GetTypeByMetadataName("Windows.Win32.Test.NARROW_RECORD")!; + Assert.Equal(mode, Assert.Single(record.GetMembers("Mode").OfType()).Type, SymbolEqualityComparer.Default); + IFieldSymbol storage = Assert.Single(record.GetMembers("_Mode").OfType()); + Assert.Equal(SpecialType.System_UInt16, storage.Type.SpecialType); + Assert.Equal(Accessibility.Private, storage.DeclaredAccessibility); + this.compilation = this.AddCode("using Windows.Win32.Test; static class ExistingFieldCaller { static ushort Use(NARROW_RECORD record) { record.Mode = NARROW_MODE.First; return (ushort)record.Mode; } }"); + this.AssertNoDiagnostics(); + } + + /// + /// Executes a scalar enum adapter through an unmanaged COM vtable without changing its native signature. + /// + [Fact] + public void NativeEnumComAdapterPreservesNativeBits() + { + const string NativeEnumInterface = """ + namespace Windows.Win32.Foundation + { + public struct HRESULT { public int Value; } + } + namespace Windows.Win32.Test + { + public struct NATIVE_RECORD { public NARROW_MODE Value; } + [ComImport, Guid("E783A761-BE81-4FD7-83F7-80DB453FB7EA"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] + public interface IScalar + { + [PreserveSig] + [return: AssociatedEnum("NARROW_MODE")] + ushort Convert([AssociatedEnum("NARROW_MODE")] ushort mode); + } + } + """; + this.GenerateSource(MetadataSource + Environment.NewLine + NativeEnumInterface, false, Platform.X64, "IScalar"); + this.compilation = this.AddCode(""" + using System.Runtime.CompilerServices; + using System.Runtime.InteropServices; + using Windows.Win32.Test; + /// Executes the native scalar adapter. + public static unsafe class ComAdapterCaller + { + /// Returns the native result after conversion. + public static uint Run() + { + void** vtable = stackalloc void*[4]; + for (int i = 0; i < 4; i++) vtable[i] = (void*)(delegate* unmanaged[Stdcall])&Convert; + IScalar instance = default; + *(void***)&instance = vtable; + return (uint)instance.Convert((NARROW_MODE)0x18001); + } + [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvStdcall) })] + private static ushort Convert(IScalar* instance, ushort mode) => (ushort)(mode ^ 2); + } + """); + this.compilation = this.compilation.WithOptions(this.compilation.Options.WithOverflowChecks(true)); + this.AssertNoDiagnostics(logAllGeneratedCode: false); + using MemoryStream stream = new(); + var result = this.compilation.Emit(stream, cancellationToken: TestContext.Current.CancellationToken); + Assert.True(result.Success, string.Join(Environment.NewLine, result.Diagnostics)); + Assert.Equal(0x8003U, Assembly.Load(stream.ToArray()).GetType("ComAdapterCaller")!.GetMethod("Run")!.Invoke(null, null)); + } + + /// + /// Keeps source-generated imports native while the scalar enum view is an ordinary managed adapter. + /// + [Fact] + public void NativeEnumAdapterKeepsLibraryImportRaw() + { + const string NativeEnumField = "namespace Windows.Win32.Test { public struct NATIVE_RECORD { public NARROW_MODE Value; } }"; + this.compilation = this.starterCompilations["net8.0-x64"]; + string path = EmitMetadata(MetadataSource + Environment.NewLine + NativeEnumField, this.compilation, this.parseOptions); + try + { + this.generator = new Generator(path, null, [], DefaultTestGeneratorOptions with { Public = true, ComInterop = new() { UseComSourceGenerators = true } }, this.compilation, this.parseOptions); + Assert.True(this.generator.TryGenerate("Narrow", TestContext.Current.CancellationToken)); + Assert.True(this.generator.TryGenerate("NarrowReturn", TestContext.Current.CancellationToken)); + this.CollectGeneratedCode(this.generator); + IMethodSymbol[] methods = this.compilation.GetTypeByMetadataName("Windows.Win32.PInvoke")!.GetMembers("Narrow").OfType().ToArray(); + Assert.Contains(methods, method => method.Parameters[0].Type.SpecialType == SpecialType.System_UInt16 && method.IsPartialDefinition); + Assert.Contains(methods, method => method.Parameters[0].Type.TypeKind == TypeKind.Enum && !method.IsPartialDefinition); + Assert.DoesNotContain(this.compilation.SyntaxTrees.SelectMany(tree => tree.GetRoot().DescendantNodes()).OfType(), attribute => + attribute.Name.ToString().Contains("MarshalUsing", StringComparison.Ordinal)); + this.AssertNoDiagnostics(this.compilation, logAllGeneratedCode: false, acceptable: diagnostic => diagnostic.Id == "CS8795"); + } + finally + { + File.Delete(path); + } + } + + /// + /// Does not decode unsupported native storage belonging to a different qualified enum with the same leaf name. + /// + [Fact] + public void UnrelatedQualifiedAssociationDoesNotAffectStorage() + { + const string UnrelatedSource = """ + namespace Windows.Win32.Other + { + public enum MODE : uint { First = 1 } + public unsafe struct RECORD + { + [Windows.Win32.Foundation.Metadata.AssociatedEnum("Windows.Win32.Other.MODE")] + public fixed float Modes[2]; + } + } + """; + this.compilation = this.starterCompilations["net8.0-x64"]; + string path = EmitMetadata(MetadataSource + Environment.NewLine + UnrelatedSource, this.compilation, this.parseOptions); + try + { + this.generator = new Generator(path, null, [], DefaultTestGeneratorOptions with { Public = true }, this.compilation, this.parseOptions); + Assert.True(this.generator.TryGenerate("RoundTrip", TestContext.Current.CancellationToken)); + this.CollectGeneratedCode(this.generator); + this.AssertNoDiagnostics(); + INamedTypeSymbol mode = this.compilation.GetTypeByMetadataName("Windows.Win32.Test.MODE")!; + Assert.Equal(SpecialType.System_UInt32, mode.EnumUnderlyingType!.SpecialType); + GenerationFailedException exception = Assert.Throws(() => this.generator.TryGenerate("Windows.Win32.Other.RECORD", TestContext.Current.CancellationToken)); + Assert.Contains("System.Single", exception.ToString(), StringComparison.Ordinal); + } + finally + { + File.Delete(path); + } + } + + /// + /// Preserves native high-bit flags when the logical enum metadata uses different signedness. + /// + /// Whether runtime marshaling is enabled. + [Theory] + [InlineData(true)] + [InlineData(false)] + public void NativeSignednessPreservesHighBitFlags(bool allowMarshaling) + { + string source = MetadataSource.Replace("using MODE_STORAGE = System.UInt32;", "using MODE_STORAGE = System.Int32;", StringComparison.Ordinal) + .Replace("fixed uint Modes", "fixed int Modes", StringComparison.Ordinal); + this.GenerateSource(source, allowMarshaling, Platform.X64, "MODE", "RoundTrip", "RECORD", "PointerReturn", "Output", "DoublePointer", "Buffer", "Unannotated"); + this.compilation = this.AddCode(Caller); + this.AssertNoDiagnostics(logAllGeneratedCode: false); + INamedTypeSymbol mode = this.compilation.GetTypeByMetadataName("Windows.Win32.Test.MODE")!; + Assert.Equal(SpecialType.System_Int32, mode.EnumUnderlyingType!.SpecialType); + Assert.Equal(int.MinValue, Assert.Single(mode.GetMembers("High").OfType()).ConstantValue); + } + + /// + /// Rejects an already available C# enum whose storage disagrees with the native use site. + /// + [Fact] + public void ReferencedEnumMustMatchNativeStorage() + { + this.compilation = this.starterCompilations["net8.0-x64"]; + string path = EmitMetadata(MetadataSource, this.compilation, this.parseOptions); + try + { + this.compilation = this.AddCode("namespace Windows.Win32.Test { public enum MODE : ushort { First = 1 } }"); + this.generator = new Generator(path, null, [], DefaultTestGeneratorOptions with { Public = true }, this.compilation, this.parseOptions); + GenerationFailedException exception = Assert.Throws(() => this.generator.TryGenerate("RoundTrip", TestContext.Current.CancellationToken)); + Assert.Contains("requires UInt32 storage", exception.ToString(), StringComparison.Ordinal); + Assert.Contains("referenced C# enum uses ushort", exception.ToString(), StringComparison.Ordinal); + } + finally + { + File.Delete(path); + } + } + + /// + /// Verifies declaring-namespace preference and explicit ambiguity errors. + /// + [Fact] + public void NamespaceResolutionAndAmbiguity() + { + const string OtherEnums = "namespace Windows.Win32.Other { public enum MODE : uint { Other = 1 } } namespace Windows.Win32.Another { public enum MODE : uint { Another = 1 } }"; + this.GenerateSource(MetadataSource + OtherEnums, false, Platform.X64, "RoundTrip"); + Assert.Equal("Windows.Win32.Test.MODE", Assert.Single(this.compilation.GetTypeByMetadataName("Windows.Win32.PInvoke")!.GetMembers("RoundTrip").OfType()).ReturnType.ToDisplayString()); + this.generator!.Dispose(); + this.generator = null; + string source = MetadataSource.Replace("enum MODE : uint", "enum LOCAL_MODE : uint", StringComparison.Ordinal) + OtherEnums; + GenerationFailedException exception = Assert.Throws(() => this.GenerateSource(source, false, Platform.X64, "RoundTrip")); + Assert.Contains("ambiguous", exception.ToString(), StringComparison.Ordinal); + } + + /// + /// Resolves annotations through actual assembly references, not arbitrary loaded metadata. + /// + /// Whether the native metadata references the enum's assembly. + [Theory] + [InlineData(true)] + [InlineData(false)] + public void QualifiedEnumReferenceClosure(bool referenced) + { + this.compilation = this.starterCompilations["net8.0-x64"]; + string external = EmitMetadata("namespace Windows.Win32.External { public enum EXTERNAL_MODE : long { First = 1 } }", this.compilation, this.parseOptions, "Windows.Win32.External.winmd"); + string source = MetadataSource.Replace("AssociatedEnum(\"MODE\")", "AssociatedEnum(\"Windows.Win32.External.EXTERNAL_MODE\")", StringComparison.Ordinal); + CSharpCompilation references = this.compilation; + if (referenced) + { + source += "namespace Windows.Win32.Test { public struct ANCHOR { public const Windows.Win32.External.EXTERNAL_MODE Value = Windows.Win32.External.EXTERNAL_MODE.First; } }"; + references = references.AddReferences(MetadataReference.CreateFromFile(external)); + } + + string native = EmitMetadata(source, references, this.parseOptions); + try + { + GeneratorOptions options = DefaultTestGeneratorOptions with { Public = true, AllowMarshaling = false }; + this.generator = SuperGenerator.Combine( + new Generator(native, null, [], options, this.compilation, this.parseOptions), + new Generator(external, null, [], options, this.compilation, this.parseOptions)); + if (referenced) + { + Assert.True(this.generator.TryGenerate("Windows.Win32.External.EXTERNAL_MODE", TestContext.Current.CancellationToken)); + Assert.True(this.generator.TryGenerate("RoundTrip", TestContext.Current.CancellationToken)); + this.CollectGeneratedCode(this.generator); + this.compilation = this.AddCode("static class ExternalCaller { static void Use() => Windows.Win32.PInvoke.RoundTrip(Windows.Win32.External.EXTERNAL_MODE.First); }"); + this.AssertNoDiagnostics(logAllGeneratedCode: false); + Assert.Equal(SpecialType.System_UInt32, this.compilation.GetTypeByMetadataName("Windows.Win32.External.EXTERNAL_MODE")!.EnumUnderlyingType!.SpecialType); + } + else + { + GenerationFailedException exception = Assert.Throws(() => this.generator.TryGenerate("RoundTrip", TestContext.Current.CancellationToken)); + Assert.Contains("metadata reference closure", exception.ToString(), StringComparison.Ordinal); + } + } + finally + { + this.generator?.Dispose(); + this.generator = null; + File.Delete(native); + File.Delete(external); + } + } + + /// + /// Verifies fixed-rank arrays and native signed element storage. + /// + [Fact] + public void MetadataArrayElements() + { + this.compilation = this.starterCompilations["net8.0-x64"]; + string path = Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location!)!, "ExternalMetadata", "AssociatedEnumArrays.winmd"); + this.generator = new Generator(path, null, [], DefaultTestGeneratorOptions with { Public = true, AllowMarshaling = false }, this.compilation, this.parseOptions); + Assert.True(this.generator.TryGenerate("ARRAY_RECORD", TestContext.Current.CancellationToken)); + Assert.True(this.generator.TryGenerate("ARRAY_FLEX", TestContext.Current.CancellationToken)); + Assert.True(this.generator.TryGenerate("SIGNED_ARRAY_RECORD", TestContext.Current.CancellationToken)); + this.CollectGeneratedCode(this.generator); + this.compilation = this.AddCode(""" + using Windows.Win32.Test; + /// Exercises metadata fixed-rank arrays. + public static unsafe class ArrayLayout + { + /// Returns fixed and flexible layout sizes. + public static int Run() + { + ARRAY_RECORD record = default; + record.Modes.AsSpan()[0] = ARRAY_MODE.First; + SIGNED_ARRAY_RECORD signed = default; + signed.Modes.AsSpan()[0] = SIGNED_MODE.First; + if (sizeof(SIGNED_ARRAY_RECORD) != 8) throw new System.Exception(); + if (record.Modes.AsSpan()[0] != ARRAY_MODE.First) throw new System.Exception(); + return sizeof(ARRAY_RECORD) * 100 + ARRAY_FLEX.SizeOf(3); + } + } + """); + this.AssertNoDiagnostics(logAllGeneratedCode: false); + using MemoryStream stream = new(); + var emitted = this.compilation.Emit(stream, cancellationToken: TestContext.Current.CancellationToken); + Assert.True(emitted.Success, string.Join(Environment.NewLine, emitted.Diagnostics)); + Assert.Equal(812, Assembly.Load(stream.ToArray()).GetType("ArrayLayout")!.GetMethod("Run")!.Invoke(null, null)); + Assert.Equal(SpecialType.System_Int32, this.compilation.GetTypeByMetadataName("Windows.Win32.Test.SIGNED_MODE")!.EnumUnderlyingType!.SpecialType); + } + + /// + /// Uses direct native-storage enum signatures, without conversion marshalers, with source generators. + /// + /// The native storage code. + [Theory, CombinatorialData] + public void SourceGeneratedIntegralProjection( + [CombinatorialValues("sbyte", "byte", "short", "ushort", "int", "uint", "long", "ulong")] string native) + { + this.compilation = this.starterCompilations["net8.0-x64"]; + string source = MetadataSource.Replace("ushort", native, StringComparison.Ordinal) + .Replace("enum NARROW_MODE : uint { First = 1, Second = 2, High = 0x8000 }", $"enum NARROW_MODE : {(native == "ulong" ? "int" : "ulong")} {{ First = 1, Second = 2, High = 4 }}", StringComparison.Ordinal); + string path = EmitMetadata(source, this.compilation, this.parseOptions); + try + { + Generator generator = new(path, null, [], DefaultTestGeneratorOptions with { Public = true, ComInterop = new() { UseComSourceGenerators = true } }, this.compilation, this.parseOptions); + this.generator = generator; + Assert.True(generator.TryGenerate("Narrow", TestContext.Current.CancellationToken)); + Assert.True(generator.TryGenerate("NarrowReturn", TestContext.Current.CancellationToken)); + this.CollectGeneratedCode(generator); + INamedTypeSymbol pinvoke = this.compilation.GetTypeByMetadataName("Windows.Win32.PInvoke")!; + IMethodSymbol method = Assert.Single(pinvoke.GetMembers("Narrow").OfType()); + INamedTypeSymbol projected = Assert.IsAssignableFrom(method.Parameters[0].Type); + Assert.Equal(native, projected.EnumUnderlyingType!.ToDisplayString()); + Assert.DoesNotContain(method.Parameters[0].GetAttributes(), attribute => attribute.AttributeClass?.Name is "MarshalUsingAttribute" or "MarshalAsAttribute"); + IMethodSymbol returning = Assert.Single(pinvoke.GetMembers("NarrowReturn").OfType()); + Assert.True(SymbolEqualityComparer.Default.Equals(projected, returning.ReturnType)); + Assert.DoesNotContain(returning.GetReturnTypeAttributes(), attribute => attribute.AttributeClass?.Name is "MarshalUsingAttribute" or "MarshalAsAttribute"); + this.AssertNoDiagnostics(this.compilation, logAllGeneratedCode: false, acceptable: diagnostic => diagnostic.Id == "CS8795"); + } + finally + { + File.Delete(path); + } + } + + /// + /// Projects indirect native enum storage directly, including counted buffers and direction annotations. + /// + /// The pointer's direction and array annotations. + [Theory] + [InlineData("")] + [InlineData("In, ")] + [InlineData("Out, ")] + [InlineData("In, Out, ")] + [InlineData("In, NativeArrayInfo(CountConst = 2), ")] + public void SourceGeneratedIndirectProjection(string attributes) + { + this.compilation = this.starterCompilations["net8.0-x64"]; + string source = MetadataSource.Replace("void NarrowPointer([AssociatedEnum", "void NarrowPointer([" + attributes + "AssociatedEnum", StringComparison.Ordinal); + string path = EmitMetadata(source, this.compilation, this.parseOptions); + try + { + this.generator = new Generator(path, null, [], DefaultTestGeneratorOptions with { Public = true, ComInterop = new() { UseComSourceGenerators = true } }, this.compilation, this.parseOptions); + Assert.True(this.generator.TryGenerate("NarrowPointer", TestContext.Current.CancellationToken)); + Assert.True(this.generator.TryGenerate("Buffer", TestContext.Current.CancellationToken)); + this.CollectGeneratedCode(this.generator); + IMethodSymbol buffer = Assert.Single(this.compilation.GetTypeByMetadataName("Windows.Win32.PInvoke")!.GetMembers("Buffer").OfType(), method => method.Parameters.Length == 2); + IPointerTypeSymbol pointer = Assert.IsAssignableFrom(buffer.Parameters[0].Type); + Assert.Equal("Windows.Win32.Test.MODE", pointer.PointedAtType.ToDisplayString()); + Assert.Equal(SpecialType.System_UInt16, this.compilation.GetTypeByMetadataName("Windows.Win32.Test.NARROW_MODE")!.EnumUnderlyingType!.SpecialType); + this.AssertNoDiagnostics(this.compilation, logAllGeneratedCode: false, acceptable: diagnostic => diagnostic.Id == "CS8795"); + } + finally + { + File.Delete(path); + } + } + + /// + /// Rejects compound bitfield associations whose arithmetic requires native primitive backing storage. + /// + [Fact] + public void BitfieldAssociationFailsExplicitly() + { + string source = MetadataSource.Replace( + "public sealed class NativeArrayInfoAttribute", + "public sealed class NativeBitfieldAttribute : Attribute { public NativeBitfieldAttribute(string name, long offset, long length) { } } public sealed class NativeArrayInfoAttribute", + StringComparison.Ordinal) + .Replace("[AssociatedEnum(\"MODE\")] public MODE_STORAGE Mode;", "[AssociatedEnum(\"MODE\"), NativeBitfield(\"FirstBit\", 0, 1)] public MODE_STORAGE Mode;", StringComparison.Ordinal); + GenerationFailedException exception = Assert.Throws(() => this.GenerateSource(source, false, Platform.X64, "RECORD")); + Assert.Contains("NativeBitfield", exception.ToString(), StringComparison.Ordinal); + } + + /// + /// Checks that the caller fixture actually detects enum and pointer-depth regressions. + /// + /// The broken metadata signature. + [Theory] + [InlineData("remove-association")] + [InlineData("pointer-depth")] + public void CallerMutationControls(string mutation) + { + string source = mutation == "remove-association" + ? MetadataSource.Replace("[AssociatedEnum(\"MODE\")] MODE_STORAGE mode", "MODE_STORAGE mode", StringComparison.Ordinal) + : MetadataSource.Replace("MODE_STORAGE** modes", "MODE_STORAGE* modes", StringComparison.Ordinal); + this.GenerateSource(source, false, Platform.X64, "RECORD", "RoundTrip", "PointerReturn", "Buffer", "Output", "DoublePointer", "Unannotated"); + this.compilation = this.AddCode(Caller); + Assert.Contains(this.compilation.GetDiagnostics(TestContext.Current.CancellationToken), diagnostic => diagnostic.Severity == DiagnosticSeverity.Error && diagnostic.Location.SourceTree?.FilePath.StartsWith("AdditionalCode", StringComparison.Ordinal) is true); + } + + /// + /// Creates the pinned canonical old or new metadata source. + /// + /// Whether enum types replace native primitives. + /// The metadata source. + internal static string GetMetadataSource(bool legacy) => legacy + ? MetadataSource.Replace("using MODE_STORAGE = System.UInt32;", "using MODE_STORAGE = Windows.Win32.Test.MODE;", StringComparison.Ordinal) + .Replace("[AssociatedEnum(\"MODE\")]", string.Empty, StringComparison.Ordinal) + .Replace("[AssociatedEnum(\"Windows.Win32.Test.MODE\")]", string.Empty, StringComparison.Ordinal) + .Replace("[return: AssociatedEnum(\"MODE\")]", string.Empty, StringComparison.Ordinal) + .Replace("[return: AssociatedEnum(\"Windows.Win32.Test.MODE\")]", string.Empty, StringComparison.Ordinal) + .Replace("AssociatedEnum(\"MODE\"), ", string.Empty, StringComparison.Ordinal) + .Replace(", AssociatedEnum(\"MODE\")", string.Empty, StringComparison.Ordinal) + : MetadataSource; + + /// + /// Creates matching synthetic/native fixtures while deliberately varying the logical enum metadata storage. + /// + /// The native integral storage. + /// Whether signatures use synthetic enums directly. + /// The OLD or NEW metadata source. + internal static string GetNativeStorageMetadata(string storage, bool legacy) + { + string systemType = storage switch + { + "sbyte" => "SByte", "byte" => "Byte", "short" => "Int16", "ushort" => "UInt16", + "int" => "Int32", "uint" => "UInt32", "long" => "Int64", "ulong" => "UInt64", + _ => throw new ArgumentException("Unsupported native enum storage.", nameof(storage)), + }; + string logicalStorage = legacy ? storage : storage == "ulong" ? "int" : "ulong"; + return GetMetadataSource(legacy) + .Replace("using MODE_STORAGE = System.UInt32;", $"using MODE_STORAGE = System.{systemType};", StringComparison.Ordinal) + .Replace("enum MODE : uint { First = 1, Second = 2, High = 0x80000000 }", $"enum MODE : {logicalStorage} {{ First = 1, Second = 2, High = 4 }}", StringComparison.Ordinal) + .Replace("fixed uint Modes", $"fixed {storage} Modes", StringComparison.Ordinal); + } + + /// + /// Emits a deterministic canonical metadata snapshot. + /// + /// The declarations and annotation definitions. + /// The pinned reference compilation. + /// The language options. + /// The metadata assembly identity. + /// The path of the snapshot, owned by the caller. + internal static string EmitMetadata(string source, CSharpCompilation references, CSharpParseOptions parseOptions, string assemblyName = "Windows.Win32.winmd") + { + CSharpCompilation metadata = CSharpCompilation.Create( + assemblyName, + [CSharpSyntaxTree.ParseText(source, parseOptions)], + references.References, + references.Options.WithDeterministic(true).WithPlatform(Platform.AnyCpu)); + string path = Path.Combine(Path.GetTempPath(), $"CsWin32-AssociatedEnum-{Guid.NewGuid():N}.winmd"); + using FileStream stream = File.Create(path); + var result = metadata.Emit(stream, cancellationToken: TestContext.Current.CancellationToken); + Assert.True(result.Success, string.Join(Environment.NewLine, result.Diagnostics)); + return path; + } + + private void GenerateFixture(bool legacy, bool allowMarshaling, params string[] apis) => + this.GenerateSource(GetMetadataSource(legacy), allowMarshaling, Platform.X64, apis); + + private void GenerateSource(string source, bool allowMarshaling, Platform platform, params string[] apis) + { + this.compilation = this.starterCompilations["net8.0"].WithOptions(this.starterCompilations["net8.0"].Options.WithPlatform(platform)); + string path = EmitMetadata(source, this.compilation, this.parseOptions); + try + { + this.generator = new Generator(path, null, [], DefaultTestGeneratorOptions with { AllowMarshaling = allowMarshaling, Public = true }, this.compilation, this.parseOptions); + foreach (string api in apis) + { + Assert.True(this.generator.TryGenerate(api, TestContext.Current.CancellationToken)); + } + + this.CollectGeneratedCode(this.generator); + } + finally + { + File.Delete(path); + } + } +} diff --git a/test/Microsoft.Windows.CsWin32.Tests/ContextualInvalidHandleTests.cs b/test/Microsoft.Windows.CsWin32.Tests/ContextualInvalidHandleTests.cs new file mode 100644 index 00000000..bc5d5418 --- /dev/null +++ b/test/Microsoft.Windows.CsWin32.Tests/ContextualInvalidHandleTests.cs @@ -0,0 +1,200 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using Microsoft.CodeAnalysis.Diagnostics; +using static Microsoft.CodeAnalysis.CSharp.SyntaxFactory; + +namespace Microsoft.Windows.CsWin32.Tests; + +/// +/// Verifies invalid values supplied by producer return and output annotations, rather than fabricated typedefs. +/// +public class ContextualInvalidHandleTests : GeneratorTestBase +{ + private const string MetadataSource = """ + using System; + using System.Runtime.InteropServices; + using Windows.Win32.Foundation.Metadata; + namespace Windows.Win32.Foundation.Metadata + { + public sealed class RAIIFreeAttribute : Attribute { public RAIIFreeAttribute(string name) { } } + [AttributeUsage(AttributeTargets.ReturnValue | AttributeTargets.Parameter | AttributeTargets.Struct, AllowMultiple = true)] + public sealed class InvalidHandleValueAttribute : Attribute { public InvalidHandleValueAttribute(long value) { } } + } + namespace Windows.Win32.Test + { + public static unsafe class Apis + { + [DllImport("fixture.dll", ExactSpelling = true)] + public static extern void CloseThing(void* resource); + [DllImport("fixture.dll", ExactSpelling = true)] + [return: RAIIFree("CloseThing"), InvalidHandleValue(0), InvalidHandleValue(-1)] + public static extern void* OpenThing(); + [DllImport("fixture.dll", ExactSpelling = true)] + public static extern int CreateThing([Out, RAIIFree("CloseThing"), InvalidHandleValue(0), InvalidHandleValue(-1)] void** resource); + [DllImport("fixture.dll", ExactSpelling = true)] + [return: RAIIFree("CloseThing"), InvalidHandleValue(0)] + public static extern void* OpenZeroInvalid(); + } + } + """; + + /// + /// Initializes a new instance of the class. + /// + /// The test output. + public ContextualInvalidHandleTests(ITestOutputHelper logger) + : base(logger) + { + } + + /// + /// Exercises invalid handles, output ownership and disposal while a typed lease is alive. + /// + /// Whether runtime marshaling is enabled. + [Theory] + [InlineData(true)] + [InlineData(false)] + public async Task ContextualInvalidValuesAndLeaseLifetime(bool allowMarshaling) + { + this.compilation = this.starterCompilations["net8.0-x64"]; + string path = AssociatedEnumTests.EmitMetadata(MetadataSource, this.compilation, this.parseOptions); + try + { + this.generator = new Generator(path, null, [], DefaultTestGeneratorOptions with { AllowMarshaling = allowMarshaling, Public = true }, this.compilation, this.parseOptions); + foreach (string api in new[] { "OpenThing", "CreateThing", "OpenZeroInvalid" }) + { + Assert.True(this.generator.TryGenerate(api, CancellationToken.None)); + } + + this.CollectGeneratedCode(this.generator); + } + finally + { + File.Delete(path); + } + + IMethodSymbol open = Assert.Single(this.compilation.GetTypeByMetadataName("Windows.Win32.PInvoke")!.GetMembers("OpenThing_SafeHandle").OfType()); + string ownerName = open.ReturnType.Name; + IMethodSymbol create = Assert.Single(this.compilation.GetTypeByMetadataName("Windows.Win32.PInvoke")!.GetMembers("CreateThing").OfType(), method => method.Parameters[0].RefKind == RefKind.Out); + Assert.True(SymbolEqualityComparer.Default.Equals(open.ReturnType, create.Parameters[0].Type)); + IMethodSymbol zero = Assert.Single(this.compilation.GetTypeByMetadataName("Windows.Win32.PInvoke")!.GetMembers("OpenZeroInvalid_SafeHandle").OfType()); + Assert.False(SymbolEqualityComparer.Default.Equals(open.ReturnType, zero.ReturnType)); + string zeroOwnerName = zero.ReturnType.Name; + + foreach (SyntaxTree tree in this.compilation.SyntaxTrees.ToArray()) + { + MethodDeclarationSyntax[] nativeMethods = tree.GetRoot(TestContext.Current.CancellationToken).DescendantNodes().OfType() + .Where(method => method.Modifiers.Any(SyntaxKind.ExternKeyword) && method.Identifier.ValueText is "CloseThing" or "CreateThing").ToArray(); + SyntaxNode rewritten = tree.GetRoot(TestContext.Current.CancellationToken).ReplaceNodes(nativeMethods, (original, current) => current.WithAttributeLists(default) + .WithModifiers(TokenList(current.Modifiers.Where(modifier => !modifier.IsKind(SyntaxKind.ExternKeyword)))) + .WithBody((BlockSyntax)ParseStatement(original.Identifier.ValueText == "CloseThing" ? "{ RuntimeUsage.Frees++; }" : "{ *resource = (void*)(-1); return -1; }")) + .WithSemicolonToken(default)); + this.compilation = this.compilation.ReplaceSyntaxTree(tree, CSharpSyntaxTree.ParseText(rewritten.ToFullString(), this.parseOptions, tree.FilePath, cancellationToken: TestContext.Current.CancellationToken)); + } + + this.compilation = this.AddCode($$""" + using System; + using Windows.Win32; + /// Executes contextual owners and leases. + public static unsafe class RuntimeUsage + { + /// Counts native cleanup calls. + public static int Frees; + /// Verifies invalid values and reference balance. + public static void Run() + { + using (var invalid = new {{ownerName}}((IntPtr)(-1))) + { + if (!invalid.IsInvalid) throw new Exception("Lost contextual -1 invalid value."); + } + using (var invalid = new {{ownerName}}(IntPtr.Zero)) + { + if (!invalid.IsInvalid) throw new Exception("Lost contextual zero invalid value."); + } + if (Frees != 0) throw new Exception("Released an invalid handle."); + using (var valid = new {{zeroOwnerName}}((IntPtr)(-1))) + { + if (valid.IsInvalid) throw new Exception("Conflated contextual invalid-value profiles."); + } + if (Frees != 1) throw new Exception("Failed to release a valid handle."); + var owner = new {{ownerName}}((IntPtr)42); + using (var lease = owner.Lease()) + { + owner.Dispose(); + if (Frees != 1 || (nint)lease.Value != 42) throw new Exception("Released a borrowed resource."); + } + if (Frees != 2) throw new Exception("Unbalanced lease release."); + PInvoke.CreateThing(out {{ownerName}} output); + using (output) + { + if (!output.IsInvalid) throw new Exception("Lost invalid output value."); + } + if (Frees != 2) throw new Exception("Released an invalid output."); + } + static void Raw() + { + void* resource; + PInvoke.CreateThing(&resource); + void* opened = PInvoke.OpenThing(); + } + } + """); + this.AssertNoDiagnostics(logAllGeneratedCode: false); + Assert.Empty(await this.compilation.WithAnalyzers(ImmutableArray.Create(new SafeHandleAnalyzer())).GetAnalyzerDiagnosticsAsync(TestContext.Current.CancellationToken)); + using MemoryStream stream = new(); + var result = this.compilation.Emit(stream, cancellationToken: TestContext.Current.CancellationToken); + Assert.True(result.Success, string.Join(Environment.NewLine, result.Diagnostics)); + Assembly.Load(stream.ToArray()).GetType("RuntimeUsage")!.GetMethod("Run")!.Invoke(null, null); + } + + /// + /// Reuses a BCL owner only when its invalid-value contract matches the producing annotation. + /// + [Fact] + public void ContextualInvalidValuesOverrideBclOwner() + { + this.compilation = this.starterCompilations["net8.0-x64"]; + string source = MetadataSource.Replace( + "public sealed class RAIIFreeAttribute", + "public sealed class NativeTypedefAttribute : Attribute { } public sealed class RAIIFreeAttribute", + StringComparison.Ordinal) + """ + + namespace Windows.Win32.Foundation + { + [NativeTypedef, RAIIFree("CloseHandle"), InvalidHandleValue(0), InvalidHandleValue(-1)] + public unsafe struct HANDLE { public void* Value; } + public static class Apis + { + [DllImport("fixture.dll", ExactSpelling = true)] + public static extern int CloseHandle(HANDLE resource); + [DllImport("fixture.dll", ExactSpelling = true)] + [return: RAIIFree("CloseHandle"), InvalidHandleValue(0), InvalidHandleValue(-1)] + public static extern HANDLE OpenBcl(); + [DllImport("fixture.dll", ExactSpelling = true)] + [return: RAIIFree("CloseHandle"), InvalidHandleValue(0)] + public static extern HANDLE OpenCustom(); + } + } + """; + string path = AssociatedEnumTests.EmitMetadata(source, this.compilation, this.parseOptions); + try + { + this.generator = new Generator(path, null, [], DefaultTestGeneratorOptions with { Public = true }, this.compilation, this.parseOptions); + Assert.True(this.generator.TryGenerate("OpenBcl", TestContext.Current.CancellationToken)); + Assert.True(this.generator.TryGenerate("OpenCustom", TestContext.Current.CancellationToken)); + this.CollectGeneratedCode(this.generator); + } + finally + { + File.Delete(path); + } + + INamedTypeSymbol pinvoke = this.compilation.GetTypeByMetadataName("Windows.Win32.PInvoke")!; + IMethodSymbol bcl = Assert.Single(pinvoke.GetMembers("OpenBcl_SafeHandle").OfType()); + Assert.Equal("Microsoft.Win32.SafeHandles.SafeFileHandle", bcl.ReturnType.ToDisplayString()); + IMethodSymbol custom = Assert.Single(pinvoke.GetMembers("OpenCustom_SafeHandle").OfType()); + Assert.Equal("Windows.Win32.CloseHandleInvalid0SafeHandle", custom.ReturnType.ToDisplayString()); + this.AssertNoDiagnostics(logAllGeneratedCode: false); + } +} diff --git a/test/Microsoft.Windows.CsWin32.Tests/ExternalMetadata/AssociatedEnumArrays.il b/test/Microsoft.Windows.CsWin32.Tests/ExternalMetadata/AssociatedEnumArrays.il new file mode 100644 index 00000000..ad581a1d --- /dev/null +++ b/test/Microsoft.Windows.CsWin32.Tests/ExternalMetadata/AssociatedEnumArrays.il @@ -0,0 +1,53 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. +// +// Assemble with: ilasm /dll /output=AssociatedEnumArrays.winmd AssociatedEnumArrays.il + +.assembly extern netstandard { .publickeytoken = (CC 7B 13 FF CD 2D DD 51) .ver 2:1:0:0 } +.assembly extern mscorlib { .publickeytoken = (B7 7A 5C 56 19 34 E0 89) .ver 4:0:0:0 } +.assembly extern Windows.Win32.winmd { .ver 0:0:0:0 } +.assembly Windows.Win32.winmd { .ver 1:0:0:0 } +.module AssociatedEnumArrays.winmd +.corflags 0x00000001 + +.class public sequential ansi sealed Windows.Win32.Foundation.ANCHOR + extends [netstandard]System.ValueType +{ + .field public uint32 Value +} + +.class public auto ansi sealed Windows.Win32.Test.ARRAY_MODE + extends [netstandard]System.Enum +{ + .field public specialname rtspecialname uint32 value__ + .field public static literal valuetype Windows.Win32.Test.ARRAY_MODE First = uint32(1) +} + +.class public sequential ansi sealed Windows.Win32.Test.ARRAY_RECORD + extends [netstandard]System.ValueType +{ + .field public uint32[2] Modes + .custom instance void [Windows.Win32.winmd]Windows.Win32.Foundation.Metadata.AssociatedEnumAttribute::.ctor(string) = (01 00 0A 41 52 52 41 59 5F 4D 4F 44 45 00 00) +} + +.class public sequential ansi sealed Windows.Win32.Test.ARRAY_FLEX + extends [netstandard]System.ValueType +{ + .field public uint32[1] Modes + .custom instance void [Windows.Win32.winmd]Windows.Win32.Foundation.Metadata.AssociatedEnumAttribute::.ctor(string) = (01 00 0A 41 52 52 41 59 5F 4D 4F 44 45 00 00) + .custom instance void [Windows.Win32.winmd]Windows.Win32.Foundation.Metadata.FlexibleArrayAttribute::.ctor() = (01 00 00 00) +} + +.class public auto ansi sealed Windows.Win32.Test.SIGNED_MODE + extends [netstandard]System.Enum +{ + .field public specialname rtspecialname uint32 value__ + .field public static literal valuetype Windows.Win32.Test.SIGNED_MODE First = uint32(1) +} + +.class public sequential ansi sealed Windows.Win32.Test.SIGNED_ARRAY_RECORD + extends [netstandard]System.ValueType +{ + .field public int32[2] Modes + .custom instance void [Windows.Win32.winmd]Windows.Win32.Foundation.Metadata.AssociatedEnumAttribute::.ctor(string) = (01 00 0B 53 49 47 4E 45 44 5F 4D 4F 44 45 00 00) +} diff --git a/test/Microsoft.Windows.CsWin32.Tests/ExternalMetadata/AssociatedEnumArrays.winmd b/test/Microsoft.Windows.CsWin32.Tests/ExternalMetadata/AssociatedEnumArrays.winmd new file mode 100644 index 00000000..83648fb4 Binary files /dev/null and b/test/Microsoft.Windows.CsWin32.Tests/ExternalMetadata/AssociatedEnumArrays.winmd differ diff --git a/test/Microsoft.Windows.CsWin32.Tests/ExternalMetadata/NOTICE.md b/test/Microsoft.Windows.CsWin32.Tests/ExternalMetadata/NOTICE.md index e301f04f..9e685e50 100644 --- a/test/Microsoft.Windows.CsWin32.Tests/ExternalMetadata/NOTICE.md +++ b/test/Microsoft.Windows.CsWin32.Tests/ExternalMetadata/NOTICE.md @@ -6,4 +6,5 @@ ServiceFabric.winmd | [youyuanwu/fabric-metadata](https://github.com/youyuanwu/f CustomIInspectable.winmd | Generated using WinMDGenerator toolchain. Project in the subdirectory [CustomIInspectable](../../CustomIInspectable) with build instructions in [readme.md](../../CustomIInspectable/readme.md). WnfWithoutStatusSuccess.winmd | Authored fixture for issue #1677 regression coverage. Its source is checked in as [WnfWithoutStatusSuccess.il](WnfWithoutStatusSuccess.il). We keep this one as IL because that is much easier to maintain than reproducing the full end-to-end WinmdGenerator workflow for a focused test asset. SelfContained.winmd | Authored fixture reproducing the OSClient single-`Generator` build configuration (a non-`Windows.Win32` metadata assembly with the Windows.Win32 projection provided by a reference), where special string typedefs (`PCWSTR`) must be referenced under `Windows.Win32.Foundation`. Its source is checked in as [SelfContained.il](SelfContained.il). Kept as IL for the same reason as WnfWithoutStatusSuccess.winmd. -SafeHandleOwnership.winmd | Authored fixture for contextual `RAIIFree` and `FreeWith` ownership, HANDLE input inference without type-level ownership, exact native value types, and caller-defined cleanup. Its source is checked in as [SafeHandleOwnership.il](SafeHandleOwnership.il), including the IL assembler command. Tests load this asset rather than emitting metadata at runtime. \ No newline at end of file +SafeHandleOwnership.winmd | Authored fixture for contextual `RAIIFree` and `FreeWith` ownership, HANDLE input inference without type-level ownership, exact native value types, and caller-defined cleanup. Its source is checked in as [SafeHandleOwnership.il](SafeHandleOwnership.il), including the IL assembler command. Tests load this asset rather than emitting metadata at runtime. +AssociatedEnumArrays.winmd | Authored fixed-rank and flexible array fixture with native primitive elements and logical enum associations, including signed native storage that determines the generated enum's underlying type. Its source is [AssociatedEnumArrays.il](AssociatedEnumArrays.il), including the IL assembler command. \ No newline at end of file diff --git a/test/Microsoft.Windows.CsWin32.Tests/HeaderAnnotationCompatibilityTests.cs b/test/Microsoft.Windows.CsWin32.Tests/HeaderAnnotationCompatibilityTests.cs new file mode 100644 index 00000000..bab5ed40 --- /dev/null +++ b/test/Microsoft.Windows.CsWin32.Tests/HeaderAnnotationCompatibilityTests.cs @@ -0,0 +1,594 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System.Reflection.PortableExecutable; +using System.Runtime.Loader; +using System.Security.Cryptography; +using System.Text.Json; + +namespace Microsoft.Windows.CsWin32.Tests; + +/// +/// Opt-in three-way compatibility checks against a separately built, pinned baseline generator. +/// +public class HeaderAnnotationCompatibilityTests : GeneratorTestBase +{ + private const string OrdinaryFieldCaller = """ + namespace Windows.Win32.Test + { + static class FieldCaller + { + static MODE Use(RECORD record) + { + record.Mode = MODE.First | MODE.Second; + return record.Mode; + } + } + } + """; + + private const string PropertyInterfaceCaller = """ + namespace Windows.Win32.Test + { + /// Represents a caller-defined property contract. + public interface IModeProperty + { + /// Gets or sets the mode. + MODE Mode { get; set; } + } + /// Extends the generated struct with a caller contract. + public partial struct RECORD : IModeProperty { } + } + """; + + private static readonly string[] CanonicalRequests = ["RECORD", "RoundTrip", "PointerReturn", "Buffer", "Output", "DoublePointer", "UseCallback", "Unannotated"]; + private static readonly string[] WinHttpRequests = + [ + "WinHttpOpen", "WinHttpConnect", "WinHttpOpenRequest", "WinHttpCloseHandle", "WinHttpCreateUrl", "WinHttpCrackUrl", + "WinHttpSetStatusCallback", "WinHttpQueryHeadersEx", "WinHttpReadData", "WinHttpGetProxyForUrl", "WinHttpCreateProxyResolver", + "WINHTTP_CREDS", "WINHTTP_CREDS_EX", "WINHTTP_PROXY_RESULT_ENTRY", "WINHTTP_PROXY_INFO", "URL_COMPONENTS", + ]; + + private static readonly SymbolDisplayFormat ApiDisplay = new( + globalNamespaceStyle: SymbolDisplayGlobalNamespaceStyle.Included, + typeQualificationStyle: SymbolDisplayTypeQualificationStyle.NameAndContainingTypesAndNamespaces, + genericsOptions: SymbolDisplayGenericsOptions.IncludeTypeParameters, + memberOptions: SymbolDisplayMemberOptions.IncludeAccessibility | SymbolDisplayMemberOptions.IncludeModifiers | SymbolDisplayMemberOptions.IncludeContainingType | SymbolDisplayMemberOptions.IncludeType | SymbolDisplayMemberOptions.IncludeParameters | SymbolDisplayMemberOptions.IncludeRef, + parameterOptions: SymbolDisplayParameterOptions.IncludeType | SymbolDisplayParameterOptions.IncludeName | SymbolDisplayParameterOptions.IncludeParamsRefOut | SymbolDisplayParameterOptions.IncludeDefaultValue, + miscellaneousOptions: SymbolDisplayMiscellaneousOptions.UseSpecialTypes); + + /// + /// Initializes a new instance of the class. + /// + /// The test output. + public HeaderAnnotationCompatibilityTests(ITestOutputHelper logger) + : base(logger) + { + } + + /// + /// Compiles the same caller and compares public symbols across all three canonical legs. + /// + /// The pinned marshaling option. + [Theory] + [InlineData(true)] + [InlineData(false)] + public void CanonicalThreeWayMatrix(bool allowMarshaling) + { + RequireBaseline(); + CSharpCompilation references = this.starterCompilations["net8.0-x64"]; + string oldPath = AssociatedEnumTests.EmitMetadata(AssociatedEnumTests.GetMetadataSource(true), references, this.parseOptions); + string newPath = AssociatedEnumTests.EmitMetadata(AssociatedEnumTests.GetMetadataSource(false), references, this.parseOptions); + try + { + string callers = GetCanonicalCaller(allowMarshaling); + MatrixResult result = this.RunMatrix("canonical", oldPath, newPath, CanonicalRequests, callers, allowMarshaling); + Assert.All(result.Runs, run => Assert.Empty(run.CallerDiagnostics)); + Assert.Equal(result.Runs[0].PublicSymbols, result.Runs[1].PublicSymbols); + Assert.Equal(result.Runs[0].PublicSymbols, result.Runs[2].PublicSymbols); + } + finally + { + File.Delete(oldPath); + File.Delete(newPath); + } + } + + /// + /// Proves direct native enum storage against OLD synthetic fields and signatures, even when logical storage differs. + /// + /// The native integral storage. + /// The pinned marshaling option. + /// Whether to request the enum before its annotated consumers. + [Theory, CombinatorialData] + public void NativeStorageThreeWayMatrix( + [CombinatorialValues("sbyte", "byte", "short", "ushort", "int", "uint", "long", "ulong")] string storage, + bool allowMarshaling, + bool enumFirst) + { + RequireBaseline(); + CSharpCompilation references = this.starterCompilations["net8.0-x64"]; + string oldPath = AssociatedEnumTests.EmitMetadata(AssociatedEnumTests.GetNativeStorageMetadata(storage, legacy: true), references, this.parseOptions); + string newPath = AssociatedEnumTests.EmitMetadata(AssociatedEnumTests.GetNativeStorageMetadata(storage, legacy: false), references, this.parseOptions); + try + { + string[] requests = enumFirst ? ["MODE", .. CanonicalRequests] : CanonicalRequests; + MatrixResult result = this.RunMatrix($"native-storage-{storage}-{(enumFirst ? "enum-first" : "consumer-first")}", oldPath, newPath, requests, GetCanonicalCaller(allowMarshaling), allowMarshaling); + Assert.All(result.Runs, run => Assert.Empty(run.CallerDiagnostics)); + Assert.Equal(result.Runs[0].PublicSymbols, result.Runs[1].PublicSymbols); + Assert.Equal(result.Runs[0].PublicSymbols, result.Runs[2].PublicSymbols); + Assert.Contains(result.Runs[2].PublicSymbols, symbol => symbol.StartsWith("NamedType ", StringComparison.Ordinal) && symbol.Contains($"enum={storage}; flags=True", StringComparison.Ordinal)); + } + finally + { + File.Delete(oldPath); + File.Delete(newPath); + } + } + + /// + /// Records source-compatible field usage and the narrower property-contract break in existing annotated metadata. + /// + /// The pinned marshaling option. + /// Whether the unchanged partial struct implements a property contract. + [Theory, CombinatorialData] + public void ExistingAnnotatedFieldRegression(bool allowMarshaling, bool requireProperty) + { + RequireBaseline(); + string source = AssociatedEnumTests.MetadataSource.Replace( + "[AssociatedEnum(\"Windows.Win32.Test.MODE\")] public MODE_STORAGE* Modes;", + string.Empty, + StringComparison.Ordinal); + string path = AssociatedEnumTests.EmitMetadata(source, this.starterCompilations["net8.0-x64"], this.parseOptions); + string caller = OrdinaryFieldCaller + (requireProperty ? Environment.NewLine + PropertyInterfaceCaller : string.Empty); + + try + { + MatrixResult result = this.RunMatrix(requireProperty ? "existing-annotation-property-contract" : "existing-annotation-field", path, null, ["RECORD"], caller, allowMarshaling, compatibilityScope: "independent-existing-annotation-regression"); + Assert.Empty(result.Runs[0].CallerDiagnostics); + foreach (RunResult run in result.Runs.Skip(1)) + { + Assert.Contains(run.RemovedPublicSymbols, symbol => symbol.StartsWith("Property ", StringComparison.Ordinal) && symbol.Contains("RECORD.Mode", StringComparison.Ordinal)); + Assert.Contains(run.AddedPublicSymbols, symbol => symbol.StartsWith("Field ", StringComparison.Ordinal) && symbol.Contains("RECORD.Mode", StringComparison.Ordinal)); + Assert.Equal(requireProperty ? "residual-source-break" : "source-compatible-difference", run.Grade); + if (requireProperty) + { + string diagnostic = Assert.Single(run.CallerDiagnostics); + Assert.Contains("CS0535", diagnostic, StringComparison.Ordinal); + Assert.Contains("IModeProperty.Mode", diagnostic, StringComparison.Ordinal); + } + else + { + Assert.Empty(run.CallerDiagnostics); + } + } + } + finally + { + File.Delete(path); + } + } + + /// + /// Separates the frozen OLD constructor defect from a corrected, unmigrated tooling control. + /// + [Fact] + public void FrozenOldConstructorLimitationsHaveCorrectedToolingControl() + { + string? oldPath = Environment.GetEnvironmentVariable("CSWIN32_COMPAT_OLD_METADATA"); + string? controlPath = Environment.GetEnvironmentVariable("CSWIN32_COMPAT_TOOLING_CONTROL_METADATA"); + Assert.SkipWhen(string.IsNullOrEmpty(oldPath) || string.IsNullOrEmpty(controlPath), "Supply the frozen OLD snapshot and a separately pinned unmigrated tooling control."); + Assert.Equal(Environment.GetEnvironmentVariable("CSWIN32_COMPAT_OLD_SHA256"), HashFile(oldPath!)); + Assert.Equal(Environment.GetEnvironmentVariable("CSWIN32_COMPAT_TOOLING_CONTROL_SHA256"), HashFile(controlPath!)); + string[] oldLimitations = MetadataConstructorLimitations(oldPath!); + string[] controlLimitations = MetadataConstructorLimitations(controlPath!); + Assert.Contains(oldLimitations, limitation => limitation.Contains("AssociatedEnumAttribute", StringComparison.Ordinal) && limitation.Contains("instance MemberRef present=True", StringComparison.Ordinal)); + Assert.Empty(controlLimitations); + if (Environment.GetEnvironmentVariable("CSWIN32_COMPAT_OUTPUT") is string output) + { + string report = JsonSerializer.Serialize( + new + { + ControlKind = "unmigrated SDK; physical constructor check only; not WinHttp NEW acceptance", + OldMetadataSHA256 = HashFile(oldPath!), + ControlMetadataSHA256 = HashFile(controlPath!), + OldMetadataPhysicalLimitations = oldLimitations, + ControlMetadataPhysicalLimitations = controlLimitations, + }, + new JsonSerializerOptions { WriteIndented = true }); + File.WriteAllText(Path.Combine(output, "unmigrated-tooling-control.json"), report); + } + } + + /// + /// Runs the real SDK pilot only when both immutable snapshots and their hashes are supplied. + /// + /// The pinned marshaling option. + [Theory] + [InlineData(true)] + [InlineData(false)] + public void WinHttpThreeWayMatrix(bool allowMarshaling) + { + RequireBaseline(); + string? oldPath = Environment.GetEnvironmentVariable("CSWIN32_COMPAT_OLD_METADATA"); + string? newPath = Environment.GetEnvironmentVariable("CSWIN32_COMPAT_NEW_METADATA"); + Assert.SkipWhen(string.IsNullOrEmpty(oldPath), "Run tools\\Test-HeaderAnnotationCompatibility.ps1 with the pinned WinHttp snapshots."); + Assert.Equal(Environment.GetEnvironmentVariable("CSWIN32_COMPAT_OLD_SHA256"), HashFile(oldPath!)); + if (!string.IsNullOrEmpty(newPath)) + { + Assert.Equal(Environment.GetEnvironmentVariable("CSWIN32_COMPAT_NEW_SHA256"), HashFile(newPath)); + } + + // The full caller corpus is shared verbatim between the three legs. + string callers = """ + using System; + using System.Runtime.InteropServices; + using Windows.Win32; + using Windows.Win32.Foundation; + using Windows.Win32.Networking.WinHttp; + [assembly: System.Runtime.Versioning.SupportedOSPlatform("windows")] + static unsafe class WinHttpCaller + { + static void EnumFields(WINHTTP_PROXY_INFO proxy, WINHTTP_CREDS credentials, WINHTTP_CREDS_EX extended, URL_COMPONENTS url, WINHTTP_PROXY_RESULT_ENTRY entry) + { + proxy.dwAccessType = WINHTTP_ACCESS_TYPE.WINHTTP_ACCESS_TYPE_NO_PROXY; + ref WINHTTP_ACCESS_TYPE access = ref proxy.dwAccessType; + credentials.dwAuthScheme = WINHTTP_CREDS_AUTHSCHEME.WINHTTP_AUTH_SCHEME_BASIC; + extended.dwAuthScheme = WINHTTP_CREDS_AUTHSCHEME.WINHTTP_AUTH_SCHEME_NEGOTIATE; + ref WINHTTP_CREDS_AUTHSCHEME auth = ref credentials.dwAuthScheme; + url.nScheme = WINHTTP_INTERNET_SCHEME.WINHTTP_INTERNET_SCHEME_HTTP; + url.nScheme = WINHTTP_INTERNET_SCHEME.WINHTTP_INTERNET_SCHEME_HTTPS; + entry.ProxyScheme = WINHTTP_INTERNET_SCHEME.WINHTTP_INTERNET_SCHEME_FTP; + entry.ProxyScheme = WINHTTP_INTERNET_SCHEME.WINHTTP_INTERNET_SCHEME_SOCKS; + ref WINHTTP_INTERNET_SCHEME scheme = ref url.nScheme; + } + static void Raw(PCWSTR agent, PCWSTR server, PCWSTR verb, PCWSTR path, void* data, WINHTTP_AUTOPROXY_OPTIONS* autoProxy) + { + void* session = PInvoke.WinHttpOpen(agent, WINHTTP_ACCESS_TYPE.WINHTTP_ACCESS_TYPE_NO_PROXY, default(PCWSTR), default(PCWSTR), 0); + void* connection = PInvoke.WinHttpConnect(session, server, 443, 0); + WINHTTP_OPEN_REQUEST_FLAGS flags = WINHTTP_OPEN_REQUEST_FLAGS.WINHTTP_FLAG_SECURE | WINHTTP_OPEN_REQUEST_FLAGS.WINHTTP_FLAG_REFRESH; + void* request = PInvoke.WinHttpOpenRequest(connection, verb, path, default(PCWSTR), default(PCWSTR), null, flags); + uint read; + PInvoke.WinHttpReadData(request, data, 16, &read); + WINHTTP_PROXY_INFO proxy; + PInvoke.WinHttpGetProxyForUrl(session, path, autoProxy, &proxy); + void* resolver; + PInvoke.WinHttpCreateProxyResolver(session, &resolver); + URL_COMPONENTS components = default; + PInvoke.WinHttpCrackUrl(path, 0, 0, &components); + uint length = 0; + WIN_HTTP_CREATE_URL_FLAGS createFlags = WIN_HTTP_CREATE_URL_FLAGS.ICU_ESCAPE | WIN_HTTP_CREATE_URL_FLAGS.ICU_REJECT_USERPWD; + PInvoke.WinHttpCreateUrl(&components, createFlags, default(PWSTR), &length); + uint index = 0, headerCount = 0, bufferLength = 16; + PInvoke.WinHttpQueryHeadersEx(request, 0, 0, 0, &index, null, data, &bufferLength, null, &headerCount); + PInvoke.WinHttpCloseHandle(resolver); + PInvoke.WinHttpCloseHandle(request); + PInvoke.WinHttpCloseHandle(connection); + PInvoke.WinHttpCloseHandle(session); + } + static void Friendly(void* request, byte* data) + { + void* session = PInvoke.WinHttpOpen("compatibility", WINHTTP_ACCESS_TYPE.WINHTTP_ACCESS_TYPE_NO_PROXY, null, null, 0); + void* connection = PInvoke.WinHttpConnect(session, "example.invalid", 443, 0); + uint read = 0; + PInvoke.WinHttpReadData(request, new Span(data, 16), ref read); + PInvoke.WinHttpCreateProxyResolver(session, out void* resolver); + } + /*CALLBACK*/ + } + """; + string callback = allowMarshaling + ? """ + static void Register(void* request) => PInvoke.WinHttpSetStatusCallback(request, Callback, 0, 0); + static void Callback(void* handle, nuint context, uint status, void* info, uint length) { } + """ + : """ + static void Register(void* request) => PInvoke.WinHttpSetStatusCallback(request, &Callback, 0, 0); + [UnmanagedCallersOnly(CallConvs = new[] { typeof(System.Runtime.CompilerServices.CallConvStdcall) })] + static void Callback(void* handle, nuint context, uint status, void* info, uint length) { } + """; + callers = callers.Replace("/*CALLBACK*/", callback, StringComparison.Ordinal); + MatrixResult result = this.RunMatrix("winhttp", oldPath!, newPath, WinHttpRequests, callers, allowMarshaling); + Assert.Empty(result.Runs[0].CallerDiagnostics); + Assert.Empty(result.Runs[1].CallerDiagnostics); + Assert.Equal(result.Runs[0].PublicSymbols, result.Runs[1].PublicSymbols); + Assert.SkipWhen(string.IsNullOrEmpty(newPath), "NEW WinHttp metadata is pending; the OLD baseline/proposed pair was checked and reported."); + RunResult migrated = result.Runs[2]; + (string Suffix, int Value)[] schemes = [("FTP", 3), ("HTTP", 1), ("HTTPS", 2), ("SOCKS", 4)]; + const string schemeSymbol = "Field global::Windows.Win32.Networking.WinHttp.WINHTTP_INTERNET_SCHEME."; + Assert.Equal("residual-source-break", migrated.Grade); + Assert.Equal( + schemes.Select(scheme => $"{schemeSymbol}WINHTTP_INTERNET_SCHEME_{scheme.Suffix} = {scheme.Value}").ToArray(), + migrated.RemovedPublicSymbols); + Assert.Equal( + schemes.Select(scheme => $"{schemeSymbol}INTERNET_SCHEME_{scheme.Suffix} = {scheme.Value}").ToArray(), + migrated.AddedPublicSymbols); + string renamedCode = schemes.Aggregate(result.Runs[0].GeneratedCode, (code, scheme) => code.Replace($"WINHTTP_INTERNET_SCHEME_{scheme.Suffix}", $"INTERNET_SCHEME_{scheme.Suffix}", StringComparison.Ordinal)); + Assert.Equal(renamedCode, migrated.GeneratedCode); + Assert.Equal(4, migrated.CallerDiagnostics.Length); + Assert.All(migrated.CallerDiagnostics, diagnostic => + { + Assert.Contains("CS0117", diagnostic, StringComparison.Ordinal); + Assert.Contains("WINHTTP_INTERNET_SCHEME_", diagnostic, StringComparison.Ordinal); + }); + string adjustedCaller = result.UnchangedCaller.Replace("WINHTTP_INTERNET_SCHEME.WINHTTP_INTERNET_SCHEME_", "WINHTTP_INTERNET_SCHEME.INTERNET_SCHEME_", StringComparison.Ordinal); + RunResult adjusted = this.Generate("new-proposed-scheme-members-adjusted", newPath!, WinHttpRequests, adjustedCaller, result.Options, baseline: false); + if (Environment.GetEnvironmentVariable("CSWIN32_COMPAT_OUTPUT") is string output) + { + string report = JsonSerializer.Serialize( + new + { + AdjustmentKind = "explicit caller member-name edits; not unchanged-caller acceptance", + result.BaselineCommit, + result.ProposedCommit, + result.ProposedAssemblySHA256, + result.NewMetadataSHA256, + result.Compilation, + result.Options, + result.Requests, + UnchangedCallerGrade = migrated.Grade, + UnchangedCallerDiagnostics = migrated.CallerDiagnostics, + MemberReplacements = schemes.Select(scheme => new + { + Old = $"WINHTTP_INTERNET_SCHEME.WINHTTP_INTERNET_SCHEME_{scheme.Suffix}", + New = $"WINHTTP_INTERNET_SCHEME.INTERNET_SCHEME_{scheme.Suffix}", + scheme.Value, + }).ToArray(), + AdjustedCaller = adjustedCaller, + AdjustedCallerDiagnostics = adjusted.CallerDiagnostics, + }, + new JsonSerializerOptions { WriteIndented = true }); + File.WriteAllText(Path.Combine(output, $"winhttp-{(allowMarshaling ? "marshaling" : "no-marshaling")}-scheme-member-adjustment.json"), report); + } + + Assert.Empty(adjusted.CallerDiagnostics); + Assert.Equal(migrated.PublicSymbols, adjusted.PublicSymbols); + } + + private static string Grade(RunResult baseline, RunResult proposed) => proposed.CallerDiagnostics.Length > 0 ? "residual-source-break" + : proposed.GeneratedCode == baseline.GeneratedCode ? "identical-generation" : "source-compatible-difference"; + + private static void RequireBaseline() + { + Assert.SkipWhen(string.IsNullOrEmpty(Environment.GetEnvironmentVariable("CSWIN32_COMPAT_BASELINE_DLL")), "Run tools\\Test-HeaderAnnotationCompatibility.ps1 to build the pinned baseline."); + Assert.Matches("^[0-9a-f]{40}$", Environment.GetEnvironmentVariable("CSWIN32_COMPAT_BASELINE_COMMIT") ?? string.Empty); + } + + private static string HashFile(string path) => Convert.ToHexString(SHA256.HashData(File.ReadAllBytes(path))); + + private static string GetCanonicalCaller(bool allowMarshaling) + { + string callback = allowMarshaling + ? "static void Register() => PInvoke.UseCallback(Callback); static MODE Callback(MODE mode, MODE* data) => mode;" + : "static void Register() => PInvoke.UseCallback(&Callback); [UnmanagedCallersOnly(CallConvs = new[] { typeof(System.Runtime.CompilerServices.CallConvStdcall) })] static MODE Callback(MODE mode, MODE* data) => mode;"; + return AssociatedEnumTests.Caller + Environment.NewLine + "namespace CallbackFixture { using System.Runtime.InteropServices; using Windows.Win32; using Windows.Win32.Test; static unsafe class CallbackCaller { " + callback + " } }"; + } + + private static string[] MetadataConstructorLimitations(string path) + { + using FileStream stream = File.OpenRead(path); + using PEReader pe = new(stream); + MetadataReader reader = pe.GetMetadataReader(); + HashSet instanceReferences = new(StringComparer.Ordinal); + foreach (MemberReferenceHandle handle in reader.MemberReferences) + { + MemberReference reference = reader.GetMemberReference(handle); + if (reference.Parent.Kind == HandleKind.TypeReference && reader.StringComparer.Equals(reference.Name, ".ctor") + && reader.GetBlobReader(reference.Signature).ReadSignatureHeader().IsInstance) + { + TypeReference type = reader.GetTypeReference((TypeReferenceHandle)reference.Parent); + instanceReferences.Add($"{reader.GetString(type.Namespace)}.{reader.GetString(type.Name)}"); + } + } + + List limitations = new(); + foreach (TypeDefinitionHandle handle in reader.TypeDefinitions) + { + TypeDefinition type = reader.GetTypeDefinition(handle); + if (!reader.StringComparer.Equals(type.Namespace, "Windows.Win32.Foundation.Metadata")) + { + continue; + } + + foreach (MethodDefinitionHandle methodHandle in type.GetMethods()) + { + MethodDefinition method = reader.GetMethodDefinition(methodHandle); + if (reader.StringComparer.Equals(method.Name, ".ctor") && !reader.GetBlobReader(method.Signature).ReadSignatureHeader().IsInstance) + { + string name = $"{reader.GetString(type.Namespace)}.{reader.GetString(type.Name)}"; + limitations.Add($"{name}..ctor MethodDef lacks HASTHIS; signature={Convert.ToHexString(reader.GetBlobBytes(method.Signature))}; instance MemberRef present={instanceReferences.Contains(name)}."); + } + } + } + + return limitations.ToArray(); + } + + private static string[] PublicSymbols(CSharpCompilation compilation) + { + List symbols = new(); + void Visit(INamespaceOrTypeSymbol container) + { + foreach (ISymbol member in container.GetMembers()) + { + if (member is INamespaceSymbol ns) + { + Visit(ns); + } + else if (member.DeclaredAccessibility == Accessibility.Public && !member.IsImplicitlyDeclared) + { + string detail = member switch + { + INamedTypeSymbol type => $" : {type.BaseType?.ToDisplayString(ApiDisplay)}; enum={type.EnumUnderlyingType?.ToDisplayString(ApiDisplay)}; flags={type.GetAttributes().Any(attribute => attribute.AttributeClass?.ToDisplayString() == "System.FlagsAttribute")}", + IFieldSymbol { HasConstantValue: true } field => $" = {Convert.ToString(field.ConstantValue, System.Globalization.CultureInfo.InvariantCulture)}", + _ => string.Empty, + }; + symbols.Add($"{member.Kind} {member.ToDisplayString(ApiDisplay)}{detail}"); + if (member is INamedTypeSymbol nested) + { + Visit(nested); + } + } + } + } + + Visit(compilation.Assembly.GlobalNamespace); + return symbols.OrderBy(symbol => symbol, StringComparer.Ordinal).ToArray(); + } + + private MatrixResult RunMatrix(string name, string oldMetadata, string? newMetadata, string[] requests, string callers, bool allowMarshaling, string compatibilityScope = "synthetic-enum-migration") + { + callers = TestUtils.NormalizeToExpectedLineEndings(callers); + GeneratorOptions options = DefaultTestGeneratorOptions with { Public = true, AllowMarshaling = allowMarshaling, UseSafeHandles = true }; + RunResult oldBaseline = this.Generate("old-baseline", oldMetadata, requests, callers, options, baseline: true); + RunResult oldProposed = this.Generate("old-proposed", oldMetadata, requests, callers, options, baseline: false); + RunResult[] runs = string.IsNullOrEmpty(newMetadata) ? [oldBaseline, oldProposed] + : [oldBaseline, oldProposed, this.Generate("new-proposed", newMetadata!, requests, callers, options, baseline: false)]; + runs = runs.Select(run => run with + { + Grade = Grade(oldBaseline, run), + RemovedPublicSymbols = oldBaseline.PublicSymbols.Except(run.PublicSymbols, StringComparer.Ordinal).ToArray(), + AddedPublicSymbols = run.PublicSymbols.Except(oldBaseline.PublicSymbols, StringComparer.Ordinal).ToArray(), + }).ToArray(); + MatrixResult result = new( + compatibilityScope, + Environment.GetEnvironmentVariable("CSWIN32_COMPAT_BASELINE_COMMIT")!, + HashFile(Environment.GetEnvironmentVariable("CSWIN32_COMPAT_BASELINE_DLL")!), + Environment.GetEnvironmentVariable("CSWIN32_COMPAT_PROPOSED_COMMIT") ?? "uncommitted", + HashFile(typeof(Generator).Assembly.Location), + typeof(CSharpCompilation).Assembly.GetName().Version!.ToString(), + "net8.0-windows; x64; unsafe; C# 12", + options, + requests, + HashFile(oldMetadata), + string.IsNullOrEmpty(newMetadata) ? null : HashFile(newMetadata!), + callers, + runs, + MetadataConstructorLimitations(oldMetadata), + string.IsNullOrEmpty(newMetadata) ? [] : MetadataConstructorLimitations(newMetadata!)); + string? output = Environment.GetEnvironmentVariable("CSWIN32_COMPAT_OUTPUT"); + if (!string.IsNullOrEmpty(output)) + { + Directory.CreateDirectory(output); + string prefix = Path.Combine(output, $"{name}-{(allowMarshaling ? "marshaling" : "no-marshaling")}"); + File.WriteAllText(prefix + ".json", JsonSerializer.Serialize(result, new JsonSerializerOptions { WriteIndented = true })); + foreach (RunResult run in result.Runs) + { + File.WriteAllText(prefix + "-" + run.Name + ".g.cs", run.GeneratedCode); + } + } + + foreach (RunResult run in result.Runs) + { + this.logger.WriteLine($"{name}/{allowMarshaling}/{run.Name}: {run.Grade}; {run.PublicSymbols.Length} public symbols; {run.CallerDiagnostics.Length} caller diagnostics."); + } + + Assert.Empty(result.NewMetadataPhysicalLimitations); + return result; + } + + private RunResult Generate(string name, string metadata, string[] requests, string callers, GeneratorOptions options, bool baseline) + { + CSharpCompilation references = this.starterCompilations["net8.0-x64"]; + IEnumerable> units; + if (baseline) + { + using BaselineGenerator generator = new(metadata, options, references, this.parseOptions); + foreach (string request in requests) + { + generator.Generate(request); + } + + units = generator.GetCompilationUnits().ToArray(); + } + else + { + using Generator generator = new(metadata, null, [], options, references, this.parseOptions); + foreach (string request in requests) + { + Assert.True(generator.TryGenerate(request, TestContext.Current.CancellationToken)); + } + + units = generator.GetCompilationUnits(TestContext.Current.CancellationToken).ToArray(); + } + + string code = string.Join(Environment.NewLine, units.OrderBy(unit => unit.Key, StringComparer.Ordinal).Select(unit => unit.Value.ToFullString())); + CSharpCompilation generated = references.AddSyntaxTrees(units.Select(unit => CSharpSyntaxTree.ParseText(unit.Value.ToFullString(), this.parseOptions, unit.Key))); + this.AssertNoDiagnostics(generated, logAllGeneratedCode: false); + string[] symbols = PublicSymbols(generated); + CSharpCompilation withCallers = generated.AddSyntaxTrees(CSharpSyntaxTree.ParseText(callers, this.parseOptions, "UnchangedCaller.cs")); + string[] diagnostics = withCallers.GetDiagnostics(TestContext.Current.CancellationToken) + .Where(diagnostic => diagnostic.Severity == DiagnosticSeverity.Error).Select(diagnostic => diagnostic.ToString()).ToArray(); + if (diagnostics.Length == 0) + { + this.AssertNoDiagnostics(withCallers, logAllGeneratedCode: false); + } + + return new(name, code, symbols, diagnostics, "ungraded", [], []); + } + + private sealed record RunResult(string Name, string GeneratedCode, string[] PublicSymbols, string[] CallerDiagnostics, string Grade, string[] RemovedPublicSymbols, string[] AddedPublicSymbols); + + private sealed record MatrixResult( + string CompatibilityScope, + string BaselineCommit, + string BaselineAssemblySHA256, + string ProposedCommit, + string ProposedAssemblySHA256, + string RoslynVersion, + string Compilation, + GeneratorOptions Options, + string[] Requests, + string OldMetadataSHA256, + string? NewMetadataSHA256, + string UnchangedCaller, + RunResult[] Runs, + string[] OldMetadataPhysicalLimitations, + string[] NewMetadataPhysicalLimitations); + + private sealed class BaselineGenerator : IDisposable + { + private readonly AssemblyLoadContext context = new("CsWin32 compatibility baseline", isCollectible: true); + private readonly object generator; + private readonly Type generatorType; + + /// + /// Initializes a new instance of the class. + /// + /// The pinned metadata snapshot. + /// The shared generation options. + /// The shared reference compilation. + /// The shared language options. + internal BaselineGenerator(string metadata, GeneratorOptions options, CSharpCompilation compilation, CSharpParseOptions parseOptions) + { + Assembly assembly = this.context.LoadFromAssemblyPath(Path.GetFullPath(Environment.GetEnvironmentVariable("CSWIN32_COMPAT_BASELINE_DLL")!)); + this.generatorType = assembly.GetType(typeof(Generator).FullName!, throwOnError: true)!; + Type optionsType = assembly.GetType(typeof(GeneratorOptions).FullName!, throwOnError: true)!; + object baselineOptions = JsonSerializer.Deserialize(JsonSerializer.Serialize(options), optionsType)!; + this.generator = Activator.CreateInstance(this.generatorType, [metadata, null, Array.Empty(), baselineOptions, compilation, parseOptions])!; + } + + public void Dispose() + { + ((IDisposable)this.generator).Dispose(); + this.context.Unload(); + } + + /// + /// Generates one requested API using the isolated baseline assembly. + /// + /// The requested API. + internal void Generate(string api) + { + MethodInfo method = this.generatorType.GetMethod("TryGenerate", [typeof(string), typeof(IReadOnlyCollection).MakeByRefType(), typeof(CancellationToken)])!; + Assert.Equal(true, method.Invoke(this.generator, [api, null, TestContext.Current.CancellationToken])); + } + + /// + /// Gets the baseline-generated syntax trees. + /// + /// The generated compilation units. + internal IEnumerable> GetCompilationUnits() => + (IEnumerable>)this.generatorType.GetMethod("GetCompilationUnits")!.Invoke(this.generator, [TestContext.Current.CancellationToken])!; + } +} diff --git a/test/Microsoft.Windows.CsWin32.Tests/StructTests.cs b/test/Microsoft.Windows.CsWin32.Tests/StructTests.cs index 1dac4bc5..b4368033 100644 --- a/test/Microsoft.Windows.CsWin32.Tests/StructTests.cs +++ b/test/Microsoft.Windows.CsWin32.Tests/StructTests.cs @@ -152,8 +152,10 @@ public void FieldWithAssociatedEnum() this.GenerateApi("SHDESCRIPTIONID"); var type = (StructDeclarationSyntax)Assert.Single(this.FindGeneratedType("SHDESCRIPTIONID")); - PropertyDeclarationSyntax property = Assert.Single(type.Members.OfType(), m => m.Identifier.ValueText == "dwDescriptionId"); - Assert.Equal("SHDID_ID", Assert.IsType(property.Type).Identifier.ValueText); + FieldDeclarationSyntax field = Assert.Single(type.Members.OfType(), m => m.Declaration.Variables.Any(variable => variable.Identifier.ValueText == "dwDescriptionId")); + Assert.Contains("SHDID_ID", field.Declaration.Type.ToString(), StringComparison.Ordinal); + this.compilation = this.AddCode("using Windows.Win32.UI.Shell; static class AssociatedFieldCaller { static void Use(SHDESCRIPTIONID record) { ref SHDID_ID id = ref record.dwDescriptionId; } }"); + this.AssertNoDiagnostics(); } [Fact] diff --git a/tools/Test-HeaderAnnotationCompatibility.ps1 b/tools/Test-HeaderAnnotationCompatibility.ps1 new file mode 100644 index 00000000..db78a46e --- /dev/null +++ b/tools/Test-HeaderAnnotationCompatibility.ps1 @@ -0,0 +1,112 @@ +<# +.SYNOPSIS +Builds a pinned CsWin32 baseline and runs the canonical and optional full-image consumer matrices. +.DESCRIPTION +All git mutations and generated artifacts stay under this worktree. The caller source, requested +APIs, generation options and compilation context are shared between the three matrix legs. +No package feeds or dependency versions are changed. +An optional unmigrated tooling control audits physical constructor signatures separately, +without replacing OLD or standing in for NEW consumer acceptance. +#> +[CmdletBinding()] +param( + [ValidatePattern('^[0-9a-f]{40}$')] + [string]$BaselineCommit = 'f3f5a69aa4823466ef7b55bbdc8e5ba41ec021e6', + [string]$OldMetadata, + [ValidatePattern('^[0-9A-Fa-f]{64}$')] + [string]$OldSHA256, + [string]$NewMetadata, + [ValidatePattern('^[0-9A-Fa-f]{64}$')] + [string]$NewSHA256, + [string]$ToolingControlMetadata, + [ValidatePattern('^[0-9A-Fa-f]{64}$')] + [string]$ToolingControlSHA256 +) + +$ErrorActionPreference = 'Stop' +$repo = Split-Path $PSScriptRoot -Parent +$output = Join-Path $repo 'obj\HeaderAnnotationCompatibility' +$baseline = Join-Path $output "baseline-$BaselineCommit" +$baselineProject = Join-Path $baseline 'src\Microsoft.Windows.CsWin32\Microsoft.Windows.CsWin32.csproj' +$baselineAssembly = Join-Path $baseline 'bin\Microsoft.Windows.CsWin32\Debug\netstandard2.0\Microsoft.Windows.CsWin32.dll' +New-Item -ItemType Directory -Path $output -Force | Out-Null + +if ($OldMetadata -or $NewMetadata -or $OldSHA256 -or $NewSHA256) { + if (!$OldMetadata -or !$OldSHA256 -or ([bool]$NewMetadata -ne [bool]$NewSHA256)) { + throw 'Supply the OLD metadata and hash, plus either both NEW metadata and hash or neither.' + } + $OldMetadata = (Resolve-Path $OldMetadata).Path + if ((Get-FileHash $OldMetadata -Algorithm SHA256).Hash -ne $OldSHA256) { + throw 'The OLD metadata snapshot does not match its pinned SHA256.' + } + if ($NewMetadata) { + $NewMetadata = (Resolve-Path $NewMetadata).Path + if ((Get-FileHash $NewMetadata -Algorithm SHA256).Hash -ne $NewSHA256) { + throw 'The NEW metadata snapshot does not match its pinned SHA256.' + } + } +} + +if ($ToolingControlMetadata -or $ToolingControlSHA256) { + if (!$OldMetadata -or !$ToolingControlMetadata -or !$ToolingControlSHA256) { + throw 'A tooling control requires the frozen OLD image and both control metadata and hash.' + } + $ToolingControlMetadata = (Resolve-Path $ToolingControlMetadata).Path + if ((Get-FileHash $ToolingControlMetadata -Algorithm SHA256).Hash -ne $ToolingControlSHA256) { + throw 'The unmigrated tooling control does not match its pinned SHA256.' + } +} + +if (!(Test-Path $baselineProject)) { + $archive = Join-Path $output "$BaselineCommit.zip" + & git -C $repo archive --format=zip "--output=$archive" $BaselineCommit + if ($LASTEXITCODE) { throw 'Could not archive the pinned baseline commit.' } + Expand-Archive -LiteralPath $archive -DestinationPath $baseline + Remove-Item -LiteralPath $archive +} + +$priorEnvironment = @{} +$variables = @('NBGV_GitEngine', 'CSWIN32_COMPAT_BASELINE_DLL', 'CSWIN32_COMPAT_BASELINE_COMMIT', + 'CSWIN32_COMPAT_OLD_METADATA', 'CSWIN32_COMPAT_NEW_METADATA', 'CSWIN32_COMPAT_OLD_SHA256', 'CSWIN32_COMPAT_NEW_SHA256', + 'CSWIN32_COMPAT_TOOLING_CONTROL_METADATA', 'CSWIN32_COMPAT_TOOLING_CONTROL_SHA256', 'CSWIN32_COMPAT_OUTPUT', 'CSWIN32_COMPAT_PROPOSED_COMMIT') +foreach ($variable in $variables) { + $priorEnvironment[$variable] = [Environment]::GetEnvironmentVariable($variable, 'Process') +} + +try { + $env:NBGV_GitEngine = 'Disabled' + $buildLog = Join-Path $output 'baseline-build.log' + & dotnet build $baselineProject --no-restore --verbosity quiet *> $buildLog + if ($LASTEXITCODE -and (Select-String -LiteralPath $buildLog -Pattern 'NETSDK1004' -Quiet)) { + & dotnet restore $baselineProject --verbosity quiet + if ($LASTEXITCODE) { throw 'Baseline restore failed.' } + & dotnet build $baselineProject --no-restore --verbosity quiet *> $buildLog + } + if ($LASTEXITCODE) { + Get-Content -LiteralPath $buildLog + throw 'Baseline build failed.' + } + $env:CSWIN32_COMPAT_BASELINE_DLL = $baselineAssembly + $env:CSWIN32_COMPAT_BASELINE_COMMIT = $BaselineCommit + $env:CSWIN32_COMPAT_PROPOSED_COMMIT = & git -C $repo rev-parse HEAD + if ($LASTEXITCODE) { throw 'Could not resolve the proposed commit.' } + if (& git -C $repo status --porcelain) { + $env:CSWIN32_COMPAT_PROPOSED_COMMIT += '+working-tree' + } + $env:CSWIN32_COMPAT_OUTPUT = $output + $env:CSWIN32_COMPAT_OLD_METADATA = $OldMetadata + $env:CSWIN32_COMPAT_NEW_METADATA = $NewMetadata + $env:CSWIN32_COMPAT_OLD_SHA256 = $OldSHA256.ToUpperInvariant() + $env:CSWIN32_COMPAT_NEW_SHA256 = $NewSHA256.ToUpperInvariant() + $env:CSWIN32_COMPAT_TOOLING_CONTROL_METADATA = $ToolingControlMetadata + $env:CSWIN32_COMPAT_TOOLING_CONTROL_SHA256 = $ToolingControlSHA256.ToUpperInvariant() + & dotnet test (Join-Path $repo 'test\Microsoft.Windows.CsWin32.Tests\Microsoft.Windows.CsWin32.Tests.csproj') --no-restore ` + --filter 'FullyQualifiedName~HeaderAnnotationCompatibilityTests|FullyQualifiedName~AssociatedEnumTests.CallerMutationControls' --logger 'trx;LogFileName=compatibility.trx' --results-directory $output --verbosity quiet + if ($LASTEXITCODE) { throw "Compatibility checks failed. See $output." } + Write-Output "Compatibility reports and generated sources: $output" +} +finally { + foreach ($variable in $variables) { + [Environment]::SetEnvironmentVariable($variable, $priorEnvironment[$variable], 'Process') + } +}