diff --git a/.agents/skills/loom-framework/SKILL.md b/.agents/skills/loom-framework/SKILL.md index 14dffd3ad..f817e17bd 100644 --- a/.agents/skills/loom-framework/SKILL.md +++ b/.agents/skills/loom-framework/SKILL.md @@ -75,6 +75,15 @@ consumer validation. ## Architecture Boundaries +- Distinct HTTP media mappings belong to endpoint/response occurrences, not to + reusable service unions. Transport IR retains each branch's schema, examples, + codec and field encoding. Form roots have independent native-presence carriers; + nested JSON fields retain JSON carriers. Conversion contexts must honor both + true and false recorded presence policies. Incoming matching and outgoing + consistency share `internal/httpcodec`; select factories before response status. + Keep the media-selection and physical-layout models and generated exchange + specimen aligned. + - `admission.Rate` serializes the maintained token bucket's Allow and complete rate/burst policy updates. Preserve accrued allowance, clamp on reductions and never reset quota on reconfiguration. `RateAdmission` owns bounded cooperative @@ -1225,6 +1234,17 @@ There is one DSL parser, one shared semantic IR, and one renderer. the same projected values emitted in the schema. Ambiguous wire shapes remain omitted. Never reinterpret typed bytes separately for a competing string branch; `internal/ir/tla/byte_examples` records that counterexample. +- Schema example analysis retains the complete prepared authored group in order. + Carry plural values through component annotation snapshots, nullable wrappers, + async overlays, schema identity and rendering. Emit `examples` instead of + `example` when a plural group survives projection; never wrap schema values + in media-level Example Objects. Keep the schema-examples model and import + round-trip coverage aligned. + Import carries schema/media provenance through `openapi:example:scope`; + select each scope's group and representative before rendering. An entirely + excluded group suppresses synthesis. Apply examples by replacement so repeated + nullable-overlay visits cannot append the same group twice. + - The IR analyzer consumes prepared examples for bodies, nested parameters, response headers and async messages. Track the captured example position separately from structural schema authority: a neutral reusable schema can diff --git a/.agents/skills/loom-framework/references/formal-models.md b/.agents/skills/loom-framework/references/formal-models.md index 6a0584fb0..b9e4a16fd 100644 --- a/.agents/skills/loom-framework/references/formal-models.md +++ b/.agents/skills/loom-framework/references/formal-models.md @@ -20,6 +20,13 @@ The [documentation schema admission model](../../../../expr/tla/documentation_sc separates valid documentation schemas from unsupported generated codecs and checks that validation visits cannot leak between those contexts. +## HTTP media contracts + +The [media contract models](../../../../internal/httpcodec/tla/media_contracts/README.md) +check most-specific selection, concrete publication, and agreement between +physical branch declarations and conversion contexts. The inherited-layout +control reproduces the former failure to honor an explicitly native child layout. + ## Unary admission The [concurrency model](../../../../admission/tla/concurrency/README.md) checks @@ -253,6 +260,7 @@ Name-allocation assumptions are not proved merely by proving declaration lookup. | Code concern | Model | | --- | --- | | Nested synthesized union selection and occurrence-specific example caches | [Nested union examples](../../../../http/codegen/openapi/internal/ir/tla/nested_union_examples/README.md) | +| Ordered schema examples, explicit nulls and duplicate values | [Schema examples](../../../../http/codegen/openapi/internal/ir/tla/schema_examples/README.md); representative-only counterexample and bounded sequence preservation, with rendered/import tests for concrete projection and serialization | | Alias-bound intersection and decoded Bytes length projection into base64 schema branches | `AliasLengthBounds.lean` and `ByteLengthProjection.lean` in the [Lean value model](../../../../expr/lean/value_projection/README.md); [production owners and checks](../../../../internal/valuecontract/BYTE_SCHEMA.md) distinguish proved arithmetic from tested codec extraction and schema allocation | | One wire representation for byte/text branch comparison | [Byte example projection](../../../../http/codegen/openapi/internal/ir/tla/byte_examples/README.md) | | Security binding ownership, explicit reservations and deterministic component names | [Security bindings](../../../../http/codegen/openapi/internal/ir/tla/security_bindings/README.md) | diff --git a/.agents/skills/loom/SKILL.md b/.agents/skills/loom/SKILL.md index a33f64643..fe7addf02 100644 --- a/.agents/skills/loom/SKILL.md +++ b/.agents/skills/loom/SKILL.md @@ -12,6 +12,12 @@ request decoder accepts valid `+json` media everywhere; services still implement media-specific semantics. Do not combine static request media with a mapped Content-Type header, form/multipart, raw, JSON-RPC or WebSocket framing. +For distinct HTTP representations, map each selected `Body` union branch with +`MediaType(media, branch)` in the endpoint and response. Select a generated union +constructor in application code. Exact media are automatic; ranges require a +mapped concrete Content-Type. Factories receive only the selected branch, and +unknown codecs remain application-owned. See [HTTP media contracts](../../../docs/http-media-contracts.md). + Use this skill when an application consumes `github.com/CaliLuke/loom`: writing or changing `design/*.go`, regenerating `gen/`, implementing service methods, or wiring generated transports and runtime packages. @@ -407,6 +413,12 @@ Author text or byte values as usual; examples encode them once. Incomplete object examples are omitted; explicit null examples are retained for nullable schemas. These rules also apply to nested parameter schemas, response-header schemas, and streaming message schemas. +Multiple schema examples retain declaration order, duplicates and explicit nulls +in `examples`; a single value uses `example`. Schema entries are JSON values, +not named Example Objects. Regenerate OpenAPI to adopt this preservation. +Import preserves schema and media example ownership with +`Meta("openapi:example:scope", "schema")` or `"media"` inside `Example`. +Keep these markers when editing imported designs to avoid merging the groups. Synthesized tagged-union examples retain every nested envelope, including when branches share the same shape. Configured discriminator/value keys and branch tags apply at each level; authored examples keep their existing interpretation. diff --git a/codegen/go_transform_fallibility.go b/codegen/go_transform_fallibility.go index bb88cea8a..0722c7aa2 100644 --- a/codegen/go_transform_fallibility.go +++ b/codegen/go_transform_fallibility.go @@ -89,18 +89,38 @@ func transformContextLayout(ctx *AttributeContext) transformLayout { } func transformPresenceContext(source, target *expr.AttributeExpr, ta *TransformAttrs) *TransformAttrs { - if !attributeUsesJSONPresence(source, ta.SourceCtx) && !attributeUsesJSONPresence(target, ta.TargetCtx) { + sourceContext := transformPhysicalContext(source, ta.SourceCtx) + targetContext := transformPhysicalContext(target, ta.TargetCtx) + if sourceContext == ta.SourceCtx && targetContext == ta.TargetCtx { return ta } nested := *ta - nested.SourceCtx, nested.TargetCtx = ta.SourceCtx.Dup(), ta.TargetCtx.Dup() - if attributeUsesJSONPresence(source, ta.SourceCtx) { - nested.SourceCtx.JSONPresence, nested.SourceCtx.CollectionElementPresence = true, true + nested.SourceCtx, nested.TargetCtx = sourceContext, targetContext + return &nested +} + +func transformPhysicalContext(attribute *expr.AttributeExpr, context *AttributeContext) *AttributeContext { + if attribute == nil || context == nil { + return context } - if attributeUsesJSONPresence(target, ta.TargetCtx) { - nested.TargetCtx.JSONPresence, nested.TargetCtx.CollectionElementPresence = true, true + named, ok := attribute.Type.(expr.UserType) + if !ok { + return context } - return &nested + key := named.Hash() + presence, found := context.JSONPresenceTypes[key] + if !found { + return context + } + nested := context.Dup() + nested.JSONPresence, nested.CollectionElementPresence = presence, presence + if pointer, found := context.PresencePointerTypes[key]; found { + nested.Pointer = pointer + } + if defaults, found := context.PresenceUseDefaultTypes[key]; found { + nested.UseDefault = defaults + } + return nested } func transformCallArguments(source, target *expr.AttributeExpr, value string, ta *TransformAttrs) []jen.Code { diff --git a/codegen/go_transform_presence.go b/codegen/go_transform_presence.go index ee68c9bb8..13abc3a2d 100644 --- a/codegen/go_transform_presence.go +++ b/codegen/go_transform_presence.go @@ -314,11 +314,3 @@ func nativeObjectFieldPointer(parent *expr.MappedAttributeExpr, name string, att func isAnyPresenceAttribute(attribute *expr.AttributeExpr) bool { return attribute != nil && expr.IsAny(attribute.Type) } - -func attributeUsesJSONPresence(attribute *expr.AttributeExpr, context *AttributeContext) bool { - if attribute == nil || context == nil || len(context.JSONPresenceTypes) == 0 { - return false - } - userType, ok := attribute.Type.(expr.UserType) - return ok && context.JSONPresenceTypes[userType.Hash()] -} diff --git a/codegen/json_boundary_transform_test.go b/codegen/json_boundary_transform_test.go index 20974a9a2..99a459a0d 100644 --- a/codegen/json_boundary_transform_test.go +++ b/codegen/json_boundary_transform_test.go @@ -33,3 +33,19 @@ func TestTransformNamedJSONCollectionBoundary(t *testing.T) { }) } } + +func TestTransformNamedNativeBoundaryInsideJSON(t *testing.T) { + named := &expr.UserTypeExpr{TypeName: "FormVariant", AttributeExpr: &expr.AttributeExpr{Type: &expr.Object{{Name: "name", Attribute: &expr.AttributeExpr{Type: expr.String}}}}} + source := &expr.AttributeExpr{Type: named} + target := expr.DupAtt(source) + scope := NewNameScope() + sourceContext := NewAttributeContext(true, false, false, "", scope) + sourceContext.JSONPresence = true + sourceContext.JSONPresenceTypes = map[string]bool{named.Hash(): false} + targetContext := NewAttributeContext(false, false, true, "", scope) + plan, err := GoTransform(source, target, "body", "payload", sourceContext, targetContext, "unmarshal", true) + require.NoError(t, err) + require.NotContains(t, plan.Code, ".Value()") + require.Contains(t, plan.Code, "Name: body.Name") + require.True(t, sourceContext.JSONPresence) +} diff --git a/docs/dsl-reference.md b/docs/dsl-reference.md index 5fb17a670..969dccd79 100644 --- a/docs/dsl-reference.md +++ b/docs/dsl-reference.md @@ -921,6 +921,14 @@ encoding for byte fields, including fields authored as text and nested collections. This applies to body schemas, nested parameter schemas, response-header schemas, and `x-loom-async` message schemas in both output formats. Explicit null examples remain null for nullable schemas. +Multiple schema examples are emitted as an ordered JSON Schema `examples` array, +including duplicate values and explicit nulls. These are JSON values, without +the names and descriptions used by media-level Example Objects. A single example +continues to use `example`. Regenerate OpenAPI to preserve all authored values. +Imported examples retain their original schema or media scope through +`Meta("openapi:example:scope", "schema")` or `"media"` inside `Example`. +This keeps re-import from combining two independently authored example groups. +Unscoped examples remain available to both schema and media documentation. Synthesized tagged-union examples retain a discriminator/value envelope at every nesting level, including unions that reuse the same branch shapes in objects, arrays, and maps. Each envelope uses its configured field names and diff --git a/docs/http-media-contracts.md b/docs/http-media-contracts.md new file mode 100644 index 000000000..aa5fb3b41 --- /dev/null +++ b/docs/http-media-contracts.md @@ -0,0 +1,112 @@ +# Distinct HTTP media contracts + +Use `MediaType(media, branch)` when one HTTP body has different representations. +The service value is an ordinary typed union. HTTP chooses the branch from the +request or response `Content-Type` and encodes only that branch's value. It does +not send the union's JSON discriminator envelope. + +```go +var Message = Type("Message", func() { + OneOf("body", func() { + Attribute("document", Int) + Attribute("text", String) + }) + Required("body") +}) + +var _ = Service("messages", func() { + Method("exchange", func() { + Payload(Message) + Result(Message) + HTTP(func() { + POST("/messages") + Body("body") + MediaType("application/json", "document") + MediaType("text/plain", "text") + Response(StatusOK, func() { + Body("body") + MediaType("application/json", "document") + MediaType("text/plain", "text") + }) + }) + }) +}) +``` + +Regenerate the service and transports. Callers and service implementations select +a generated union constructor, such as `NewBodyText("hello")`. The generated +client publishes `text/plain`; the server decodes the text branch, validates it, +and passes the typed union to the service. A returned text branch produces a +text response. `Accept` does not select or change the returned branch. + +Each branch must have exactly one media mapping. Equal schemas can have separate +branches with separate examples. Examples on an enclosing payload or result are +projected to their selected branch; examples authored directly on a branch take +precedence. Ordinary union JSON serialization and single-media endpoints keep +their existing behavior. + +## Ranges and parameters + +Incoming selection prefers an exact type, then `type/*`, then `*/*`. Within one +type specificity, more matching authored parameters take precedence. Additional +incoming parameters are allowed. Equally specific overlapping declarations are +rejected. A parameter authored with an empty value still requires that parameter. +Media type names and charset values are case insensitive. + +A mapped `Content-Type` must be a non-nullable string. A range requires this +field on the payload or result envelope: + +```go +Header("content_type:Content-Type") +MediaType("application/*", "binary") +``` + +The caller or service must supply a concrete content type for that branch. It +must select the same branch under the incoming matching rules. An exact mapping +supplies its own content type when no mapped value is supplied. A mapped value +that would select another branch fails before encoding. OpenAPI describes this +selection through its content map, without an ignored `Content-Type` parameter. + +Missing, malformed, wildcard, and unmatched incoming content types are rejected. +An optional absent request body needs no content type. Empty text, binary, and form +bodies remain supplied values when their content type is present. An empty form +selects an object with all optional fields absent. JSON empty +input represents absence; JSON null follows the selected branch's null contract. + +## Codecs and encoding + +The generated transports call the configured encoder or decoder factory after +selection and pass only the selected typed branch. Built-in factories support +JSON (including `+json`), text, octet streams, URL-encoded forms, and multipart. +XML, GOB, and other concrete media types require an application codec factory; +they do not silently fall back to JSON. A range itself does not select a codec. + +A custom error formatter replaces the declared error body and uses the configured +encoder directly, as it does for single-media errors. + +Text requires a non-nullable string or byte branch. Octet streams require a +non-nullable byte branch. Form and multipart variants +require non-nullable objects without untagged unions. Form fields use the existing +scalar, repeated-array, and JSON-object field rules. Multipart records each +part's concrete content type. Per-part media ranges and lists are rejected because +there is no per-part concrete-type selector. Use the existing `openapi:encoding::contentType`, +`style`, `explode`, and `allowReserved` metadata on that branch to describe an +equivalent encoding; incompatible declarations fail validation. Custom part +headers and incompatible field serialization remain outside the import subset. + +Do not combine `MediaType` with static `ContentType`, documentation-only body +mappings, raw-body flags, endpoint-wide `FormRequest` or `MultipartRequest`, or +streaming/JSON-RPC framing. Forms belong to their individual media branches. +The enclosing union cannot be nullable; individual JSON branches may be nullable. + +## OpenAPI import + +The importer preserves each request and response content entry as a distinct +named branch, including entries with equal schemas. Media examples and field +encoding stay on their own branch. Ranges add a mapped content-type field so +application code can select a concrete representation. Re-import and regenerate +to replace the former shared-schema raw request body with typed variants. + +Multi-media form and multipart root maps are rejected; these variants require +structured object schemas. This replaces the former shared-schema raw fallback +for multi-media maps. Single-media root-map imports keep their existing behavior. diff --git a/docs/openapi-import-coverage.md b/docs/openapi-import-coverage.md index 79a9561d9..31dd6c711 100644 --- a/docs/openapi-import-coverage.md +++ b/docs/openapi-import-coverage.md @@ -73,6 +73,14 @@ so other uses of the base keep their field names and constraints. Re-import and regenerate affected designs, then use the generated Go selectors in service code. This does not assign protobuf tags or change protobuf field-name admission. +Multiple request and response media types retain independent schemas, examples, +and encoding contracts. They render as typed service unions with HTTP +`MediaType(media, branch)` mappings, including equal schemas under different +media types. Exact types outrank subtype and global ranges. Ranges expose a mapped +concrete Content-Type field. The generated client/server select a codec for the +selected branch; unknown codecs require an application factory. Single-media +behavior is unchanged. See [HTTP media contracts](http-media-contracts.md). + The following table summarizes fixed fields on objects that the importer reads. Specification extensions are patterns, not fixed fields. @@ -87,7 +95,7 @@ reads. Specification extensions are patterns, not fixed fields. | Components | Supported schemas, parameters, request bodies, responses, headers, security schemes, and examples | None | Links, callbacks, path items, media types, and `x-*` component entries | | Parameter | Identity, location, requiredness, schema, description, supported style, equivalent `explode` defaults, `allowReserved`, named query-map JSON `content`, and supported `x-*` | `deprecated`, unsupported examples | Incompatible serialization or content | | Request Body | `description`, `required`, supported content, and supported `x-*` | None | unresolved, cyclic, or wrong-kind `$ref` plans | -| Media Type | Supported schema, examples, and equivalent URL-encoded field encoding | OpenAPI 3.2 `description` | `itemSchema`, unsupported encoding fields, `prefixEncoding`, and `x-*` | +| Media Type | Distinct per-media schemas and examples, ranges, equivalent form field encoding, and multipart part content types | OpenAPI 3.2 `description` | `itemSchema`, unsupported encoding fields, `prefixEncoding`, and `x-*` | | Responses | Concrete status-code entries | None | `default` and `x-*` | | Response | `summary`, `description`, supported headers and content, and supported `x-*` | None | Unresolved, cyclic, or wrong-kind `$ref` plans and links | | Header | `description`, `required`, schema, `style: simple`, and `allowReserved` | `deprecated`, unsupported examples | Other serialization, content, unresolved/cyclic/wrong-kind `$ref`, and `x-*` | @@ -315,12 +323,13 @@ The following constructs are conditional: maps to a typed root map. A body-only optional form also emits `OptionalRequestBody()`. - A form map combined with transport parameters uses an object payload and a - selected `Body(...)` attribute. Multi-media and optional multipart maps use - the documented raw-body contract instead of inventing codec policy. -- A form schema containing an unconstrained value, including a free-form + selected `Body(...)` attribute. Optional single-media multipart maps use + the documented raw-body contract. Multi-media form/multipart variants require + structured object schemas. +- A single-media form schema containing an unconstrained value, including a free-form `additionalProperties: true` map, stays raw because typed form decoding cannot reconstruct JSON-valued data. -- An optional form object with required members also stays raw so an absent +- An optional single-media form object with required members also stays raw so an absent body does not spuriously fail nested required-field validation. Loom rejects boolean `false` schemas, constrained schemas without an explicit diff --git a/dsl/http_media_type.go b/dsl/http_media_type.go new file mode 100644 index 000000000..7c974fc66 --- /dev/null +++ b/dsl/http_media_type.go @@ -0,0 +1,26 @@ +package dsl + +import ( + "github.com/CaliLuke/loom/eval" + "github.com/CaliLuke/loom/expr" +) + +// MediaType binds an HTTP media type or range to a branch of the selected Body +// union. Use it inside an endpoint HTTP block or Response. Every union branch +// must have exactly one mapping. Requests and client responses select the most +// specific mapping from Content-Type; clients and services select outgoing +// representations through the ordinary union constructors. A range requires a +// mapped Content-Type header supplying a concrete outgoing type. MediaType does +// not perform Accept negotiation or change the union's JSON representation +// outside this HTTP body occurrence. +func MediaType(media, branch string) { + mapping := &expr.HTTPMediaTypeExpr{ContentType: media, Branch: branch} + switch current := eval.Current().(type) { + case *expr.HTTPEndpointExpr: + current.MediaTypes = append(current.MediaTypes, mapping) + case *expr.HTTPResponseExpr: + current.MediaTypes = append(current.MediaTypes, mapping) + default: + eval.IncompatibleDSL() + } +} diff --git a/dsl/http_media_type_test.go b/dsl/http_media_type_test.go new file mode 100644 index 000000000..3842a7704 --- /dev/null +++ b/dsl/http_media_type_test.go @@ -0,0 +1,210 @@ +package dsl_test + +import ( + "testing" + + "github.com/stretchr/testify/require" + + . "github.com/CaliLuke/loom/dsl" + "github.com/CaliLuke/loom/expr" +) + +func TestHTTPMediaTypeDSL(t *testing.T) { + root := expr.RunDSL(t, func() { + API("media", func() { + }) + Service("media", func() { + Method("exchange", func() { + Payload(func() { + OneOf("body", func() { + Attribute("json", Int) + Attribute("text", String) + }) + Required("body") + }) + Result(func() { + OneOf("body", func() { + Attribute("json", Int) + Attribute("text", String) + }) + Required("body") + }) + HTTP(func() { + POST("/") + Body("body") + MediaType("application/json", "json") + MediaType("text/plain", "text") + Response(StatusOK, func() { + Body("body") + MediaType("application/json", "json") + MediaType("text/plain", "text") + }) + }) + }) + }) + }) + endpoint := root.API.HTTP.Services[0].HTTPEndpoints[0] + require.Len(t, endpoint.MediaTypes, 2) + require.Equal(t, "json", endpoint.MediaTypes[0].Branch) + require.Len(t, endpoint.Responses[0].MediaTypes, 2) +} + +func TestHTTPMediaTypeAdmission(t *testing.T) { + for _, tc := range []struct { + name string + mapping func() + want string + }{ + {"unknown branch", func() { + MediaType("application/json", "missing") + MediaType("text/plain", "text") + }, "unknown body branch"}, + {"unmapped branch", func() { + MediaType("application/json", "json") + }, "no mapping"}, + {"duplicate branch", func() { + MediaType("application/json", "json") + MediaType("application/xml", "json") + MediaType("text/plain", "text") + }, "more than once"}, + {"overlapping parameters", func() { + MediaType("application/json; a=1", "json") + MediaType("application/json; b=2", "text") + }, "ambiguous media"}, + {"range without header", func() { + MediaType("application/*", "json") + MediaType("text/plain", "text") + }, "mapped Content-Type"}, + {"text number", func() { + MediaType("text/plain", "json") + MediaType("application/json", "text") + }, "String or Bytes"}, + {"static type", func() { + MediaType("application/json", "json") + MediaType("text/plain", "text") + ContentType("application/json") + }, "without ContentType"}, + {"form scalar", func() { + MediaType("application/x-www-form-urlencoded", "json") + MediaType("text/plain", "text") + }, "require non-nullable objects"}, + } { + t.Run(tc.name, func(t *testing.T) { + err := expr.RunInvalidDSL(t, func() { + Service("media", func() { + Method("exchange", func() { + Payload(func() { + OneOf("body", func() { + Attribute("json", Int) + Attribute("text", String) + }) + }) + HTTP(func() { + POST("/") + Body("body") + tc.mapping() + }) + }) + }) + }) + require.ErrorContains(t, err, tc.want) + }) + } +} + +func TestHTTPMediaContentTypeHeaderAdmission(t *testing.T) { + for _, response := range []string{"request", "response", "error"} { + for _, media := range []string{"application/json", "application/*"} { + for _, tc := range []struct { + name string + typ expr.DataType + nullable bool + }{{"integer", Int, false}, {"array", ArrayOf(String), false}, {"nullable", String, true}} { + t.Run(tc.name+media+response, func(t *testing.T) { + err := expr.RunInvalidDSL(t, func() { + value := Type("WithMedia", func() { + Attribute("content_type", tc.typ, func() { + if tc.nullable { + Nullable() + } + }) + OneOf("body", func() { + Attribute("json", Int) + }) + }) + Service("media", func() { + Method("exchange", func() { + Payload(value) + Result(value) + Error("bad", value) + mapping := func() { + Header("content_type:Content-Type") + Body("body") + MediaType(media, "json") + } + HTTP(func() { + POST("/") + switch response { + case "error": + Response("bad", StatusBadRequest, mapping) + case "response": + Response(StatusOK, mapping) + default: + mapping() + } + }) + }) + }) + }) + require.ErrorContains(t, err, "Content-Type") + }) + } + } + } +} + +func TestHTTPMediaParameterWildcardIsExact(t *testing.T) { + expr.RunDSL(t, func() { + Service("media", func() { + Method("exchange", func() { + Payload(func() { + OneOf("body", func() { + Attribute("json", Int) + }) + }) + HTTP(func() { + POST("/") + Body("body") + MediaType(`application/json; profile="*"`, "json") + }) + }) + }) + }) +} + +func TestHTTPMediaMultipartPartRequiresConcreteType(t *testing.T) { + for _, media := range []string{"image/*", "*/*", "image"} { + t.Run(media, func(t *testing.T) { + err := expr.RunInvalidDSL(t, func() { + Service("media", func() { + Method("exchange", func() { + Payload(func() { + OneOf("body", func() { + Attribute("file", func() { + Attribute("data", Bytes) + Meta("openapi:encoding:data:contentType", media) + }) + }) + }) + HTTP(func() { + POST("/") + Body("body") + MediaType("multipart/form-data", "file") + }) + }) + }) + }) + require.ErrorContains(t, err, "concrete") + }) + } +} diff --git a/expr/http_endpoint.go b/expr/http_endpoint.go index 049e2385b..5a33f5ee1 100644 --- a/expr/http_endpoint.go +++ b/expr/http_endpoint.go @@ -41,6 +41,8 @@ type ( Cookies *MappedAttributeExpr // Body describes the HTTP request body. Body *AttributeExpr + // MediaTypes maps request media contracts to branches of the selected body union. + MediaTypes []*HTTPMediaTypeExpr // RequestContentType is the authored typed JSON request media, or nil when // implicit. A nonnil empty string is an invalid explicit declaration. RequestContentType *string diff --git a/expr/http_endpoint_validate.go b/expr/http_endpoint_validate.go index 0da165bbf..26d38558e 100644 --- a/expr/http_endpoint_validate.go +++ b/expr/http_endpoint_validate.go @@ -24,12 +24,16 @@ func (e *HTTPEndpointExpr) Validate() error { e.validateResponses(verr) e.validateBodyAndPayload(verr) e.validateRequestContentType(verr) + e.validateMediaTypes(verr) e.validateRequestReconstruction(verr) e.validateNullableTransportLocations(verr) e.validateBodyElementNames(verr) e.validateAPIKeyLocations(verr) for _, er := range e.HTTPErrors { verr.Merge(er.Validate()) + if definition := e.MethodExpr.Error(er.Name); definition != nil { + er.Response.validateMediaTypes(e, definition.AttributeExpr, verr) + } } e.validateStatusErrorNames(verr) diff --git a/expr/http_media_encoding.go b/expr/http_media_encoding.go new file mode 100644 index 000000000..29f143728 --- /dev/null +++ b/expr/http_media_encoding.go @@ -0,0 +1,134 @@ +package expr + +import ( + "fmt" + "mime" + "strings" + + "github.com/CaliLuke/loom/internal/httpencoding" +) + +// HTTPMediaPartTypes returns the effective multipart Content-Type of each +// authored object field. It validates metadata against the generated part codec. +func HTTPMediaPartTypes(body *AttributeExpr) (map[string]string, error) { + object := AsObject(body.Type) + if object == nil || containsNullable(body) || containsUntaggedUnion(body) { + return nil, fmt.Errorf("multipart media requires a non-nullable object without untagged unions") + } + result := make(map[string]string, len(*object)) + for _, field := range *object { + name := ElementName(field.Name) + authored, _ := HTTPMediaMetadata(body).Last("openapi:encoding:" + name + ":contentType") + media, err := httpencoding.MultipartContentType(httpMediaFieldShape(field.Attribute), httpMediaBaseType(field.Attribute.Type) == Bytes, authored) + if err != nil { + return nil, fmt.Errorf("multipart field %q: %w", name, err) + } + result[name] = media + } + return result, nil +} + +func validateMediaFields(body *AttributeExpr, media string) error { + base, _, err := mime.ParseMediaType(media) + if err != nil { + return err + } + if base != "application/x-www-form-urlencoded" && base != "multipart/form-data" { + return nil + } + object := AsObject(body.Type) + if object == nil || containsNullable(body) || containsUntaggedUnion(body) { + return fmt.Errorf("form and multipart media require non-nullable objects without untagged unions") + } + meta := HTTPMediaMetadata(body) + if err := validateMediaEncodingKeys(meta, object); err != nil { + return err + } + for _, field := range *object { + prefix := "openapi:encoding:" + ElementName(field.Name) + ":" + value := httpencoding.Serialization{} + value.MediaType, _ = meta.Last(prefix + "contentType") + value.Style, _ = meta.Last(prefix + "style") + if raw, ok := meta.Last(prefix + "explode"); ok { + explode := raw != "false" + value.Explode = &explode + } + if raw, ok := meta.Last(prefix + "allowReserved"); ok { + reserved := raw != "false" + value.AllowReserved = &reserved + } + if base == "multipart/form-data" { + value.MediaType = "" // Part media is checked against the field codec below. + } + if err := httpencoding.Match(httpencoding.FormField, httpMediaFieldShape(field.Attribute), value); err != nil { + return fmt.Errorf("media field %q: %w", field.Name, err) + } + } + if base == "multipart/form-data" { + _, err := HTTPMediaPartTypes(body) + return err + } + return nil +} + +func httpMediaFieldShape(attribute *AttributeExpr) httpencoding.Shape { + switch { + case IsMap(attribute.Type): + return httpencoding.Map + case IsObject(attribute.Type), IsUnion(attribute.Type): + return httpencoding.Object + case IsArray(attribute.Type): + if element := AsArray(attribute.Type).ElemType; IsPrimitive(element.Type) && !IsAny(element.Type) { + return httpencoding.Sequence + } + return httpencoding.UnsupportedShape + default: + return httpencoding.Scalar + } +} + +func httpMediaBaseType(typ DataType) DataType { + for { + named, ok := typ.(UserType) + if !ok { + return typ + } + typ = named.Attribute().Type + } +} + +// HTTPMediaMetadata resolves occurrence-first metadata through body aliases. +// It returns a detached map so transport owners can retain the effective contract. +func HTTPMediaMetadata(attribute *AttributeExpr) MetaExpr { + result := MetaExpr{} + seen := make(map[*AttributeExpr]bool) + for attribute != nil && !seen[attribute] { + seen[attribute] = true + for name, values := range attribute.Meta { + if _, found := result[name]; !found { + result[name] = append([]string(nil), values...) + } + } + named, ok := attribute.Type.(UserType) + if !ok { + return result + } + attribute = named.Attribute() + } + return result +} + +func validateMediaEncodingKeys(meta MetaExpr, object *Object) error { + allowed := make(map[string]bool) + for _, field := range *object { + for _, suffix := range []string{"contentType", "style", "explode", "allowReserved"} { + allowed["openapi:encoding:"+ElementName(field.Name)+":"+suffix] = true + } + } + for key := range meta { + if strings.HasPrefix(key, "openapi:encoding:") && !allowed[key] || strings.HasPrefix(key, "openapi:itemEncoding:") || strings.HasPrefix(key, "openapi:prefixEncoding:") { + return fmt.Errorf("media encoding %q has no generated field contract", key) + } + } + return nil +} diff --git a/expr/http_media_type.go b/expr/http_media_type.go new file mode 100644 index 000000000..25821037a --- /dev/null +++ b/expr/http_media_type.go @@ -0,0 +1,135 @@ +package expr + +import ( + "mime" + "strings" + + "github.com/CaliLuke/loom/eval" + "github.com/CaliLuke/loom/internal/httpcodec" +) + +// HTTPMediaTypeExpr binds one HTTP media contract to a semantic body union branch. +// The endpoint or response owns the mapping; reusable service unions are unchanged. +type HTTPMediaTypeExpr struct { + // ContentType is the authored media type or range, including its parameters. + ContentType string + // Branch names a branch of the endpoint or response's selected Body union. + Branch string +} + +// HTTPMediaBranch resolves a mapping against its effective body union. It returns +// nil when the body is not a union or the branch does not exist. +func HTTPMediaBranch(body *AttributeExpr, name string) *NamedAttributeExpr { + if body == nil { + return nil + } + union := AsUnion(body.Type) + if union != nil { + for _, branch := range union.Values { + if AttributeName(branch.Name) == name { + return branch + } + } + } + return nil +} + +func (e *HTTPEndpointExpr) validateMediaTypes(verr *eval.ValidationErrors) { + if len(e.MediaTypes) == 0 { + return + } + _, rpc := e.MethodExpr.Meta["jsonrpc"] + if e.RequestContentType != nil || e.OpenAPIRequestBody != nil || e.FormRequest || + e.MultipartRequest || e.SkipRequestBodyEncodeDecode || e.MethodExpr.IsStreaming() || rpc { + verr.Add(e, "MediaType requires an ordinary typed HTTP body without ContentType, documentation-only body, form, multipart, raw or streaming framing") + } + validateHTTPMediaTypes(e, httpRequestBody(e), e.MethodExpr.Payload, e.MediaTypes, e.Headers, verr) +} + +func (r *HTTPResponseExpr) validateMediaTypes(e *HTTPEndpointExpr, source *AttributeExpr, verr *eval.ValidationErrors) { + if len(r.MediaTypes) == 0 { + return + } + _, rpc := e.MethodExpr.Meta["jsonrpc"] + if r.ContentType != "" || r.OpenAPIBody != nil || e.SkipResponseBodyEncodeDecode || + e.FileResponse || e.MethodExpr.IsStreaming() || rpc { + verr.Add(r, "MediaType requires an ordinary typed HTTP body without ContentType, documentation-only body, raw or streaming framing") + } + validateHTTPMediaTypes(r, buildHTTPResponseBody(e.Name(), source, r, e.Service), source, r.MediaTypes, r.Headers, verr) +} + +func validateHTTPMediaTypes(parent eval.Expression, body, source *AttributeExpr, mappings []*HTTPMediaTypeExpr, headers *MappedAttributeExpr, verr *eval.ValidationErrors) { + union := AsUnion(body.Type) + if union == nil || IsNullable(body) { + verr.Add(parent, "MediaType requires a non-nullable union Body") + return + } + header := mediaContentTypeHeader(headers, source) + if header != nil && (httpMediaBaseType(header.Type) != String || AllowsNull(header)) { + verr.Add(parent, "MediaType mapped Content-Type header requires a non-nullable String") + } + media := make([]string, len(mappings)) + branches := make(map[string]bool, len(mappings)) + for i, mapping := range mappings { + media[i] = mapping.ContentType + branch := HTTPMediaBranch(body, mapping.Branch) + if branch == nil { + verr.Add(parent, "MediaType references unknown body branch %q", mapping.Branch) + continue + } + if branches[branch.Name] { + verr.Add(parent, "MediaType maps body branch %q more than once", mapping.Branch) + } + branches[branch.Name] = true + base, _, mediaErr := mime.ParseMediaType(mapping.ContentType) + if mediaErr == nil && base == "application/octet-stream" && + (httpMediaBaseType(branch.Attribute.Type) != Bytes || AllowsNull(branch.Attribute)) { + verr.Add(parent, "MediaType %q requires a non-nullable Bytes branch", mapping.ContentType) + } + if err := validateMediaFields(branch.Attribute, mapping.ContentType); err != nil { + verr.Add(parent, "%s", err) + } + if strings.Contains(base, "*") && header == nil { + verr.Add(parent, "MediaType range %q requires a mapped Content-Type header for concrete publication", mapping.ContentType) + } + if httpcodec.RequestContentType(mapping.ContentType).Kind == httpcodec.Text && + (!httpMediaTextBranch(branch.Attribute.Type) || AllowsNull(branch.Attribute)) { + verr.Add(parent, "MediaType %q requires a non-nullable String or Bytes branch", mapping.ContentType) + } + } + for _, branch := range union.Values { + if !branches[branch.Name] { + verr.Add(parent, "MediaType has no mapping for body branch %q", branch.Name) + } + } + if err := httpcodec.ValidateMediaContracts(media); err != nil { + verr.Add(parent, "%s", err) + } +} + +func httpMediaTextBranch(typ DataType) bool { + for { + named, ok := typ.(UserType) + if !ok { + return typ == String || typ == Bytes + } + typ = named.Attribute().Type + } +} + +func mediaContentTypeHeader(headers *MappedAttributeExpr, source *AttributeExpr) *AttributeExpr { + if headers == nil || headers.IsEmpty() { + return nil + } + for _, field := range *AsObject(headers.Type) { + if strings.EqualFold(headers.ElemName(field.Name), "Content-Type") { + if source != nil && IsObject(source.Type) { + if _, attribute := source.FindAttribute(field.Name); attribute != nil { + return attribute + } + } + return field.Attribute + } + } + return nil +} diff --git a/expr/http_media_type_test.go b/expr/http_media_type_test.go new file mode 100644 index 000000000..58a38d5ad --- /dev/null +++ b/expr/http_media_type_test.go @@ -0,0 +1,16 @@ +package expr + +import ( + "testing" + + "github.com/stretchr/testify/require" +) + +func TestHTTPResponseMediaTypesDup(t *testing.T) { + source := &HTTPResponseExpr{MediaTypes: []*HTTPMediaTypeExpr{{ContentType: "application/json", Branch: "document"}}} + copy := source.Dup() + require.Equal(t, source.MediaTypes, copy.MediaTypes) + require.NotSame(t, source.MediaTypes[0], copy.MediaTypes[0]) + copy.MediaTypes[0].Branch = "other" + require.Equal(t, "document", source.MediaTypes[0].Branch) +} diff --git a/expr/http_response.go b/expr/http_response.go index 1e9c260a8..bfc37d0dd 100644 --- a/expr/http_response.go +++ b/expr/http_response.go @@ -98,6 +98,8 @@ type ( Cookies []*HTTPResponseCookieExpr // Response body if any Body *AttributeExpr + // MediaTypes maps response media contracts to branches of the selected body union. + MediaTypes []*HTTPMediaTypeExpr // OpenAPIBody describes a documentation-only response body used by // OpenAPI generation when runtime response encoding/decoding is skipped. OpenAPIBody *AttributeExpr @@ -158,6 +160,7 @@ func (r *HTTPResponseExpr) Validate(e *HTTPEndpointExpr) *eval.ValidationErrors r.validateHeadersAndCookies(e, resultType, verr) r.validateBodyAndLinks(e, resultType, verr) r.validateResultReconstruction(e, verr) + r.validateMediaTypes(e, e.MethodExpr.Result, verr) return verr } @@ -525,6 +528,15 @@ func (r *HTTPResponseExpr) Dup() *HTTPResponseExpr { StatusErrorName: r.StatusErrorName, Meta: r.Meta, } + if len(r.MediaTypes) > 0 { + res.MediaTypes = make([]*HTTPMediaTypeExpr, len(r.MediaTypes)) + for i, mapping := range r.MediaTypes { + if mapping != nil { + copy := *mapping + res.MediaTypes[i] = © + } + } + } if r.Body != nil { res.Body = DupAtt(r.Body) } diff --git a/expr/value_plan.go b/expr/value_plan.go index 1a6e1b605..9f4590554 100644 --- a/expr/value_plan.go +++ b/expr/value_plan.go @@ -27,8 +27,9 @@ type ( // It preserves that snapshot even if the original attribute later changes. // Exactly one of Target and TargetOccurrence must be supplied. TargetOccurrence ValueOccurrence - // Selection is an authored member path from the semantic root, for a - // selected body or location. An empty path selects the root itself. + // Selection is an authored member or union-branch path from the semantic + // root. A branch projects only values selecting that branch; other union + // values are absent. An empty path selects the root itself. Selection []string // Codec identifies the actual encoding owner, not a documentation MIME. Codec ValueCodec diff --git a/expr/value_plan_branch_selection_test.go b/expr/value_plan_branch_selection_test.go new file mode 100644 index 000000000..b34f2332e --- /dev/null +++ b/expr/value_plan_branch_selection_test.go @@ -0,0 +1,44 @@ +package expr + +import ( + "testing" + + "github.com/stretchr/testify/require" +) + +func TestValuePlanSelectsUnionBranch(t *testing.T) { + left := &AttributeExpr{Type: String} + right := &AttributeExpr{Type: Int} + source := &AttributeExpr{Type: &Object{{Name: "body", Attribute: &AttributeExpr{Type: &Union{TypeName: "Media", Values: []*NamedAttributeExpr{ + {Name: "text", Attribute: left}, {Name: "number", Attribute: right}, + }}}}}} + context := NewValueContext() + occurrence, err := context.NewOccurrence(source) + require.NoError(t, err) + plan, err := context.NewValuePlan(occurrence, ValuePlanRequest{Target: DupAtt(left), Selection: []string{"body", "text"}, Codec: ValueCodecJSON, Use: ValuePlanDocumentation}) + require.NoError(t, err) + for _, tc := range []struct { + name string + body any + emitted bool + }{ + {"selected", "hello", true}, + {"other", 42, false}, + } { + t.Run(tc.name, func(t *testing.T) { + result := context.Resolve(occurrence, context.SupplyValue(ValueInput{Raw: map[string]any{"body": tc.body}}), ValueRoleExample) + require.Equal(t, ValueResolved, result.Outcome()) + wire, ok := context.ProjectJSON(result, plan).JSON() + require.Equal(t, tc.emitted, ok) + if ok { + require.JSONEq(t, `"hello"`, string(wire)) + } + }) + } + branch := occurrence.Members()[0].Occurrence.Branches()[0].Occurrence + local, err := plan.ForOccurrence(branch, plan.Root()) + require.NoError(t, err) + require.Empty(t, local.selection) + _, err = context.NewValuePlan(occurrence, ValuePlanRequest{Target: DupAtt(left), Selection: []string{"body", "missing"}, Codec: ValueCodecJSON, Use: ValuePlanDocumentation}) + require.Error(t, err) +} diff --git a/expr/value_plan_build.go b/expr/value_plan_build.go index 2f5e8c0f2..428c7a6f3 100644 --- a/expr/value_plan_build.go +++ b/expr/value_plan_build.go @@ -94,23 +94,30 @@ func valuePlanSelection(source *valueOccurrenceNode, names []string) (*valueOccu selection := make([]uint64, 0, len(names)) for _, name := range names { selected = valueUnderlyingOccurrence(selected) - var match *valueOccurrenceMember - for index := range selected.declaration.members { - member := &selected.declaration.members[index] - if member.name == name { - match = member - break - } - } - if match == nil { - return nil, nil, fmt.Errorf("value plan selection has no authored member %q", name) + child, identity := valuePlanSelectedChild(selected, name) + if child == nil { + return nil, nil, fmt.Errorf("value plan selection has no authored member or branch %q", name) } - selection = append(selection, match.id) - selected = match.node + selection = append(selection, identity) + selected = child } return selected, selection, nil } +func valuePlanSelectedChild(parent *valueOccurrenceNode, name string) (*valueOccurrenceNode, uint64) { + for _, member := range parent.declaration.members { + if member.name == name { + return member.node, member.id + } + } + for _, branch := range parent.declaration.branches { + if branch.name == name { + return branch.node, branch.id + } + } + return nil, 0 +} + func (b *valuePlanBuilder) validatePolicies() error { for index, used := range b.used { if !used { diff --git a/expr/value_projection.go b/expr/value_projection.go index 4f03cc1d3..09b8cb458 100644 --- a/expr/value_projection.go +++ b/expr/value_projection.go @@ -259,6 +259,9 @@ func projectionBuildFailure(outcome ProjectionOutcome, format string, args ...an func projectionMember(value ResolvedValue, identity uint64) ResolvedValue { if value.node != nil { + if value.node.kind == ValueKindUnion && value.node.branch.index == identity { + return value.node.payload + } for _, field := range value.node.fields { if field.Member.index == identity { return field.Value diff --git a/http/codegen/internal/representation/example_preparation.go b/http/codegen/internal/representation/example_preparation.go index a1a06add0..4144cfc97 100644 --- a/http/codegen/internal/representation/example_preparation.go +++ b/http/codegen/internal/representation/example_preparation.go @@ -143,7 +143,7 @@ func PrepareTargetExamples( if !target.Plan.Root().Valid() || !target.ExamplePlan.Root().Valid() || !target.AnchorPlan.Root().Valid() { return fmt.Errorf("HTTP example target has no captured schema and example plans") } - anchor, err := selectedValueOccurrence(target.Anchor.Occurrence, target.Selection) + anchor, branchSelection, err := selectedValueOccurrence(target.Anchor.Occurrence, target.Selection) if err != nil { return err } @@ -157,7 +157,7 @@ func PrepareTargetExamples( position := targetExamplePosition{ schema: target.Plan.Root(), examplePlan: target.ExamplePlan.Root(), anchorPlan: target.AnchorPlan.Root(), exampleProjection: target.ExamplePlan, anchorProjection: anchorProjection, - example: target.ExampleOccurrence, anchor: anchor, + example: target.ExampleOccurrence, anchor: anchor, branchSelection: branchSelection, } target.ExampleSets = make(map[expr.ValuePlanNode]*transportir.ValueExampleSet) target.Annotations = make(map[expr.ValuePlanNode]expr.JSONSchemaAnnotations) @@ -182,6 +182,7 @@ type targetExamplePosition struct { anchorProjection expr.ValuePlan example expr.ValueOccurrence anchor expr.ValueOccurrence + branchSelection bool } func prepareTargetExamplePosition( @@ -250,6 +251,9 @@ func prepareTargetExampleSet( Reachable: true, SuppressGenerated: suppressGenerated, }) entries := selection.Entries() + if len(entries) == 0 && position.schema == target.Plan.Root() && position.branchSelection { + entries = target.Anchor.Context.SelectExample(target.Anchor.Occurrence, expr.ExamplePolicy{Reachable: true, SuppressGenerated: suppressGenerated}).Entries() + } if len(entries) == 0 { data := target.Source plan := position.exampleProjection @@ -526,22 +530,29 @@ func appendExamplePath(path []string, parts ...string) []string { return append(append([]string(nil), path...), parts...) } -func selectedValueOccurrence(root expr.ValueOccurrence, selection []string) (expr.ValueOccurrence, error) { +func selectedValueOccurrence(root expr.ValueOccurrence, selection []string) (expr.ValueOccurrence, bool, error) { current := root + branchSelection := false for _, name := range selection { var found expr.ValueOccurrence for _, member := range current.Members() { if member.Name == name || expr.AttributeName(member.Name) == expr.AttributeName(name) { if found.ID() != (expr.ValueIdentity{}) { - return expr.ValueOccurrence{}, fmt.Errorf("HTTP example selection %q is ambiguous", name) + return expr.ValueOccurrence{}, false, fmt.Errorf("HTTP example selection %q is ambiguous", name) } found = member.Occurrence } } + for _, branch := range current.Branches() { + if branch.Name == name { + branchSelection = true + found = branch.Occurrence + } + } if found.ID() == (expr.ValueIdentity{}) { - return expr.ValueOccurrence{}, fmt.Errorf("HTTP example selection has no member %q", name) + return expr.ValueOccurrence{}, false, fmt.Errorf("HTTP example selection has no member %q", name) } current = found } - return current, nil + return current, branchSelection, nil } diff --git a/http/codegen/internal/representation/example_preparation_endpoint.go b/http/codegen/internal/representation/example_preparation_endpoint.go index 40fcaaa86..6d939c0b8 100644 --- a/http/codegen/internal/representation/example_preparation_endpoint.go +++ b/http/codegen/internal/representation/example_preparation_endpoint.go @@ -42,6 +42,11 @@ func prepareEndpointRequestExamples( security *transportir.Security, prepare endpointExamplePreparer, ) error { + for _, variant := range request.MediaVariants { + if err := prepare(variant.Value, variant.Body, "request-media", variant.ContentType); err != nil { + return err + } + } for _, input := range []struct { target *transportir.ValueTarget attribute *expr.AttributeExpr @@ -84,6 +89,11 @@ func prepareEndpointRequestExamples( func prepareEndpointResponseExamples(response *transportir.Response, prepare endpointExamplePreparer) error { for _, statusResponse := range append(append([]*transportir.ResponseStatus(nil), response.Responses...), response.ErrorResponses...) { status := fmt.Sprint(statusResponse.StatusCode) + for _, variant := range statusResponse.MediaVariants { + if err := prepare(variant.Value, variant.Body, "response-media-variant", status, variant.ContentType); err != nil { + return err + } + } if err := prepare(statusResponse.BodyValue, statusResponse.Body, "response-body", status); err != nil { return err } diff --git a/http/codegen/internal/representation/layout.go b/http/codegen/internal/representation/layout.go index 37bc01f8e..bb37b9f27 100644 --- a/http/codegen/internal/representation/layout.go +++ b/http/codegen/internal/representation/layout.go @@ -65,6 +65,9 @@ func (l Layout) Bind(context *codegen.AttributeContext) { // Record preserves the highest-priority physical layout for a declaration. func (l Layout) Record(userType expr.UserType, presence, pointer, defaults bool) { + if _, form := userType.Attribute().Meta["http:media:form-root"]; form { + presence = false + } key := userType.Hash() if old, found := l.JSONPresence[key]; found && layoutPriority(old, l.Pointer[key]) >= layoutPriority(presence, pointer) { return diff --git a/http/codegen/internal/representation/media_variants.go b/http/codegen/internal/representation/media_variants.go new file mode 100644 index 000000000..b456f489d --- /dev/null +++ b/http/codegen/internal/representation/media_variants.go @@ -0,0 +1,56 @@ +package representation + +import ( + "mime" + + "github.com/CaliLuke/loom/codegen" + "github.com/CaliLuke/loom/expr" + "github.com/CaliLuke/loom/http/codegen/internal/transportir" + "github.com/CaliLuke/loom/internal/httpcodec" +) + +func attachMediaTargets(variants []*transportir.MediaVariant, body *transportir.ValueTarget) { + if body == nil || body.Source == nil { + return + } + for _, variant := range variants { + occurrence, err := body.Source.Context.NewOccurrence(variant.Body) + variant.Value = &transportir.ValueTarget{ + Source: body.Source, Anchor: body.Source, + Selection: append(append([]string(nil), body.Selection...), variant.Branch), + ExampleOccurrence: occurrence, Codec: mediaVariantCodec(variant.ContentType), Error: err, + } + if variant.Value.Codec == expr.ValueCodecCustom { + variant.Value.Boundary = "media variant codec is selected from the concrete Content-Type or supplied factory" + } + } +} + +func mediaVariantCodec(media string) expr.ValueCodec { + base, _, err := mime.ParseMediaType(media) + if err != nil { + return expr.ValueCodecCustom + } + switch base { + case "application/x-www-form-urlencoded": + return expr.ValueCodecForm + case "multipart/form-data": + return expr.ValueCodecMultipart + case "application/octet-stream": + return expr.ValueCodecRaw + } + switch httpcodec.RequestContentType(media).Kind { + case httpcodec.JSON: + return expr.ValueCodecJSON + case httpcodec.Text: + return expr.ValueCodecText + default: + return expr.ValueCodecCustom + } +} + +func prepareMediaTargets(variants []*transportir.MediaVariant, context *codegen.AttributeContext) { + for _, variant := range variants { + variant.Value = BuildValuePlan(variant.Value, variant.Body, context, expr.ValuePlanSchema) + } +} diff --git a/http/codegen/internal/representation/prepare.go b/http/codegen/internal/representation/prepare.go index ce9088365..c0acdb4df 100644 --- a/http/codegen/internal/representation/prepare.go +++ b/http/codegen/internal/representation/prepare.go @@ -98,6 +98,7 @@ func prepareSchemaTargets(endpoint *transportir.Endpoint, layout Layout) { requestContext.CollectionElementPresence = requestContext.JSONPresence layout.Bind(requestContext) request := endpoint.Request + prepareMediaTargets(request.MediaVariants, requestContext) request.BodyValue = BuildValuePlan(request.BodyValue, request.Body, requestContext, expr.ValuePlanSchema) request.StreamingValue = BuildValuePlan(request.StreamingValue, request.StreamingBody, requestContext, expr.ValuePlanSchema) request.DocumentValue = BuildValuePlan(request.DocumentValue, request.DocumentBody, requestContext, expr.ValuePlanSchema) @@ -112,6 +113,7 @@ func prepareSchemaTargets(endpoint *transportir.Endpoint, layout Layout) { responseContext := schemaContext(false) layout.Bind(responseContext) for _, response := range serviceResponses(endpoint) { + prepareMediaTargets(response.MediaVariants, responseContext) response.BodyValue = BuildValuePlan(response.BodyValue, response.Body, responseContext, expr.ValuePlanSchema) for media, value := range response.BodyValues { response.BodyValues[media] = BuildValuePlan(value, response.Body, responseContext, expr.ValuePlanSchema) diff --git a/http/codegen/internal/representation/response_codec.go b/http/codegen/internal/representation/response_codec.go index 52e5cf42e..003bddad4 100644 --- a/http/codegen/internal/representation/response_codec.go +++ b/http/codegen/internal/representation/response_codec.go @@ -12,6 +12,7 @@ import ( func attachResponseTargets(endpoint *transportir.Endpoint, status *transportir.ResponseStatus, source *service.ValueData, schemaOnly bool) { codec, boundary := responseCodec(endpoint, status) status.BodyValue = ValueTarget(source, status.Body, status.BodyOrigin, codec, schemaOnly) + attachMediaTargets(status.MediaVariants, status.BodyValue) if status.BodyValue != nil && boundary != "" { status.BodyValue.Boundary = boundary } diff --git a/http/codegen/internal/representation/value_carriers.go b/http/codegen/internal/representation/value_carriers.go index 92ad77985..f6d0e4afa 100644 --- a/http/codegen/internal/representation/value_carriers.go +++ b/http/codegen/internal/representation/value_carriers.go @@ -23,6 +23,7 @@ func attachTargets(endpoint *transportir.Endpoint, method *service.MethodData, e codec = expr.ValueCodecForm } request.BodyValue = ValueTarget(method.PayloadValue, request.Body, request.BodyOriginKey, codec, schemaOnly) + attachMediaTargets(request.MediaVariants, request.BodyValue) request.StreamingValue = ValueTarget(method.StreamingPayloadValue, request.StreamingBody, "", expr.ValueCodecJSON, schemaOnly) for _, group := range [][]*transportir.Parameter{request.PathParams, request.QueryParams, request.Headers, request.Cookies} { for _, parameter := range group { diff --git a/http/codegen/internal/transportir/media_variant.go b/http/codegen/internal/transportir/media_variant.go new file mode 100644 index 000000000..8ba7917ab --- /dev/null +++ b/http/codegen/internal/transportir/media_variant.go @@ -0,0 +1,86 @@ +package transportir + +import ( + "mime" + + "github.com/CaliLuke/loom/expr" + "github.com/CaliLuke/loom/internal/httpencoding" +) + +// MediaVariant owns one HTTP representation of a semantic body union branch. +// Codec, examples and schema plans belong to this occurrence, not to other media +// that happen to use an equal schema. +type MediaVariant struct { + // ContentType is the authored media type or range. + ContentType string + // Branch is the semantic union branch name. + Branch string + // Tag is the generated union discriminator for this branch. + Tag string + // Body retains the branch's effective transport attribute. + Body *expr.AttributeExpr + // Value retains service provenance and the selected branch's representation plan. + Value *ValueTarget + // FormFieldModes records the field encoding policy for this media alone. + FormFieldModes map[string]httpencoding.Mode + // PartTypes binds multipart fields to their effective wire Content-Type. + PartTypes map[string]string + // Metadata retains occurrence-first media annotations through aliases. + Metadata expr.MetaExpr +} + +func buildMediaVariants(body *expr.AttributeExpr, mappings []*expr.HTTPMediaTypeExpr) []*MediaVariant { + variants := make([]*MediaVariant, 0, len(mappings)) + for _, mapping := range mappings { + branch := expr.HTTPMediaBranch(body, mapping.Branch) + if branch == nil { + continue // Expression validation owns malformed mappings. + } + base, _, err := mime.ParseMediaType(mapping.ContentType) + if err != nil { + continue // Expression validation owns malformed media types. + } + if base == "application/x-www-form-urlencoded" || base == "multipart/form-data" { + branch.Attribute = mediaFormCarrier(branch.Attribute) + } + var parts map[string]string + if base == "multipart/form-data" { + parts, err = expr.HTTPMediaPartTypes(branch.Attribute) + if err != nil { + panic(err) // Expression validation establishes this contract. + } + } + variants = append(variants, &MediaVariant{ + ContentType: mapping.ContentType, + PartTypes: parts, + Metadata: expr.HTTPMediaMetadata(branch.Attribute), + Branch: branch.Name, + Tag: expr.UnionVariantTag(branch), + Body: branch.Attribute, + FormFieldModes: formFieldModes(branch.Attribute, base == "application/x-www-form-urlencoded"), + }) + } + return variants +} + +// mediaFormCarrier gives the form root an independent physical identity. Its +// nested object fields retain JSON layouts because the form codec serializes +// those fields as JSON. Only the root and its aliases use native form presence. +func mediaFormCarrier(attribute *expr.AttributeExpr) *expr.AttributeExpr { + attribute = expr.DupAtt(attribute) + current := attribute + seen := make(map[expr.UserType]bool) + for { + named, ok := current.Type.(expr.UserType) + if !ok || seen[named] { + return attribute + } + seen[named] = true + named.Rename(named.Name() + "MediaForm") + current = named.Attribute() + if current.Meta == nil { + current.Meta = expr.MetaExpr{} + } + current.Meta["http:media:form-root"] = []string{"true"} + } +} diff --git a/http/codegen/internal/transportir/model.go b/http/codegen/internal/transportir/model.go index d06bbc84a..ac915e64a 100644 --- a/http/codegen/internal/transportir/model.go +++ b/http/codegen/internal/transportir/model.go @@ -110,6 +110,8 @@ type ( } Request struct { + // MediaVariants retains each typed request representation independently. + MediaVariants []*MediaVariant // ContentType is the admitted authored typed request media, empty when implicit. ContentType string // JSONRPCID retains the canonical envelope binding. @@ -170,6 +172,8 @@ type ( } ResponseStatus struct { + // MediaVariants retains each typed response representation independently. + MediaVariants []*MediaVariant // BodyValue carries the effective result or error occurrence. BodyValue *ValueTarget // BodyValues binds each advertised media label to the actual server encoder diff --git a/http/codegen/internal/transportir/request_response.go b/http/codegen/internal/transportir/request_response.go index 98aff747c..9b3da98b3 100644 --- a/http/codegen/internal/transportir/request_response.go +++ b/http/codegen/internal/transportir/request_response.go @@ -33,6 +33,7 @@ func buildRequest(endpoint *expr.HTTPEndpointExpr) *Request { contentType = *endpoint.RequestContentType } return &Request{ + MediaVariants: buildMediaVariants(body, endpoint.MediaTypes), ContentType: contentType, JSONRPCID: endpoint.JSONRPCID, Payload: payload, @@ -99,8 +100,10 @@ func buildResponseStatus(status *expr.HTTPResponseExpr, httpErrorExpr *expr.HTTP return nil } documentBody := responseDocumentBody(status) + body := normalizeHTTPAttribute(status.Body) httpError := buildError(httpErrorExpr) return &ResponseStatus{ + MediaVariants: buildMediaVariants(body, status.MediaTypes), Error: httpError, ErrorSelector: httpErrorExpr.JSONRPCErrorSelector(), StatusErrorName: status.StatusErrorName, @@ -110,7 +113,7 @@ func buildResponseStatus(status *expr.HTTPResponseExpr, httpErrorExpr *expr.HTTP ContentTypes: responseContentTypes(status), Headers: buildHeaders(status.Headers), Cookies: buildCookies(status.Cookies), - Body: normalizeHTTPAttribute(status.Body), + Body: body, DocumentBody: documentBody, IndependentDocumentBody: status.OpenAPIBody != nil, BodyOrigin: attributeOrigin(status.Body), diff --git a/http/codegen/media_contract_integration_test.go b/http/codegen/media_contract_integration_test.go new file mode 100644 index 000000000..37e10236f --- /dev/null +++ b/http/codegen/media_contract_integration_test.go @@ -0,0 +1,24 @@ +package codegen + +import ( + _ "embed" + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/require" + + "github.com/CaliLuke/loom/http/codegen/testdata" +) + +//go:embed testdata/media_contract_harness.go.txt +var mediaContractHarness string + +func TestGeneratedMediaContracts(t *testing.T) { + root := RunHTTPDSL(t, testdata.MultipleMediaContractsDSL) + directory := t.TempDir() + renderHTTPModule(t, directory, "example.com/mediacontracts", root) + require.NoError(t, os.WriteFile(filepath.Join(directory, "media_contract_test.go"), []byte(mediaContractHarness), 0o600)) + runGoCommand(t, directory, "mod", "tidy") + runGoCommand(t, directory, "test", "-race", "-count=1", ".") +} diff --git a/http/codegen/openapi/internal/ir/analyzer.go b/http/codegen/openapi/internal/ir/analyzer.go index 62a6faa8d..269d9f42e 100644 --- a/http/codegen/openapi/internal/ir/analyzer.go +++ b/http/codegen/openapi/internal/ir/analyzer.go @@ -28,8 +28,8 @@ type ( discriminatorReferences []discriminatorReference declarationRoots map[string]*Schema asyncAcquisition *asyncBaselineAcquisition - componentAnnotations map[componentAnnotationIdentity]map[componentAnnotationPosition]any - annotationOwner map[componentAnnotationPosition]any + componentAnnotations map[componentAnnotationIdentity]map[componentAnnotationPosition]componentExamples + annotationOwner map[componentAnnotationPosition]componentExamples occurrences []schemaOccurrenceAnalysis registrations []*Schema plan expr.ValuePlanNode @@ -479,53 +479,6 @@ func (a *Analyzer) applySchemaAnnotations(s *Schema, attr *expr.AttributeExpr, n } } -func (a *Analyzer) applySchemaExample(s *Schema, attr *expr.AttributeExpr, context string) { - if a.annotationOwner != nil { - position := componentAnnotationPosition{context, fingerprintAttribute(attr, a.closeObjects)} - if example, found := a.annotationOwner[position]; found { - s.Example = copyComponentAnnotation(example) - return - } - defer func() { - a.annotationOwner[position] = copyComponentAnnotation(s.Example) - }() - } - suppress := false - if a.suppressExamples != nil { - suppress = a.suppressExamples(attr, a.closeObjects) - } - if !suppress { - generator := exampleGeneratorForAttribute(a.rand, attr, a.closeObjects, context) - if a.customExampleValue { - raw := attr.Example(generator) - if example, ok := a.exampleValue(attr, raw); ok { - s.Example = example - } - return - } - if a.preparedExamples == nil { - return - } - set, prepared := a.preparedExamples.ExampleSets[a.examplePlan] - if !prepared || set == nil || !set.Prepared { - return - } - if set.Representative != nil { - var example any - var ok bool - if a.declaredExampleValue { - example, ok = preparedDeclaredExampleValue(*set.Representative) - } else { - example, ok = preparedOpenAPIExampleValue(*set.Representative) - } - if ok { - s.Example = example - } - } - return - } -} - func applyNullableSchema(schema *Schema) { if schema == nil || schema.Type == string(openapi.Null) { return @@ -535,6 +488,7 @@ func applyNullableSchema(schema *Schema) { base.Description = "" base.DefaultValue = nil base.Example = nil + base.Examples = nil base.ReadOnly = false base.WriteOnly = false base.Deprecated = false @@ -545,6 +499,7 @@ func applyNullableSchema(schema *Schema) { Description: schema.Description, DefaultValue: schema.DefaultValue, Example: schema.Example, + Examples: schema.Examples, ReadOnly: schema.ReadOnly, WriteOnly: schema.WriteOnly, Deprecated: schema.Deprecated, diff --git a/http/codegen/openapi/internal/ir/analyzer_body_types.go b/http/codegen/openapi/internal/ir/analyzer_body_types.go index 68d3e45dd..58b5f4149 100644 --- a/http/codegen/openapi/internal/ir/analyzer_body_types.go +++ b/http/codegen/openapi/internal/ir/analyzer_body_types.go @@ -121,6 +121,9 @@ func analyzeEndpointBodies(a *Analyzer, endpoint *transportir.Endpoint) *Endpoin ResponseBodies: responseBodies, } bodies.requestMedia = analyzeDocumentMedia(a, endpoint.Request.DocumentBody, endpoint.Request.DocumentValues, schemaUsageRequest, "request-schema") + if len(endpoint.Request.MediaVariants) > 0 { + bodies.requestMedia = analyzeMediaVariants(a, endpoint.Request.MediaVariants, schemaUsageRequest) + } if len(endpoint.Request.DocumentContentTypes) > 0 && bodies.requestMedia != nil { bodies.RequestBody = bodies.requestMedia[endpoint.Request.DocumentContentTypes[0]] } @@ -128,6 +131,9 @@ func analyzeEndpointBodies(a *Analyzer, endpoint *transportir.Endpoint) *Endpoin for _, responses := range [][]*transportir.ResponseStatus{endpoint.Response.Responses, endpoint.Response.ErrorResponses} { for _, response := range responses { bodies.responseMedia[response] = analyzeDocumentMedia(a, response.DocumentBody, response.DocumentValues, schemaUsageResponse, "response-schema", strconv.Itoa(response.StatusCode)) + if len(response.MediaVariants) > 0 { + bodies.responseMedia[response] = analyzeMediaVariants(a, response.MediaVariants, schemaUsageResponse) + } } } indexes := make(map[int]int) @@ -144,6 +150,9 @@ func analyzeEndpointBodies(a *Analyzer, endpoint *transportir.Endpoint) *Endpoin } func analyzeRequestBody(a *Analyzer, endpoint *transportir.Endpoint) *Schema { + if len(endpoint.Request.MediaVariants) > 0 { + return nil + } body := expr.UnwrapInlineHTTPBody(endpoint.Request.Body) if endpoint.Request.DocumentBody != nil { body = endpoint.Request.DocumentBody @@ -171,7 +180,7 @@ func analyzeResponseBodies(a *Analyzer, endpoint *transportir.Endpoint) map[int] responseBodies := make(map[int][]*Schema) appendBodies := func(responses []*transportir.ResponseStatus, projectSSE bool) { for _, resp := range responses { - if len(resp.DocumentValues) > 0 { + if len(resp.DocumentValues) > 0 || len(resp.MediaVariants) > 0 { responseBodies[resp.StatusCode] = append(responseBodies[resp.StatusCode], nil) continue } diff --git a/http/codegen/openapi/internal/ir/async_schema_examples.go b/http/codegen/openapi/internal/ir/async_schema_examples.go index de79a8482..20a111c85 100644 --- a/http/codegen/openapi/internal/ir/async_schema_examples.go +++ b/http/codegen/openapi/internal/ir/async_schema_examples.go @@ -85,6 +85,7 @@ func applyPreparedTargetExamples( } } schema.Example = nil + schema.Examples = nil restore := a.schemaPlanScope(a.plan, plan) a.applySchemaExample(schema, attr, context) restore() @@ -138,6 +139,7 @@ func applyAsyncExamples(a *Analyzer, schema *Schema, attr *expr.AttributeExpr, c } } schema.Example = nil + schema.Examples = nil a.applySchemaExample(schema, attr, context) } diff --git a/http/codegen/openapi/internal/ir/async_schema_inline.go b/http/codegen/openapi/internal/ir/async_schema_inline.go index 1baacf32c..3a2b34079 100644 --- a/http/codegen/openapi/internal/ir/async_schema_inline.go +++ b/http/codegen/openapi/internal/ir/async_schema_inline.go @@ -186,6 +186,7 @@ func asyncAnnotationOnly(schema *Schema) bool { copy := *schema copy.Title, copy.Description = "", "" copy.DefaultValue, copy.Example = nil, nil + copy.Examples = nil copy.ReadOnly, copy.WriteOnly, copy.Deprecated = false, false, false return onlySchemaReference(©) } @@ -200,8 +201,9 @@ func overlayAsyncAnnotations(base, local *Schema) *Schema { if local.DefaultValue != nil { base.DefaultValue = local.DefaultValue } - if local.Example != nil { + if local.Example != nil || len(local.Examples) > 0 { base.Example = local.Example + base.Examples = local.Examples } base.ReadOnly = base.ReadOnly || local.ReadOnly base.WriteOnly = base.WriteOnly || local.WriteOnly diff --git a/http/codegen/openapi/internal/ir/component_annotations.go b/http/codegen/openapi/internal/ir/component_annotations.go index 397fe5750..cb51e0b40 100644 --- a/http/codegen/openapi/internal/ir/component_annotations.go +++ b/http/codegen/openapi/internal/ir/component_annotations.go @@ -3,6 +3,11 @@ package ir import "github.com/CaliLuke/loom/expr" type ( + componentExamples struct { + representative any + values []any + } + componentAnnotationIdentity struct { declaration string baseline string @@ -25,11 +30,11 @@ func (a *Analyzer) componentAnnotationScope(declaration, baseline, context strin declaration: declaration, baseline: baseline, context: context, examplePlan: a.examplePlan, } if a.componentAnnotations == nil { - a.componentAnnotations = make(map[componentAnnotationIdentity]map[componentAnnotationPosition]any) + a.componentAnnotations = make(map[componentAnnotationIdentity]map[componentAnnotationPosition]componentExamples) } owner, ok := a.componentAnnotations[identity] if !ok { - owner = make(map[componentAnnotationPosition]any) + owner = make(map[componentAnnotationPosition]componentExamples) a.componentAnnotations[identity] = owner } prior := a.annotationOwner @@ -48,3 +53,12 @@ func copyComponentAnnotation(value any) any { } return expr.DupAtt(&expr.AttributeExpr{Type: expr.Any, DefaultValue: value}).DefaultValue } + +// snapshot detaches all builtin containers without invoking custom codecs. +func (e componentExamples) snapshot() componentExamples { + result := componentExamples{representative: copyComponentAnnotation(e.representative)} + if e.values != nil { + result.values = copyComponentAnnotation(e.values).([]any) + } + return result +} diff --git a/http/codegen/openapi/internal/ir/component_annotations_test.go b/http/codegen/openapi/internal/ir/component_annotations_test.go index b5fcbeedd..3b5a979b2 100644 --- a/http/codegen/openapi/internal/ir/component_annotations_test.go +++ b/http/codegen/openapi/internal/ir/component_annotations_test.go @@ -175,11 +175,11 @@ func TestComponentAnnotationControlledWrapperReuse(t *testing.T) { originalSchema := analyzer.schemas["SampleData"] original := copyComponentAnnotation(originalSchema.Example) originalChild := copyComponentAnnotation(analyzer.schemas["SampleData"].Properties["data"].Example) - originalOwners := make(map[componentAnnotationIdentity]map[componentAnnotationPosition]any) + originalOwners := make(map[componentAnnotationIdentity]map[componentAnnotationPosition]componentExamples) for identity, positions := range analyzer.componentAnnotations { - originalOwners[identity] = make(map[componentAnnotationPosition]any) + originalOwners[identity] = make(map[componentAnnotationPosition]componentExamples) for position, value := range positions { - originalOwners[identity][position] = copyComponentAnnotation(value) + originalOwners[identity][position] = value.snapshot() } } for _, endpoint := range prepared.Endpoints { diff --git a/http/codegen/openapi/internal/ir/document.go b/http/codegen/openapi/internal/ir/document.go index ac2255e02..3f9daa372 100644 --- a/http/codegen/openapi/internal/ir/document.go +++ b/http/codegen/openapi/internal/ir/document.go @@ -180,18 +180,22 @@ func buildRequestBody(endpointIR *transportir.Endpoint, bodies *EndpointBodies, contentTypes = []string{"application/x-www-form-urlencoded"} } content := make(map[string]*MediaType, len(contentTypes)) - for _, contentType := range contentTypes { - schema := bodies.RequestBody - target := endpointIR.Request.BodyValue - if prepared, exists := bodies.requestMedia[contentType]; exists { - schema = prepared - target = endpointIR.Request.DocumentValues[contentType] - } else if endpointIR.Request.DocumentBody != nil { - target = endpointIR.Request.DocumentValue + if len(endpointIR.Request.MediaVariants) > 0 { + content = buildMediaVariantContent(endpointIR.Request.MediaVariants, bodies.requestMedia, closeObjects, schemaUsageRequest) + } else { + for _, contentType := range contentTypes { + schema := bodies.RequestBody + target := endpointIR.Request.BodyValue + if prepared, exists := bodies.requestMedia[contentType]; exists { + schema = prepared + target = endpointIR.Request.DocumentValues[contentType] + } else if endpointIR.Request.DocumentBody != nil { + target = endpointIR.Request.DocumentValue + } + content[contentType] = buildMediaType(bodyAttr, schema, closeObjects, + endpointIR.Request.DocumentBody != nil, target) + applyFormFieldEncoding(endpointIR.Request, content[contentType]) } - content[contentType] = buildMediaType(bodyAttr, schema, closeObjects, - endpointIR.Request.DocumentBody != nil, target) - applyFormFieldEncoding(endpointIR.Request, content[contentType]) } return &RequestBody{ ReferenceMetadata: componentMetadata(bodyAttr, "openapi:reference:requestBody"), @@ -287,6 +291,13 @@ func buildResponse( mediaSchemas map[string]*Schema, ) *Response { headers := headersFromIR(resp.Headers, rand, closeObjects, prepared) + if len(resp.MediaVariants) > 0 { + for name := range headers { + if strings.EqualFold(name, "Content-Type") { + delete(headers, name) // The content map owns representation selection. + } + } + } if cookieHeader := responseCookieHeader(resp.Cookies); cookieHeader != nil { if headers == nil { headers = make(map[string]*HeaderRef) @@ -322,6 +333,9 @@ func buildResponseContent( websocketHandshake bool, mediaSchemas map[string]*Schema, ) map[string]*MediaType { + if len(resp.MediaVariants) > 0 { + return buildMediaVariantContent(resp.MediaVariants, mediaSchemas, closeObjects, schemaUsageResponse) + } body := attributeForSchemaUsage(resp.DocumentBody, schemaUsageResponse) contentTypes := resp.ContentTypes var content map[string]*MediaType @@ -530,9 +544,12 @@ func hasStructuredExampleMetadata(meta expr.MetaExpr) bool { // represents its attribute's OpenAPI schema. func OpenAPIExampleValue(attr *expr.AttributeExpr, raw any) (any, bool) { if raw == nil { - if examples := attr.ExtractUserExamples(); len(examples) > 0 && - examples[len(examples)-1].ExplicitNull && expr.AllowsNull(attr) { - return NullExample{}, true + if expr.AllowsNull(attr) { + for _, example := range attr.ExtractUserExamples() { + if example.ExplicitNull { + return NullExample{}, true + } + } } return nil, false } diff --git a/http/codegen/openapi/internal/ir/media_variants.go b/http/codegen/openapi/internal/ir/media_variants.go new file mode 100644 index 000000000..bd97f59dc --- /dev/null +++ b/http/codegen/openapi/internal/ir/media_variants.go @@ -0,0 +1,25 @@ +package ir + +import "github.com/CaliLuke/loom/http/codegen/internal/transportir" + +func analyzeMediaVariants(a *Analyzer, variants []*transportir.MediaVariant, usage schemaUsage) map[string]*Schema { + schemas := make(map[string]*Schema, len(variants)) + for _, variant := range variants { + attribute := attributeForSchemaUsage(variant.Body, usage) + context := attributeExampleContext(attribute, a.closeObjects, "media-variant", variant.ContentType) + schemas[variant.ContentType] = a.analyzePreparedOccurrence(attribute, context, variant.Value) + } + return schemas +} + +func buildMediaVariantContent(variants []*transportir.MediaVariant, schemas map[string]*Schema, closeObjects bool, usage schemaUsage) map[string]*MediaType { + content := make(map[string]*MediaType, len(variants)) + for _, variant := range variants { + attribute := attributeForSchemaUsage(variant.Body, usage) + media := buildMediaType(attribute, schemas[variant.ContentType], closeObjects, false, variant.Value) + media.Metadata = mediaTypeMetadata(variant.Metadata) + applyFormFieldEncoding(&transportir.Request{Body: variant.Body, FormEncoded: len(variant.FormFieldModes) > 0, FormFieldModes: variant.FormFieldModes}, media) + content[variant.ContentType] = media + } + return content +} diff --git a/http/codegen/openapi/internal/ir/model.go b/http/codegen/openapi/internal/ir/model.go index b818bc11a..e0c916960 100644 --- a/http/codegen/openapi/internal/ir/model.go +++ b/http/codegen/openapi/internal/ir/model.go @@ -234,6 +234,8 @@ type ( Description string DefaultValue any Example any + // Examples retains multiple schema example values in declaration order. + Examples []any `json:",omitzero,omitempty"` ReadOnly bool WriteOnly bool @@ -304,6 +306,8 @@ type ( Description string DefaultValue any Example any + // Examples retains multiple schema example values in declaration order. + Examples []any `json:",omitzero,omitempty"` Media any ReadOnly bool @@ -361,6 +365,7 @@ func (s *Schema) MarshalJSON() ([]byte, error) { Description: s.Description, DefaultValue: s.DefaultValue, Example: s.Example, + Examples: s.Examples, ReadOnly: s.ReadOnly, WriteOnly: s.WriteOnly, Deprecated: s.Deprecated, diff --git a/http/codegen/openapi/internal/ir/operation_metadata.go b/http/codegen/openapi/internal/ir/operation_metadata.go index aa25cb3d5..50290987c 100644 --- a/http/codegen/openapi/internal/ir/operation_metadata.go +++ b/http/codegen/openapi/internal/ir/operation_metadata.go @@ -160,6 +160,9 @@ func paramsFromHeadersAndCookies(endpointIR *transportir.Endpoint, rand *expr.Ex var params []*ParameterRef for _, parameter := range endpointIR.Request.Headers { + if len(endpointIR.Request.MediaVariants) > 0 && strings.EqualFold(parameter.HTTPName, "Content-Type") { + continue // The content map owns representation selection in OpenAPI. + } if endpointIR.Security.IsParameter("header", parameter.HTTPName) { continue } @@ -202,7 +205,11 @@ func preparedParamFor( if _, explicit := attr.Meta.Last("openapi:description:schema"); !explicit { schema.Description = "" } - schema.Example = nil + if value == nil || !slices.ContainsFunc(value.Examples, func(example transportir.ValueExample) bool { + return metaValue(example.Meta, "openapi:example:scope") == "schema" + }) { + schema.Example = nil + } parameter := &Parameter{ Name: name, In: in, diff --git a/http/codegen/openapi/internal/ir/prepared_examples.go b/http/codegen/openapi/internal/ir/prepared_examples.go index 44007cc6d..5faaa210b 100644 --- a/http/codegen/openapi/internal/ir/prepared_examples.go +++ b/http/codegen/openapi/internal/ir/prepared_examples.go @@ -3,6 +3,7 @@ package ir import ( "encoding/json/jsontext" + "github.com/CaliLuke/loom/expr" "github.com/CaliLuke/loom/http/codegen/internal/transportir" ) @@ -40,16 +41,17 @@ func initPreparedExamples(target interface { } return true } - if len(prepared.Examples) == 1 && - !hasStructuredExampleMetadata(prepared.Examples[0].Meta) && - metaValue(prepared.Examples[0].Meta, "openapi:component:example") == "" { - if value, ok := preparedOpenAPIExampleValue(prepared.Examples[0]); ok { + examples := scopedPreparedExamples(prepared.Examples, "media") + if len(examples) == 1 && + !hasStructuredExampleMetadata(examples[0].Meta) && + metaValue(examples[0].Meta, "openapi:component:example") == "" { + if value, ok := preparedOpenAPIExampleValue(examples[0]); ok { target.setExample(value) } return true } - references := make(map[string]*ExampleRef, len(prepared.Examples)) - for _, example := range prepared.Examples { + references := make(map[string]*ExampleRef, len(examples)) + for _, example := range examples { value, ok := preparedOpenAPIExampleValue(example) if !ok { continue @@ -66,6 +68,21 @@ func initPreparedExamples(target interface { return true } +func scopedPreparedExamples(examples []transportir.ValueExample, scope string) []transportir.ValueExample { + selected := make([]transportir.ValueExample, 0, len(examples)) + for _, example := range examples { + if exampleInScope(example.Meta, scope) { + selected = append(selected, example) + } + } + return selected +} + +func exampleInScope(meta expr.MetaExpr, scope string) bool { + owner, present := meta.Last("openapi:example:scope") + return !present || owner == scope +} + func buildPreparedExample(example transportir.ValueExample, value any) *Example { summary := example.Summary if authored, ok := example.Meta.Last("openapi:example:summary"); ok { diff --git a/http/codegen/openapi/internal/ir/render.go b/http/codegen/openapi/internal/ir/render.go index 5232578e6..b395efa3b 100644 --- a/http/codegen/openapi/internal/ir/render.go +++ b/http/codegen/openapi/internal/ir/render.go @@ -37,6 +37,10 @@ func RenderSchema(schema *Schema) *openapi.Schema { out.Description = schema.Description out.DefaultValue = schema.DefaultValue out.Example = schema.Example + out.Examples = schema.Examples + if len(out.Examples) > 0 { + out.Example = nil + } out.ReadOnly = schema.ReadOnly out.WriteOnly = schema.WriteOnly out.Deprecated = schema.Deprecated diff --git a/http/codegen/openapi/internal/ir/representation_registry.go b/http/codegen/openapi/internal/ir/representation_registry.go index 3d244bf34..4557f07a1 100644 --- a/http/codegen/openapi/internal/ir/representation_registry.go +++ b/http/codegen/openapi/internal/ir/representation_registry.go @@ -67,7 +67,7 @@ func (a *Analyzer) analyzePreparedSchemaPositions( }() examplePlan := representationRoot(attribute, target) result := a.analyzeSchemaPositions(attribute, context, plan, examplePlan, noRef...) - if !a.customExampleValue && result != nil && result.Example == nil { + if !a.customExampleValue && result != nil && result.Example == nil && len(result.Examples) == 0 { restore := a.schemaPlanScope(plan, examplePlan) a.applySchemaExample(result, attribute, context) restore() diff --git a/http/codegen/openapi/internal/ir/schema_examples.go b/http/codegen/openapi/internal/ir/schema_examples.go new file mode 100644 index 000000000..678d900be --- /dev/null +++ b/http/codegen/openapi/internal/ir/schema_examples.go @@ -0,0 +1,86 @@ +package ir + +import "github.com/CaliLuke/loom/expr" + +func (a *Analyzer) applySchemaExample(s *Schema, attr *expr.AttributeExpr, context string) { + if a.annotationOwner != nil { + position := componentAnnotationPosition{context, fingerprintAttribute(attr, a.closeObjects)} + if example, found := a.annotationOwner[position]; found { + values := example.snapshot() + s.Example = values.representative + s.Examples = values.values + return + } + defer func() { + a.annotationOwner[position] = (componentExamples{representative: s.Example, values: s.Examples}).snapshot() + }() + } + s.Example, s.Examples = nil, nil + if a.suppressExamples != nil && a.suppressExamples(attr, a.closeObjects) { + return + } + if a.customExampleValue { + a.applyCustomSchemaExamples(s, attr, context) + return + } + a.applyPreparedSchemaExamples(s) +} + +func (a *Analyzer) applyCustomSchemaExamples(s *Schema, attr *expr.AttributeExpr, context string) { + if authored := attr.ExtractUserExamples(); len(authored) > 0 { + examples := make([]*expr.ExampleExpr, 0, len(authored)) + for _, example := range authored { + if exampleInScope(example.Meta, "schema") { + examples = append(examples, example) + } + } + for index, entry := range examples { + if value, ok := a.exampleValue(attr, entry.Value); ok { + if len(examples) > 1 { + s.Examples = append(s.Examples, value) + } + if index == len(examples)-1 { + s.Example = value + } + } + } + return + } + generator := exampleGeneratorForAttribute(a.rand, attr, a.closeObjects, context) + if example, ok := a.exampleValue(attr, attr.Example(generator)); ok { + s.Example = example + } +} + +func (a *Analyzer) applyPreparedSchemaExamples(s *Schema) { + if a.preparedExamples == nil { + return + } + set, prepared := a.preparedExamples.ExampleSets[a.examplePlan] + if !prepared || set == nil || !set.Prepared { + return + } + project := preparedOpenAPIExampleValue + if a.declaredExampleValue { + project = preparedDeclaredExampleValue + } + if len(set.Examples) > 0 { + examples := scopedPreparedExamples(set.Examples, "schema") + for index, entry := range examples { + if value, ok := project(entry); ok { + if len(examples) > 1 { + s.Examples = append(s.Examples, value) + } + if index == len(examples)-1 { + s.Example = value + } + } + } + return + } + if set.Representative != nil { + if value, ok := project(*set.Representative); ok { + s.Example = value + } + } +} diff --git a/http/codegen/openapi/internal/ir/schema_examples_test.go b/http/codegen/openapi/internal/ir/schema_examples_test.go new file mode 100644 index 000000000..5bc1b4585 --- /dev/null +++ b/http/codegen/openapi/internal/ir/schema_examples_test.go @@ -0,0 +1,120 @@ +package ir + +import ( + "encoding/json/v2" + "fmt" + "strings" + "testing" + + "github.com/stretchr/testify/require" + + "github.com/CaliLuke/loom/expr" + "github.com/CaliLuke/loom/http/codegen/internal/representation" +) + +func TestSchemaPreservesOrderedExamples(t *testing.T) { + for _, test := range []struct { + name string + attribute *expr.AttributeExpr + want string + }{ + {"strings", &expr.AttributeExpr{Type: expr.String, UserExamples: []*expr.ExampleExpr{{Value: "red"}, {Value: "blue"}}}, `["red","blue"]`}, + {"null and duplicate", &expr.AttributeExpr{Type: expr.String, Nullable: true, UserExamples: []*expr.ExampleExpr{{ExplicitNull: true}, {Value: "blue"}, {Value: "blue"}}}, `[null,"blue","blue"]`}, + {"bytes", &expr.AttributeExpr{Type: expr.Bytes, UserExamples: []*expr.ExampleExpr{{Value: []byte("a")}, {Value: []byte("b")}}}, `["YQ==","Yg=="]`}, + {"incomplete object omitted", &expr.AttributeExpr{Type: &expr.Object{{Name: "id", Attribute: &expr.AttributeExpr{Type: expr.String}}}, Validation: &expr.ValidationExpr{Required: []string{"id"}}, UserExamples: []*expr.ExampleExpr{{Value: map[string]any{}}, {Value: map[string]any{"id": "blue"}}}}, `[{"id":"blue"}]`}, + } { + t.Run(test.name, func(t *testing.T) { + schema := NewAnalyzer(expr.NewRandom("schema-examples"), false).AnalyzeSchema(test.attribute) + data, err := json.Marshal(RenderSchema(schema), json.Deterministic(true)) + require.NoError(t, err) + var object map[string]any + require.NoError(t, json.Unmarshal(data, &object)) + require.Contains(t, object, "examples") + require.NotContains(t, object, "example") + values, err := json.Marshal(object["examples"], json.Deterministic(true)) + require.NoError(t, err) + require.JSONEq(t, test.want, string(values)) + }) + } +} + +func TestSchemaExampleSnapshotsOwnPluralValues(t *testing.T) { + original := componentExamples{values: []any{map[string]any{"value": []any{"red"}}, NullExample{}}} + copy := original.snapshot() + copy.values[0].(map[string]any)["value"].([]any)[0] = "blue" + require.Equal(t, "red", original.values[0].(map[string]any)["value"].([]any)[0]) + require.Equal(t, NullExample{}, copy.values[1]) +} + +func TestSchemaExamplesCustomProjection(t *testing.T) { + attribute := &expr.AttributeExpr{Type: expr.String, UserExamples: []*expr.ExampleExpr{{Value: "red"}, {Value: "blue"}}} + calls := 0 + analyzer := NewAnalyzer(expr.NewRandom("custom"), false, WithExampleValue(func(attr *expr.AttributeExpr, value any) (any, bool) { + require.Same(t, attribute, attr) + calls++ + return "custom:" + value.(string), true + })) + schema := analyzer.AnalyzeSchema(attribute) + require.Equal(t, []any{"custom:red", "custom:blue"}, schema.Examples) + require.Equal(t, 2, calls) +} + +func TestSchemaExampleOverlayReplacesWholeGroup(t *testing.T) { + for _, local := range []*Schema{ + {Example: "local"}, + {Example: NullExample{}}, + {Example: "last", Examples: []any{"first", "last"}}, + } { + base := &Schema{Example: "base-last", Examples: []any{"base-first", "base-last"}} + result := overlayAsyncAnnotations(base, local) + require.Equal(t, local.Example, result.Example) + require.Equal(t, local.Examples, result.Examples) + } +} + +func TestSchemaExamplesCustomProjectionKeepsEarlierNull(t *testing.T) { + attribute := &expr.AttributeExpr{Type: expr.String, Nullable: true, UserExamples: []*expr.ExampleExpr{{ExplicitNull: true}, {Value: "blue"}}} + analyzer := NewAnalyzer(expr.NewRandom("custom-null"), false, WithExampleValue(OpenAPIExampleValue)) + require.Equal(t, []any{NullExample{}, "blue"}, analyzer.AnalyzeSchema(attribute).Examples) +} + +func TestSchemaExamplesNullableOverlayIsIdempotent(t *testing.T) { + minimum := 1 + base := &expr.UserTypeExpr{TypeName: "Color", AttributeExpr: &expr.AttributeExpr{Type: expr.String}} + attribute := &expr.AttributeExpr{Type: base, Nullable: true, Validation: &expr.ValidationExpr{MinLength: &minimum}, UserExamples: []*expr.ExampleExpr{{Value: "red"}, {Value: "blue"}}} + schema := NewAnalyzer(expr.NewRandom("nullable-overlay"), false).AnalyzeSchema(attribute) + require.Equal(t, []any{"red", "blue"}, schema.Examples) +} + +func TestSchemaExamplesRespectSourceScope(t *testing.T) { + for _, scopes := range [][]string{{"schema"}, {"media"}, {"schema", "media"}, {"schema", "schema", "media"}} { + t.Run(strings.Join(scopes, "/"), func(t *testing.T) { + attribute := &expr.AttributeExpr{Type: expr.String} + var wantSchema, wantMedia []any + for index, scope := range scopes { + value := fmt.Sprintf("value-%d", index) + attribute.UserExamples = append(attribute.UserExamples, &expr.ExampleExpr{Value: value, Meta: expr.MetaExpr{"openapi:example:scope": {scope}}}) + if scope == "schema" { + wantSchema = append(wantSchema, value) + } else { + wantMedia = append(wantMedia, value) + } + } + rand := expr.NewRandom("scoped") + target, err := representation.PrepareStandaloneExamples(attribute, rand) + require.NoError(t, err) + parameter := paramFor(attribute, target, "color", "query", rand).Value + if len(wantSchema) == 1 { + require.Equal(t, wantSchema[0], parameter.Schema.Example) + } else { + require.Equal(t, wantSchema, parameter.Schema.Examples) + } + if len(wantMedia) == 0 { + require.Nil(t, parameter.Example) + require.Empty(t, parameter.Examples) + } else { + require.Equal(t, wantMedia[0], parameter.Example) + } + }) + } +} diff --git a/http/codegen/openapi/internal/ir/tla/schema_examples/README.md b/http/codegen/openapi/internal/ir/tla/schema_examples/README.md new file mode 100644 index 000000000..4078f08bf --- /dev/null +++ b/http/codegen/openapi/internal/ir/tla/schema_examples/README.md @@ -0,0 +1,56 @@ +# Ordered schema examples + +`SchemaExamples.tla` models the projection of an already prepared authored group +onto a schema. `representative.cfg` reproduces #1002: retaining only the final +value loses the earlier value. `checked.cfg` retains the full ordered sequence, +including a null token and duplicate values. TLC 1.7.4 checks nine distinct states. + +Run explicitly with the pinned TLA+ 1.7.4 tools jar and temporary output paths: + +```sh +java -cp /path/to/tla2tools-1.7.4.jar tlc2.TLC -metadir /tmp/schema-examples-old -config representative.cfg SchemaExamples.tla +java -cp /path/to/tla2tools-1.7.4.jar tlc2.TLC -metadir /tmp/schema-examples-new -config checked.cfg SchemaExamples.tla +``` + +The first command must fail `OrderedValuesPreserved`; the second must pass. +The additional `collapsed.cfg` control merges schema/media groups and fails on +the first visit; `append.cfg` appends the group again and fails on the second +visit. Run them with the same command and separate temporary metadirs. The +checked mode keeps both scopes separate and replaces output on each visit. +`Analyzer.applySchemaExample` consumes the prepared sequence; `RenderSchema` +emits plural values without the redundant representative. Singleton output +continues to use `example`, which denotes the same one-value sequence. + +This finite model assumes valid, already projected values. It does not establish +Go copying, codec projection, provenance-marker extraction, filtering, component +identity or JSON/YAML encoding. +`schema_examples_test.go` checks the concrete projection; rendered and imported +contract tests cover serialization, regeneration and compilation. Proof tools +remain opt-in and are not CI dependencies. + +Validation for #1002 uses base `73762ebc0` and Go 1.27.3. The bounded generated +comparison covers `OpenAPIReusableComponentsDSL` (multiple object and header +examples and reusable references), `StreamingPartialExamplesDSL` (unchanged +singletons, invalid partial objects and streaming controls), and +`MultipleSchemaExamplesDSL` (nullable order, duplicates and byte values). Each +base and changed application was generated and compiled. With identical CLI +basenames, every generated Go file matches byte for byte; only OpenAPI JSON/YAML +change for the two plural fixtures. The singleton streaming fixture is entirely +unchanged. Intended differences are replacement of representative schema +`example` values with complete `examples` arrays. Media Example Objects and +component names remain unchanged in the existing unscoped fixtures. The new +scoped fixture additionally excludes schema-only entries from media output, +retaining its separately authored green media example. + +The import test extends `supported.json`, compiles the regenerated service, +checks ordered null/duplicate values, repeats generation in an independent +process, and strictly re-analyzes the schema after removing separately +unsupported document-identity/server fields. Direct tests additionally check +partial-example omission, detached snapshots, custom projection, scope-specific +representatives, and complete local annotation replacement. Review exposed +parameter-array loss, inline schema/media conflation, and repeated nullable +overlay appends; durable regressions cover all three. Imported source groups +retain their scope rather than deduplicating equal values. The OpenAPI package tests check parser admission, +JSON/YAML goldens and isolated-process byte stability for both output versions; +the focused rendered test supports the pinned Vacuum gate through +`LOOM_OPENAPI_CONTRACT=1`. diff --git a/http/codegen/openapi/internal/ir/tla/schema_examples/SchemaExamples.tla b/http/codegen/openapi/internal/ir/tla/schema_examples/SchemaExamples.tla new file mode 100644 index 000000000..02f6603f9 --- /dev/null +++ b/http/codegen/openapi/internal/ir/tla/schema_examples/SchemaExamples.tla @@ -0,0 +1,19 @@ +-------------------------- MODULE SchemaExamples -------------------------- +EXTENDS Naturals, Sequences +CONSTANT Mode +VARIABLES source, media, output, mediaOutput, visits +vars == <> +Inputs == {<<"red", "blue">>, <<"null", "blue", "blue">>, <<"red">>} +Init == /\ source \in Inputs /\ media = <<"green">> + /\ output = <<>> /\ mediaOutput = <<>> /\ visits = 0 +Emit == /\ visits < 2 + /\ output' = CASE Mode = "representative" -> <> + [] Mode = "collapsed" -> source \o media + [] Mode = "append" -> output \o source + [] OTHER -> source + /\ mediaOutput' = IF Mode = "collapsed" THEN source \o media ELSE media + /\ visits' = visits + 1 + /\ UNCHANGED <> +Spec == Init /\ [][Emit]_vars +OrderedValuesPreserved == visits > 0 => output = source /\ mediaOutput = media +============================================================================= diff --git a/http/codegen/openapi/internal/ir/tla/schema_examples/append.cfg b/http/codegen/openapi/internal/ir/tla/schema_examples/append.cfg new file mode 100644 index 000000000..bf390be36 --- /dev/null +++ b/http/codegen/openapi/internal/ir/tla/schema_examples/append.cfg @@ -0,0 +1,4 @@ +SPECIFICATION Spec +CONSTANT Mode = "append" +INVARIANT OrderedValuesPreserved +CHECK_DEADLOCK FALSE diff --git a/http/codegen/openapi/internal/ir/tla/schema_examples/checked.cfg b/http/codegen/openapi/internal/ir/tla/schema_examples/checked.cfg new file mode 100644 index 000000000..93c68d927 --- /dev/null +++ b/http/codegen/openapi/internal/ir/tla/schema_examples/checked.cfg @@ -0,0 +1,4 @@ +SPECIFICATION Spec +CONSTANT Mode = "ordered" +INVARIANT OrderedValuesPreserved +CHECK_DEADLOCK FALSE diff --git a/http/codegen/openapi/internal/ir/tla/schema_examples/collapsed.cfg b/http/codegen/openapi/internal/ir/tla/schema_examples/collapsed.cfg new file mode 100644 index 000000000..2d8f72d0c --- /dev/null +++ b/http/codegen/openapi/internal/ir/tla/schema_examples/collapsed.cfg @@ -0,0 +1,4 @@ +SPECIFICATION Spec +CONSTANT Mode = "collapsed" +INVARIANT OrderedValuesPreserved +CHECK_DEADLOCK FALSE diff --git a/http/codegen/openapi/internal/ir/tla/schema_examples/representative.cfg b/http/codegen/openapi/internal/ir/tla/schema_examples/representative.cfg new file mode 100644 index 000000000..747947330 --- /dev/null +++ b/http/codegen/openapi/internal/ir/tla/schema_examples/representative.cfg @@ -0,0 +1,4 @@ +SPECIFICATION Spec +CONSTANT Mode = "representative" +INVARIANT OrderedValuesPreserved +CHECK_DEADLOCK FALSE diff --git a/http/codegen/openapi/schema.go b/http/codegen/openapi/schema.go index 44fa74794..085302084 100644 --- a/http/codegen/openapi/schema.go +++ b/http/codegen/openapi/schema.go @@ -27,6 +27,8 @@ type ( DefaultValue any `json:"default,omitzero" yaml:"default,omitempty"` // Example is an example value accepted by the schema. Example any `json:"example,omitzero" yaml:"example,omitempty"` + // Examples contains ordered JSON values, without Example Object metadata. + Examples []any `json:"examples,omitzero,omitempty" yaml:"examples,omitempty"` // ReadOnly marks a value as response-only. ReadOnly bool `json:"readOnly,omitzero" yaml:"readOnly,omitempty"` diff --git a/http/codegen/openapi/v3/example_determinism_test.go b/http/codegen/openapi/v3/example_determinism_test.go index 8fe7266ba..7c7c14326 100644 --- a/http/codegen/openapi/v3/example_determinism_test.go +++ b/http/codegen/openapi/v3/example_determinism_test.go @@ -103,23 +103,23 @@ func TestFilesAreByteStableAcrossIndependentProcesses(t *testing.T) { { name: "OpenAPI 3.2 default formatting", wantVersion: openapiv3.OpenAPIVersion, - wantJSONHash: "c424b21ecd4f4a6563d4af4f5406ce197b91acb22fc8ee2a58f5efe62cc8b70a", - wantYAMLHash: "d1d45c323a539d611e98bd5fa949cde3ad400a8ff1768cbaba778e4b109b5104", + wantJSONHash: "65bd277f226adc9a7b147be7418f3b9818785587cf897e4a3ea6b9e129f06676", + wantYAMLHash: "98807a8d5efecc7f99463f176e26099b37a4b03bd53df665ebc4e43ab27e2864", }, { name: "OpenAPI 3.1 compatibility", target: "3.1", wantVersion: openapiv3.OpenAPICompatibilityVersion, - wantJSONHash: "f7bc3bae9cd33a6b5d0944c2936d601023d5b989cc71439855c8cda16c426673", - wantYAMLHash: "a45feb0261f1cbe4223cb86d7b40ebfc8b7ff5eb0f86d00e148a08a0ac36c590", + wantJSONHash: "b1898a08fc6bd712bccf1baaf917515f6f0b81ecde68419750cfdb821cac48a1", + wantYAMLHash: "fd6dfd5cdb3e37fc3a3dedaa21395d110e1749e6e54799f94a7b43249161c1ff", }, { name: "OpenAPI 3.2 configured formatting", prefix: " ", indent: "\t", wantVersion: openapiv3.OpenAPIVersion, - wantJSONHash: "0a8ac167549214951bc9cd89941c8800f2ed4d30f0352062365c036e861b7d6d", - wantYAMLHash: "d1d45c323a539d611e98bd5fa949cde3ad400a8ff1768cbaba778e4b109b5104", + wantJSONHash: "78dbbb59f9795f44cdd52158a6422014e8f5af673eb344431a5665dc2329b8c8", + wantYAMLHash: "98807a8d5efecc7f99463f176e26099b37a4b03bd53df665ebc4e43ab27e2864", }, } diff --git a/http/codegen/openapi/v3/ir_adapter.go b/http/codegen/openapi/v3/ir_adapter.go index 7d6932831..1a7fd8ab1 100644 --- a/http/codegen/openapi/v3/ir_adapter.go +++ b/http/codegen/openapi/v3/ir_adapter.go @@ -339,6 +339,7 @@ func schemaToIR(schema *openapi.Schema) *openapiir.Schema { Description: schema.Description, DefaultValue: schema.DefaultValue, Example: schema.Example, + Examples: schema.Examples, ReadOnly: schema.ReadOnly, WriteOnly: schema.WriteOnly, Deprecated: schema.Deprecated, diff --git a/http/codegen/openapi/v3/media_contract_test.go b/http/codegen/openapi/v3/media_contract_test.go new file mode 100644 index 000000000..a84e2ece6 --- /dev/null +++ b/http/codegen/openapi/v3/media_contract_test.go @@ -0,0 +1,63 @@ +package openapiv3_test + +import ( + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/require" + + "github.com/CaliLuke/loom/http/codegen/testdata" +) + +func TestRenderedDistinctMediaContracts(t *testing.T) { + var vacuum contractVacuum + if os.Getenv("LOOM_OPENAPI_CONTRACT") != "" { + vacuum = installContractVacuum(t) + } + for _, version := range []struct{ target, version string }{{"3.1", "3.1.1"}, {"3.2", "3.2.0"}} { + t.Run(version.target, func(t *testing.T) { + artifacts := renderOpenAPIArtifactsForVersion(t, testdata.MultipleMediaContractsDSL, version.target, version.version) + for _, spec := range []map[string]any{decodeOpenAPIJSON(t, artifacts.JSON), decodeOpenAPIExampleYAML(t, artifacts.YAML)} { + op := requireOperation(t, spec, "/media-contracts", "post") + request := requireMap(t, op["requestBody"], "request body") + requestContent := requireMap(t, request["content"], "request content") + require.Len(t, requestContent, 7) + responses := requireMap(t, op["responses"], "responses") + response := requireMap(t, responses["200"], "response") + responseContent := requireMap(t, response["content"], "response content") + require.Len(t, responseContent, 7) + for _, content := range []map[string]any{requestContent, responseContent} { + plain := requireMap(t, content["text/plain"], "text media") + require.Equal(t, "hello", plain["example"]) + schema := requireMap(t, plain["schema"], "text schema") + require.Equal(t, "string", schema["type"]) + vendor := requireMap(t, content["application/vnd.loom+json"], "vendor media") + require.EqualValues(t, 7, vendor["example"]) + schema = requireMap(t, vendor["schema"], "vendor schema") + require.Equal(t, "integer", schema["type"]) + regular := requireMap(t, content["application/json"], "json media") + examples := requireMap(t, regular["examples"], "json examples") + require.Len(t, examples, 2) + } + } + if vacuum.binary != "" { + for ext, data := range map[string][]byte{"json": artifacts.JSON, "yaml": artifacts.YAML} { + path := filepath.Join(t.TempDir(), "openapi."+ext) + require.NoError(t, os.WriteFile(path, data, 0600)) + require.NoError(t, vacuum.lint(path)) + } + } + }) + } +} + +func TestRenderedMediaContractsProjectParentExamples(t *testing.T) { + artifacts := renderOpenAPIArtifactsForVersion(t, testdata.MultipleMediaContractsDSL, "3.2", "3.2.0") + op := requireOperation(t, decodeOpenAPIJSON(t, artifacts.JSON), "/media-root-examples", "post") + request := requireMap(t, op["requestBody"], "request") + content := requireMap(t, request["content"], "content") + media := requireMap(t, content["application/json"], "json") + examples := requireMap(t, media["examples"], "examples") + require.Len(t, examples, 2) +} diff --git a/http/codegen/openapi/v3/schema_examples_test.go b/http/codegen/openapi/v3/schema_examples_test.go new file mode 100644 index 000000000..4b4e04c5d --- /dev/null +++ b/http/codegen/openapi/v3/schema_examples_test.go @@ -0,0 +1,44 @@ +package openapiv3_test + +import ( + "os" + "path/filepath" + "strings" + "testing" + + "github.com/stretchr/testify/require" + + "github.com/CaliLuke/loom/http/codegen/testdata" +) + +func TestRenderedMultipleSchemaExamples(t *testing.T) { + var vacuum contractVacuum + if os.Getenv("LOOM_OPENAPI_CONTRACT") != "" { + vacuum = installContractVacuum(t) + } + for _, version := range []struct{ target, version string }{{"3.1", "3.1.1"}, {"3.2", "3.2.0"}} { + t.Run(version.target, func(t *testing.T) { + artifacts := renderOpenAPIArtifactsForVersion(t, testdata.MultipleSchemaExamplesDSL, version.target, version.version) + for _, spec := range []map[string]any{decodeOpenAPIJSON(t, artifacts.JSON), decodeOpenAPIExampleYAML(t, artifacts.YAML)} { + components := requireMap(t, spec["components"], "components") + schemas := requireMap(t, components["schemas"], "schemas") + operation := requireOperation(t, spec, "/schema-examples", "post") + media := requireResponseMediaType(t, operation, "application/json") + require.Equal(t, "green", media["example"]) + colors := requireMap(t, media["schema"], "colors") + if ref, ok := colors["$ref"].(string); ok && colors["examples"] == nil { + colors = requireMap(t, schemas[strings.TrimPrefix(ref, "#/components/schemas/")], "referenced colors") + } + require.Equal(t, []any{nil, "red", "blue", "blue"}, colors["examples"]) + require.NotContains(t, colors, "example") + } + if vacuum.binary != "" { + for ext, data := range map[string][]byte{"json": artifacts.JSON, "yaml": artifacts.YAML} { + path := filepath.Join(t.TempDir(), "openapi."+ext) + require.NoError(t, os.WriteFile(path, data, 0o600)) + require.NoError(t, vacuum.lint(path)) + } + } + }) + } +} diff --git a/http/codegen/openapi/v3/schema_refs.go b/http/codegen/openapi/v3/schema_refs.go index 02f3d8939..26df257c2 100644 --- a/http/codegen/openapi/v3/schema_refs.go +++ b/http/codegen/openapi/v3/schema_refs.go @@ -233,6 +233,7 @@ func isPureRefSchema(schema *openapi.Schema) bool { schema.Description == "" && schema.DefaultValue == nil && schema.Example == nil && + len(schema.Examples) == 0 && !schema.ReadOnly && !schema.WriteOnly && !schema.Deprecated && diff --git a/http/codegen/openapi/v3/testdata/golden/reusable-components_file0.golden b/http/codegen/openapi/v3/testdata/golden/reusable-components_file0.golden index 849c06458..a6d61988f 100644 --- a/http/codegen/openapi/v3/testdata/golden/reusable-components_file0.golden +++ b/http/codegen/openapi/v3/testdata/golden/reusable-components_file0.golden @@ -53,7 +53,10 @@ } }, "schema": { - "example": "trace_456", + "examples": [ + "trace_123", + "trace_456" + ], "type": "string" } } @@ -72,12 +75,20 @@ }, "schema": { "$ref": "#/components/schemas/Credentials", - "example": { - "legacyToken": "legacy-token-2", - "password": "swordfish", - "profile": "eyJyb2xlIjoiZWRpdG9yIn0=", - "username": "bob" - } + "examples": [ + { + "legacyToken": "legacy-token", + "password": "hunter2", + "profile": "eyJyb2xlIjoiYWRtaW4ifQ==", + "username": "alice" + }, + { + "legacyToken": "legacy-token-2", + "password": "swordfish", + "profile": "eyJyb2xlIjoiZWRpdG9yIn0=", + "username": "bob" + } + ] } } }, @@ -101,10 +112,16 @@ }, "schema": { "$ref": "#/components/schemas/Session_cf64610497754895", - "example": { - "expiresAt": "2026-03-21T12:00:00Z", - "token": "token-456" - } + "examples": [ + { + "expiresAt": "2026-03-20T12:00:00Z", + "token": "token-123" + }, + { + "expiresAt": "2026-03-21T12:00:00Z", + "token": "token-456" + } + ] } } }, @@ -118,12 +135,20 @@ }, "schemas": { "Credentials": { - "example": { - "legacyToken": "legacy-token-2", - "password": "swordfish", - "profile": "eyJyb2xlIjoiZWRpdG9yIn0=", - "username": "bob" - }, + "examples": [ + { + "legacyToken": "legacy-token", + "password": "hunter2", + "profile": "eyJyb2xlIjoiYWRtaW4ifQ==", + "username": "alice" + }, + { + "legacyToken": "legacy-token-2", + "password": "swordfish", + "profile": "eyJyb2xlIjoiZWRpdG9yIn0=", + "username": "bob" + } + ], "properties": { "legacyToken": { "deprecated": true, @@ -153,10 +178,16 @@ "type": "object" }, "Session_cf64610497754895": { - "example": { - "expiresAt": "2026-03-21T12:00:00Z", - "token": "token-456" - }, + "examples": [ + { + "expiresAt": "2026-03-20T12:00:00Z", + "token": "token-123" + }, + { + "expiresAt": "2026-03-21T12:00:00Z", + "token": "token-456" + } + ], "properties": { "expiresAt": { "example": "2026-03-20T12:00:00Z", diff --git a/http/codegen/openapi/v3/testdata/golden/reusable-components_file1.golden b/http/codegen/openapi/v3/testdata/golden/reusable-components_file1.golden index 95da2f594..ba1efe0d2 100644 --- a/http/codegen/openapi/v3/testdata/golden/reusable-components_file1.golden +++ b/http/codegen/openapi/v3/testdata/golden/reusable-components_file1.golden @@ -69,11 +69,15 @@ components: username: type: string example: alice - example: - legacyToken: legacy-token-2 - password: swordfish - profile: eyJyb2xlIjoiZWRpdG9yIn0= - username: bob + examples: + - legacyToken: legacy-token + password: hunter2 + profile: eyJyb2xlIjoiYWRtaW4ifQ== + username: alice + - legacyToken: legacy-token-2 + password: swordfish + profile: eyJyb2xlIjoiZWRpdG9yIn0= + username: bob required: - username - password @@ -88,9 +92,11 @@ components: type: string example: token-123 readOnly: true - example: - expiresAt: "2026-03-21T12:00:00Z" - token: token-456 + examples: + - expiresAt: "2026-03-20T12:00:00Z" + token: token-123 + - expiresAt: "2026-03-21T12:00:00Z" + token: token-456 required: - token - expiresAt @@ -98,7 +104,9 @@ components: XTraceIDHeader: schema: type: string - example: trace_456 + examples: + - trace_123 + - trace_456 examples: trace-a: $ref: '#/components/examples/XTraceIDHeaderTraceAExample' @@ -110,11 +118,15 @@ components: content: application/json: schema: - example: - legacyToken: legacy-token-2 - password: swordfish - profile: eyJyb2xlIjoiZWRpdG9yIn0= - username: bob + examples: + - legacyToken: legacy-token + password: hunter2 + profile: eyJyb2xlIjoiYWRtaW4ifQ== + username: alice + - legacyToken: legacy-token-2 + password: swordfish + profile: eyJyb2xlIjoiZWRpdG9yIn0= + username: bob $ref: '#/components/schemas/Credentials' examples: backup: @@ -132,9 +144,11 @@ components: content: application/json: schema: - example: - expiresAt: "2026-03-21T12:00:00Z" - token: token-456 + examples: + - expiresAt: "2026-03-20T12:00:00Z" + token: token-123 + - expiresAt: "2026-03-21T12:00:00Z" + token: token-456 $ref: '#/components/schemas/Session_cf64610497754895' examples: current: diff --git a/http/codegen/service_data_body_types.go b/http/codegen/service_data_body_types.go index 5ec25f0a1..78d7e463e 100644 --- a/http/codegen/service_data_body_types.go +++ b/http/codegen/service_data_body_types.go @@ -61,6 +61,8 @@ func (sds *ServicesData) buildRequestBodyType(body, att *expr.AttributeExpr, end } init := sds.buildRequestBodyInit(body, att, endpointName, pkg, details.validateDefinition, svr, svcctx, httpctx, sd) return &TypeData{ + attribute: body, + context: httpctx, Name: body.Type.Name(), VarName: details.varName, Description: details.description, @@ -293,6 +295,8 @@ func (sds *ServicesData) buildResponseBodyType(body, att *expr.AttributeExpr, lo data.ref = init.ReturnTypeRef } result := &TypeData{ + attribute: body, + context: httpctx, Name: data.name, VarName: data.varName, Description: data.desc, diff --git a/http/codegen/service_data_element_types.go b/http/codegen/service_data_element_types.go index 016b52257..45fcf657e 100644 --- a/http/codegen/service_data_element_types.go +++ b/http/codegen/service_data_element_types.go @@ -2,6 +2,7 @@ package codegen import ( "github.com/CaliLuke/loom/codegen" + "github.com/CaliLuke/loom/expr" "github.com/CaliLuke/loom/http/codegen/internal/transportir" "github.com/CaliLuke/loom/internal/securitygen" ) @@ -111,6 +112,10 @@ type ( // TypeData contains the data needed to render a type definition. TypeData struct { + // Media carries the selected HTTP occurrence's media dispatch plan. + Media *MediaBodyData + attribute *expr.AttributeExpr + context *codegen.AttributeContext // Value retains the semantic source and finalized body representation plan. Value *transportir.ValueTarget // Name is the type name. diff --git a/http/codegen/service_data_json_input.go b/http/codegen/service_data_json_input.go index 2811c63d1..a2e5c0f4d 100644 --- a/http/codegen/service_data_json_input.go +++ b/http/codegen/service_data_json_input.go @@ -68,6 +68,7 @@ func newJSONBodyInput(attribute *expr.AttributeExpr, endpoint string, sd *Servic } switch actual := typ.(type) { case expr.UserType: + delete(actual.Attribute().Meta, "http:media:form-root") actual.Rename(name) case *expr.Union: actual.TypeName = name diff --git a/http/codegen/service_data_media.go b/http/codegen/service_data_media.go new file mode 100644 index 000000000..6769a0453 --- /dev/null +++ b/http/codegen/service_data_media.go @@ -0,0 +1,75 @@ +package codegen + +import ( + "fmt" + "maps" + "slices" + "strconv" + "strings" + + "github.com/CaliLuke/loom/codegen" + "github.com/CaliLuke/loom/expr" + "github.com/CaliLuke/loom/http/codegen/internal/representation" + "github.com/CaliLuke/loom/http/codegen/internal/transportir" +) + +type ( + // MediaBodyData describes dispatch for one generated HTTP union body. + MediaBodyData struct { + // Contracts is the Go literal of the ordered media map keys. + Contracts string + // Variants binds each key to the actual emitted union field. + Variants []*MediaVariantData + } + // MediaVariantData is one typed codec branch of a generated HTTP body. + MediaVariantData struct { + // Tag is the service union discriminator selected by this media. + Tag string + // Field is the generated union field and setter suffix. + Field string + // TypeRef is the actual generated branch carrier. + TypeRef string + // AllowsNull records explicit JSON null admission for this branch. + AllowsNull bool + // Value retains the actual branch carrier's runtime representation plan. + Value *transportir.ValueTarget + // Parts is the Go literal of per-field multipart content types. + Parts string + } +) + +func attachMediaBodyData(body *TypeData, variants []*transportir.MediaVariant) { + if body == nil || len(variants) == 0 { + return + } + data := &MediaBodyData{} + contracts := make([]string, len(variants)) + for i, variant := range variants { + branch := expr.HTTPMediaBranch(body.attribute, expr.AttributeName(variant.Branch)) + if branch == nil { + panic(fmt.Sprintf("HTTP media branch %q is absent from emitted body %s", variant.Branch, body.Name)) + } + contracts[i] = strconv.Quote(variant.ContentType) + data.Variants = append(data.Variants, &MediaVariantData{ + Tag: variant.Tag, + Parts: mediaPartTypesLiteral(variant.PartTypes), + Field: codegen.Goify(branch.Name, true), + TypeRef: body.context.Scope.Scope().GoTypeRef(branch.Attribute), + AllowsNull: expr.AllowsNull(branch.Attribute), + Value: representation.BuildValuePlan(variant.Value, branch.Attribute, body.context, expr.ValuePlanRuntime), + }) + } + data.Contracts = "[]string{" + strings.Join(contracts, ", ") + "}" + body.Media = data +} + +func mediaPartTypesLiteral(parts map[string]string) string { + if len(parts) == 0 { + return "nil" + } + entries := make([]string, 0, len(parts)) + for _, name := range slices.Sorted(maps.Keys(parts)) { + entries = append(entries, strconv.Quote(name)+":"+strconv.Quote(parts[name])) + } + return "map[string]string{" + strings.Join(entries, ",") + "}" +} diff --git a/http/codegen/service_data_payload.go b/http/codegen/service_data_payload.go index 6d9a61687..90ae1649b 100644 --- a/http/codegen/service_data_payload.go +++ b/http/codegen/service_data_payload.go @@ -125,6 +125,8 @@ func (b *payloadBuilder) buildInit(request *RequestData) *InitData { func (b *payloadBuilder) buildRequestData() (*RequestData, *ParamData) { serverBodyData, clientBodyData := b.buildRequestBodies() + attachMediaBodyData(serverBodyData, b.endpointIR.Request.MediaVariants) + attachMediaBodyData(clientBodyData, b.endpointIR.Request.MediaVariants) paramsData, queryData, headersData, cookiesData, mapQueryParam := b.buildRequestElements() multipartGen, multipartFiles := generatedMultipartRequestData(b.endpointIR.Request) registerRequestBodyTypeNames(serverBodyData, b.sd) diff --git a/http/codegen/service_data_response.go b/http/codegen/service_data_response.go index e668648e3..fc8eacbe7 100644 --- a/http/codegen/service_data_response.go +++ b/http/codegen/service_data_response.go @@ -124,6 +124,10 @@ func (sds *ServicesData) buildSingleResponseData( field := selectBodyField(result, resp.Body) origin := field.Name serverBodyData, clientBodyData := sds.buildResponseBodyData(resp, result, origin, viewed, md, endpointIR, sd) + for _, body := range serverBodyData { + attachMediaBodyData(body, resp.MediaVariants) + } + attachMediaBodyData(clientBodyData, resp.MediaVariants) init := sds.buildResponseResultInit(resp, result, field, md, httpclictx, svcctx, headersData, cookiesData, endpointIR, sd) tag := responseTagData(resp.Tag, result, viewed, svcctx) return newResponseData( @@ -413,9 +417,10 @@ func (sds *ServicesData) buildClientBodyUnmarshalCode( src = sourceVar[0] } sourceContext := httpclictx - if len(sourceVar) == 0 && httpclictx.JSONPresence { + if len(sourceVar) == 0 { sourceContext = httpclictx.Dup() - sourceContext.CollectionElementPresence = true + sourceContext.CollectionElementPresence = sourceContext.JSONPresence + applyUserTypeLayout(sourceContext, sd, source, false) } plan, err := unmarshal(source, target, src, sourceContext, targetctx) if err != nil { diff --git a/http/codegen/service_data_response_errors.go b/http/codegen/service_data_response_errors.go index 993a0bfa0..1c766b931 100644 --- a/http/codegen/service_data_response_errors.go +++ b/http/codegen/service_data_response_errors.go @@ -138,6 +138,10 @@ func (b *errorBuilder) buildResultInitCode(errorResponse *transportir.ResponseSt func (b *errorBuilder) buildResponseData(errorResponse *transportir.ResponseStatus, errctx *codegen.AttributeContext, init *InitData) *ResponseData { httpError := errorResponse.Error serverBodyData, clientBodyData := b.buildResponseBodyData(errorResponse) + for _, body := range serverBodyData { + attachMediaBodyData(body, errorResponse.MediaVariants) + } + attachMediaBodyData(clientBodyData, errorResponse.MediaVariants) vars := newResponseVarScope(b.sd) headers := b.sds.extractHeaders(errorResponse.Headers, httpError.Attribute, errctx, b.sd.Scope, vars, b.sds.examplesFor(b.sd)) cookies := b.sds.extractResponseCookies(errorResponse.Cookies, httpError.Attribute, errctx, b.sd.Scope, vars, b.sds.examplesFor(b.sd)) diff --git a/http/codegen/template_sources_request.go b/http/codegen/template_sources_request.go index 73b5a1b41..8eb3f85ee 100644 --- a/http/codegen/template_sources_request.go +++ b/http/codegen/template_sources_request.go @@ -183,7 +183,22 @@ func {{ .RequestEncoder }}(encoder func(*http.Request) loomhttp.Encoder) func(*h body.ID = id {{- end }} {{- end }} - {{- if .Payload.Request.FormEncoded }} + {{- if and .Payload.Request.ClientBody .Payload.Request.ClientBody.Media }} + { + mediaIndex := -1 + var mediaValue any + var mediaParts map[string]string + switch string(body.Kind()) { + {{- range $i, $variant := .Payload.Request.ClientBody.Media.Variants }} + case {{ printf "%q" .Tag }}: + mediaIndex, mediaValue, mediaParts = {{ $i }}, body.{{ .Field }}, {{ .Parts }} + {{- end }} + } + if err := loomhttp.EncodeMediaRequest(req, mediaIndex, {{ .Payload.Request.ClientBody.Media.Contracts }}, mediaValue, mediaParts, encoder); err != nil { + return loomhttp.ErrEncodingError("{{ .ServiceName }}", "{{ .Method.Name }}", err) + } + } + {{- else if .Payload.Request.FormEncoded }} if err := loomhttp.SetFormRequest(req, &body); err != nil { return loomhttp.ErrEncodingError("{{ .ServiceName }}", "{{ .Method.Name }}", err) } diff --git a/http/codegen/template_sources_request_decoder.go b/http/codegen/template_sources_request_decoder.go index 72dd675ea..ed2e465c3 100644 --- a/http/codegen/template_sources_request_decoder.go +++ b/http/codegen/template_sources_request_decoder.go @@ -105,7 +105,27 @@ func {{ .RequestDecoder }}(mux loomhttp.Muxer, {{ if $usesDecoder }}decoder{{ el } } {{- else }} +{{- if .Payload.Request.ServerBody.Media }} + { + mediaIndex, mediaErr := loomhttp.SelectRequestMediaType(r, {{ .Payload.Request.ServerBody.Media.Contracts }}, {{ not .Payload.Request.MustHaveBody }}) + if mediaErr != nil { + err = mediaErr + } else { + switch mediaIndex { + {{- range $i, $variant := .Payload.Request.ServerBody.Media.Variants }} + case {{ $i }}: + var mediaValue {{ .TypeRef }} + err = loomhttp.DecodeMediaRequest(r, &mediaValue, {{ .AllowsNull }}, {{ .Parts }}, decoder) + if err == nil { + body.Set{{ .Field }}(mediaValue) + } + {{- end }} + } + } + } +{{- else }} err = decoder({{ if .Payload.Request.BodyAllowsNull }}r{{ else }}loomhttp.WithNonNullableBody(r){{ end }}).Decode({{ if not (or .Payload.Request.OptionalObjectBody .Payload.Request.OptionalPrimitiveBody) }}&{{ end }}body) +{{- end }} if err != nil { {{- if .Payload.Request.MustHaveBody }} if errors.Is(err, io.EOF) { diff --git a/http/codegen/template_sources_response.go b/http/codegen/template_sources_response.go index ac1d6d6dd..e8d40500d 100644 --- a/http/codegen/template_sources_response.go +++ b/http/codegen/template_sources_response.go @@ -100,7 +100,9 @@ func {{ .ErrorEncoder }}(encoder func(context.Context, http.ResponseWriter) loom problem := loomhttp.NewProblemResponse(ctx, {{ if .ErrorHeader }}v{{ else }}res{{ end }}, {{ .Code }}, {{ .ProblemTypeOverride }}, {{ .ProblemTitleOverride }}) {{- end }} {{- if .EncodePlan.HasBody }} + {{- if not .EncodePlan.FirstBody.Media }} enc := encoder(ctx, w) + {{- end }} {{- if .ErrorHeader }} var body any if formatter != nil { @@ -232,6 +234,36 @@ func {{ .ErrorEncoder }}(encoder func(context.Context, http.ResponseWriter) loom {{- if and .ErrorHeader (not .ErrorNameFromStatus) }} w.Header().Set("loom-error", res.LoomErrorName()) {{- end }} + {{- if and .EncodePlan.HasBody .EncodePlan.FirstBody.Media }} + {{- if .ErrorHeader }} + var enc loomhttp.Encoder + if formatter != nil { + enc = encoder(ctx, w) + } else { + {{- end }} + mediaIndex := -1 + var mediaValue any + var mediaParts map[string]string + switch mediaBody := any(body).(type) { + {{- range .ServerBody }} + case {{ .Ref }}: + switch string(mediaBody.Kind()) { + {{- range $i, $variant := .Media.Variants }} + case {{ printf "%q" .Tag }}: + mediaIndex, mediaValue, mediaParts = {{ $i }}, mediaBody.{{ .Field }}, {{ .Parts }} + {{- end }} + } + {{- end }} + } + mediaEncoder, mediaErr := loomhttp.MediaResponseEncoder(ctx, w, mediaIndex, {{ .EncodePlan.FirstBody.Media.Contracts }}, mediaValue, mediaParts, encoder) + if mediaErr != nil { + return mediaErr + } + enc {{ if .ErrorHeader }}={{ else }}:={{ end }} mediaEncoder + {{- if .ErrorHeader }} + } + {{- end }} + {{- end }} {{- if not .DeferStatus }} w.WriteHeader({{ .StatusCode }}) {{- end }}`}, diff --git a/http/codegen/template_sources_response_decoder.go b/http/codegen/template_sources_response_decoder.go index df638635a..4711bf69b 100644 --- a/http/codegen/template_sources_response_decoder.go +++ b/http/codegen/template_sources_response_decoder.go @@ -146,7 +146,27 @@ func {{ .ResponseDecoder }}(decoder func(*http.Response) loomhttp.Decoder, resto body {{ .ClientBody.ValueRef }} err error ) + {{- if .ClientBody.Media }} + { + mediaIndex, mediaErr := loomhttp.SelectMediaType(resp.Header.Get("Content-Type"), {{ .ClientBody.Media.Contracts }}) + if mediaErr != nil { + err = mediaErr + } else { + switch mediaIndex { + {{- range $i, $variant := .ClientBody.Media.Variants }} + case {{ $i }}: + var mediaValue {{ .TypeRef }} + err = loomhttp.DecodeMediaResponse(resp, &mediaValue, {{ .AllowsNull }}, {{ .Parts }}, decoder) + if err == nil { + body.Set{{ .Field }}(mediaValue) + } + {{- end }} + } + } + } + {{- else }} err = decoder(resp).Decode(&body) + {{- end }} if err != nil { return nil, loomhttp.ErrDecodingError("{{ $.ServiceName }}", "{{ $.Method.Name }}", err) } diff --git a/http/codegen/testdata/media_contract_harness.go.txt b/http/codegen/testdata/media_contract_harness.go.txt new file mode 100644 index 000000000..8f51020bd --- /dev/null +++ b/http/codegen/testdata/media_contract_harness.go.txt @@ -0,0 +1,184 @@ +package mediacontracts_test + +import ( + "context" + "encoding/json/jsontext" + "encoding/json/v2" + "io" + "net/http" + "net/http/httptest" + "net/url" + "strings" + "sync/atomic" + "testing" + + "github.com/stretchr/testify/require" + + client "example.com/mediacontracts/gen/http/media_contracts/client" + server "example.com/mediacontracts/gen/http/media_contracts/server" + service "example.com/mediacontracts/gen/media_contracts" + loomhttp "github.com/CaliLuke/loom/http" +) + +func TestMediaExchange(t *testing.T) { + var calls atomic.Int32 + mux := loomhttp.NewMuxer() + endpoint := func(_ context.Context, value any) (any, error) { + calls.Add(1) + return value, nil + } + server.Mount(mux, server.New(&service.Endpoints{Exchange: endpoint}, mux, loomhttp.RequestDecoder, loomhttp.ResponseEncoder, nil, nil)) + remote := httptest.NewServer(mux) + defer remote.Close() + parsed, err := url.Parse(remote.URL) + require.NoError(t, err) + transport := client.NewClient(parsed.Scheme, parsed.Host, remote.Client(), loomhttp.RequestEncoder, loomhttp.ResponseDecoder, false) + binaryType := "application/octet-stream" + note := "hello" + var nullable service.Body + require.NoError(t, json.Unmarshal([]byte(`{"type":"nullable","value":null}`), &nullable)) + for _, tc := range []struct { + name string + body service.Body + media *string + }{ + {"json", service.NewBodyJSON(42), nil}, + {"vendor", service.NewBodyVendor(7), nil}, + {"nullable", nullable, nil}, + {"text", service.NewBodyText("hello"), nil}, + {"binary", service.NewBodyBinary([]byte{0, 255}), &binaryType}, + {"form", service.NewBodyForm(&service.BodyForm{Name: ¬e, Note: ¬e, Numbers: []int{1, 2}, Labels: map[string]string{"a": "b"}}), nil}, + {"empty form", service.NewBodyForm(&service.BodyForm{}), nil}, + {"multipart", service.NewBodyMultipart(&service.BodyMultipart{Name: "file", Data: []byte{0, 255}}), nil}, + } { + t.Run(tc.name, func(t *testing.T) { + got, err := transport.Exchange()(context.Background(), &service.MediaEnvelope{Body: tc.body, ContentType: tc.media}) + require.NoError(t, err) + result := got.(*service.MediaEnvelope) + require.Equal(t, tc.body.Kind(), result.Body.Kind()) + switch tc.name { + case "json": + require.EqualValues(t, 42, result.Body.JSON) + case "vendor": + require.EqualValues(t, 7, result.Body.Vendor) + case "text": + require.EqualValues(t, "hello", result.Body.Text) + case "binary": + require.Equal(t, service.BodyBinary{0, 255}, result.Body.Binary) + case "nullable": + require.True(t, result.Body.Nullable.IsNull()) + case "form", "empty form": + require.Equal(t, tc.body.Form, result.Body.Form) + case "multipart": + require.Equal(t, tc.body.Multipart, result.Body.Multipart) + } + }) + } + require.EqualValues(t, 8, calls.Load()) + for _, tc := range []struct { + name, media, body string + status int + }{ + {"missing media", "", "42", 415}, + {"malformed media", "invalid", "42", 415}, + {"unsupported codec", "image/png", "data", 415}, + {"wrong JSON shape", "application/json", `"text"`, 400}, + {"empty JSON", "application/json", "", 400}, + {"whitespace JSON", "application/json", " \n\t", 400}, + {"null JSON", "application/json", "null", 400}, + {"truncated JSON", "application/json", "{", 400}, + {"constraint", "application/json", "101", 400}, + } { + t.Run(tc.name, func(t *testing.T) { + request, err := http.NewRequest("POST", remote.URL+"/media-contracts", strings.NewReader(tc.body)) + require.NoError(t, err) + request.Header.Set("Content-Type", tc.media) + response, err := remote.Client().Do(request) + require.NoError(t, err) + data, err := io.ReadAll(response.Body) + require.NoError(t, err) + require.NoError(t, response.Body.Close()) + require.Equal(t, tc.status, response.StatusCode, string(data)) + var problem map[string]any + require.NoError(t, json.Unmarshal(data, &problem)) + wantCode := "decode_payload" + switch { + case tc.status == 415: + wantCode = "unsupported_media_type" + case tc.name == "empty JSON" || tc.name == "whitespace JSON": + wantCode = "missing_payload" + case tc.name == "constraint": + wantCode = "invalid_range" + } + require.Equal(t, wantCode, problem["code"]) + require.NotEmpty(t, problem["detail"]) + require.EqualValues(t, 8, calls.Load()) + }) + } + wrongType := "text/plain" + _, err = transport.Exchange()(context.Background(), &service.MediaEnvelope{Body: service.NewBodyJSON(1), ContentType: &wrongType}) + require.Error(t, err) + require.EqualValues(t, 8, calls.Load()) +} + +func TestOptionalMediaBody(t *testing.T) { + mux := loomhttp.NewMuxer() + var calls atomic.Int32 + var present atomic.Bool + endpoint := func(_ context.Context, value any) (any, error) { + calls.Add(1) + present.Store(value.(*service.OptionalPayload).Choice != nil) + return "ok", nil + } + server.Mount(mux, server.New(&service.Endpoints{Optional: endpoint}, mux, loomhttp.RequestDecoder, loomhttp.ResponseEncoder, nil, nil)) + for _, tc := range []struct { + name, media, body string + status int + present bool + }{ + {"absent", "", "", 200, false}, + {"empty JSON", "application/json", "", 200, false}, + {"zero JSON", "application/json", "0", 200, true}, + {"empty text", "text/plain", "", 200, true}, + {"empty form", "application/x-www-form-urlencoded", "", 200, true}, + {"untyped body", "", "0", 415, false}, + {"null", "application/json", "null", 400, false}, + {"truncated", "application/json", "{", 400, false}, + } { + t.Run(tc.name, func(t *testing.T) { + before := calls.Load() + request := httptest.NewRequest("POST", "/media-contracts/optional", strings.NewReader(tc.body)) + request.Header.Set("Content-Type", tc.media) + writer := httptest.NewRecorder() + mux.ServeHTTP(writer, request) + require.Equal(t, tc.status, writer.Code, writer.Body.String()) + if tc.status == 200 { + require.Equal(t, before+1, calls.Load()) + require.Equal(t, tc.present, present.Load()) + } else { + require.Equal(t, before, calls.Load()) + } + }) + } +} + +func TestMediaErrors(t *testing.T) { + failure := &service.MediaFailure{Detail: service.NewDetailText("rejected")} + for _, formatted := range []bool{false, true} { + writer := httptest.NewRecorder() + var formatter func(context.Context, error) loomhttp.StatusCoder + if formatted { + formatter = loomhttp.NewErrorResponse + } + encode := server.EncodeFailError(loomhttp.ResponseEncoder, formatter) + require.NoError(t, encode(context.Background(), writer, failure)) + require.Equal(t, 400, writer.Code) + if formatted { + require.Contains(t, writer.Header().Get("Content-Type"), "json") + require.True(t, jsontext.Value(writer.Body.Bytes()).IsValid()) + } else { + require.Equal(t, "text/plain", writer.Header().Get("Content-Type")) + require.Equal(t, "rejected", writer.Body.String()) + } + } +} diff --git a/http/codegen/testdata/openapi_dsls_22.go b/http/codegen/testdata/openapi_dsls_22.go index a5bdbd590..8f983deab 100644 --- a/http/codegen/testdata/openapi_dsls_22.go +++ b/http/codegen/testdata/openapi_dsls_22.go @@ -87,3 +87,177 @@ var StreamingPartialExamplesDSL = func() { }) }) } + +var MultipleSchemaExamplesDSL = func() { + StreamingPartialExamplesDSL() + colors := Type("ExampleColors", String, func() { + Nullable() + for _, example := range []struct { + name string + value any + }{{"empty", Null()}, {"red", "red"}, {"blue", "blue"}, {"blue-again", "blue"}} { + Example(example.name, func() { + Meta("openapi:example:scope", "schema") + Value(example.value) + }) + } + Example("media", func() { + Meta("openapi:example:scope", "media") + Value("green") + }) + }) + Service("schemaExamples", func() { + Method("echo", func() { + Payload(func() { + Attribute("color", colors) + Attribute("bytes", Bytes, func() { + Example("first", []byte("a")) + Example("second", []byte("b")) + }) + }) + Result(colors) + HTTP(func() { + POST("/schema-examples") + Response(StatusOK) + }) + }) + }) +} + +var MultipleMediaContractsDSL = func() { + StreamingPartialExamplesDSL() + envelope := Type("MediaEnvelope", func() { + Attribute("content_type", String) + OneOf("body", func() { + Attribute("json", Int, func() { + Minimum(0) + Maximum(100) + Example("one", 1) + Example("two", 2) + }) + Attribute("nullable", String, func() { + Nullable() + Example(Null()) + }) + Attribute("vendor", Int, func() { + Example("vendor", 7) + }) + Attribute("text", String, func() { + Example("text", "hello") + }) + Attribute("binary", Bytes, func() { + Example("binary", []byte("bytes")) + }) + Attribute("form", func() { + Attribute("name", String) + Attribute("note", String) + Attribute("numbers", ArrayOf(Int)) + Attribute("labels", MapOf(String, String)) + Meta("openapi:encoding:labels:contentType", "application/json") + }) + Attribute("multipart", func() { + Attribute("name", String) + Attribute("data", Bytes) + Required("name", "data") + Meta("openapi:encoding:data:contentType", "image/png") + }) + }) + Required("body") + }) + payload := Type("MediaRootExamples", func() { + OneOf("body", func() { + Attribute("number", Int) + Attribute("text", String) + }) + Required("body") + Example("first", map[string]any{"body": 11}) + Example("second", map[string]any{"body": 22}) + Example("text", map[string]any{"body": "hello"}) + }) + failure := Type("MediaFailure", func() { + OneOf("detail", func() { + Attribute("number", Int) + Attribute("text", String) + }) + Required("detail") + }) + Service("mediaContracts", func() { + Method("fail", func() { + Error("rejected", failure) + HTTP(func() { + GET("/media-contracts/fail") + Response("rejected", StatusBadRequest, func() { + Body("detail") + MediaType("application/json", "number") + MediaType("text/plain", "text") + }) + }) + }) + Method("optional", func() { + Payload(func() { + OneOf("choice", func() { + Attribute("number", Int) + Attribute("text", String) + Attribute("form", func() { + Attribute("name", String) + }) + }) + }) + Result(String) + HTTP(func() { + POST("/media-contracts/optional") + Body("choice") + MediaType("application/x-www-form-urlencoded", "form") + MediaType("application/json", "number") + MediaType("text/plain", "text") + Response(StatusOK) + }) + }) + + Method("exchange", func() { + Payload(envelope) + Result(envelope) + HTTP(func() { + POST("/media-contracts") + Body("body") + Header("content_type:Content-Type") + MediaType("application/json", "json") + MediaType("application/vnd.loom+json", "vendor") + MediaType("application/vnd.null+json", "nullable") + MediaType("text/plain", "text") + MediaType("*/*", "binary") + MediaType("application/x-www-form-urlencoded", "form") + MediaType("multipart/form-data", "multipart") + Response(StatusOK, func() { + Body("body") + Header("content_type:Content-Type") + MediaType("application/json", "json") + MediaType("application/vnd.loom+json", "vendor") + MediaType("application/vnd.null+json", "nullable") + MediaType("text/plain", "text") + MediaType("*/*", "binary") + MediaType("application/x-www-form-urlencoded", "form") + MediaType("multipart/form-data", "multipart") + }) + }) + }) + }) + Service("mediaExampleSelections", func() { + Method("rootExamples", func() { + + Payload(payload) + Result(payload) + HTTP(func() { + POST("/media-root-examples") + Body("body") + MediaType("application/json", "number") + MediaType("text/plain", "text") + Response(StatusOK, func() { + Body("body") + MediaType("application/json", "number") + MediaType("text/plain", "text") + }) + }) + }) + }) +} diff --git a/http/encoding.go b/http/encoding.go index a8c8e5607..6089a9547 100644 --- a/http/encoding.go +++ b/http/encoding.go @@ -85,6 +85,9 @@ const ( // RequestDecoder defaults to JSON when Content-Type is missing and rejects // unsupported media types. func RequestDecoder(r *http.Request) Decoder { + if mediaContractContext(r.Context()) { + return mediaRequestDecoder(r) + } limit := RequestBodyLimit(r.Context()) selected := httpcodec.RequestContentType(r.Header.Get("Content-Type")) switch selected.Kind { @@ -118,6 +121,9 @@ func RequestDecoder(r *http.Request) Decoder { // ContentTypeKey value does not match any of the supported mime types or is // missing altogether. func ResponseEncoder(ctx context.Context, w http.ResponseWriter) Encoder { + if mediaContractContext(ctx) { + return mediaResponseEncoder(ctx, w) + } accept := stringContextValue(ctx, AcceptTypeKey) ct := stringContextValue(ctx, ContentTypeKey) if ct != "" { @@ -167,6 +173,9 @@ func selectedResponseEncoder(w http.ResponseWriter, selected httpcodec.Selection // RequestEncoder returns a HTTP request encoder. // The encoder uses package encoding/json/v2. func RequestEncoder(r *http.Request) Encoder { + if mediaContractContext(r.Context()) { + return mediaRequestEncoder(r) + } const k = "Content-Type" if h := r.Header.Get(k); h == "" { r.Header.Set(k, "application/json") @@ -247,6 +256,10 @@ func (je *jsonEncoder) GetBody() (io.ReadCloser, error) { // - application/gob using package encoding/gob // - text/html and text/plain for strings func ResponseDecoder(resp *http.Response) Decoder { + if resp.Request != nil && mediaContractContext(resp.Request.Context()) { + rejectNull, _ := resp.Request.Context().Value(nonNullableBodyKey{}).(bool) + return mediaDecoder(resp.Body, resp.Header.Get("Content-Type"), decodeResponse, DefaultMaxRequestBodyBytes, !rejectNull) + } selected := httpcodec.ResponseDecode(resp.Header.Get("Content-Type")) switch selected.Kind { case httpcodec.XML: diff --git a/http/integration_tests/fixtures/ticktock/gen/loom.json b/http/integration_tests/fixtures/ticktock/gen/loom.json index 964a0b176..127f13447 100644 --- a/http/integration_tests/fixtures/ticktock/gen/loom.json +++ b/http/integration_tests/fixtures/ticktock/gen/loom.json @@ -1,5 +1,5 @@ { "loom_version": "(devel)", "design_package": "example.com/http-ticktock/design", - "design_digest": "a6dba4ef0097bbba8876f37f2226874ba8576b4f3f1ffea5b4a89a15b37cd081" + "design_digest": "f0eef1964eb0e0df1aa6d38aa037a819bd6aeaf29e8c8eba88c6cc9611847016" } diff --git a/http/media_codec.go b/http/media_codec.go new file mode 100644 index 000000000..16058dd5b --- /dev/null +++ b/http/media_codec.go @@ -0,0 +1,243 @@ +package http + +import ( + "bytes" + "context" + "fmt" + "io" + "maps" + "mime" + "mime/multipart" + "net/http" + "net/textproto" + "net/url" + "reflect" + "slices" + + "github.com/CaliLuke/loom/internal/httpcodec" + loom "github.com/CaliLuke/loom/pkg" +) + +type mediaCodecError struct{ err error } + +func (e *mediaCodecError) Encode(any) error { + return e.err +} +func (e *mediaCodecError) Decode(any) error { + return e.err +} + +func mediaRequestDecoder(request *http.Request) Decoder { + rejectNull, _ := request.Context().Value(nonNullableBodyKey{}).(bool) + return mediaDecoder(request.Body, request.Header.Get("Content-Type"), decodeRequest, RequestBodyLimit(request.Context()), !rejectNull) +} + +func mediaDecoder(reader io.Reader, media string, mode decodeMode, limit int64, allowsNull bool) Decoder { + base, _, err := mime.ParseMediaType(media) + if err != nil { + return newUnsupportedDecoder(media) + } + switch base { + case "application/octet-stream": + return mediaRawDecoder(reader, media, mode, limit, false) + case "application/x-www-form-urlencoded": + return newLimitedDecoder(reader, func(data []byte, target any) error { + values, err := url.ParseQuery(string(data)) + if err != nil { + return err + } + return decodeMediaFormValues(values, target) + }, mode, limit) + case "multipart/form-data": + return EncodingFunc(func(target any) error { + request := &http.Request{Header: http.Header{"Content-Type": {media}}, Body: io.NopCloser(reader)} + form, err := ReadMultipartFormWithLimit(request, limit) + if err != nil { + return requestBodyDecodeError(err, mode) + } + for name, files := range form.Files { + if len(files) != 1 { + return fmt.Errorf("multiple multipart files provided for field %s", name) + } + if _, exists := form.Values[name]; exists { + return fmt.Errorf("duplicate multipart field %s", name) + } + form.Values.Set(name, string(files[0].Data)) + } + return decodeMediaFormValues(form.Values, target) + }) + } + switch httpcodec.RequestContentType(media).Kind { + case httpcodec.JSON: + decode := decodeJSON + if !allowsNull { + decode = decodeNonNullableJSON + } + return newLimitedDecoder(reader, decode, mode, limit) + case httpcodec.Text: + return mediaRawDecoder(reader, media, mode, limit, true) + default: + return newUnsupportedDecoder(media) + } +} + +func mediaRawDecoder(reader io.Reader, media string, mode decodeMode, limit int64, text bool) Decoder { + return EncodingFunc(func(target any) error { + value := reflect.ValueOf(target) + if value.Kind() != reflect.Pointer || value.IsNil() { + return fmt.Errorf("cannot decode %s into %T", media, target) + } + value = value.Elem() + data, err := readAllLimited(reader, limit) + if err != nil { + return requestBodyDecodeError(err, mode) + } + switch { + case text && value.Kind() == reflect.String: + value.SetString(string(data)) + case value.Kind() == reflect.Slice && value.Type().Elem().Kind() == reflect.Uint8: + value.SetBytes(data) + default: + return fmt.Errorf("cannot decode %s into %T", media, target) + } + return nil + }) +} + +func mediaRequestEncoder(request *http.Request) Encoder { + return EncodingFunc(func(value any) error { + var buffer bytes.Buffer + encoder, media, err := mediaEncoder(&buffer, request.Header.Get("Content-Type"), mediaPartTypes(request.Context())) + if err != nil { + return err + } + if err = encoder.Encode(value); err != nil { + return err + } + data := buffer.Bytes() + request.Header.Set("Content-Type", media) + request.Body = io.NopCloser(bytes.NewReader(data)) + request.ContentLength = int64(len(data)) + request.GetBody = func() (io.ReadCloser, error) { + return io.NopCloser(bytes.NewReader(data)), nil + } + return nil + }) +} + +func mediaResponseEncoder(ctx context.Context, writer http.ResponseWriter) Encoder { + encoder, media, err := mediaEncoder(writer, stringContextValue(ctx, ContentTypeKey), mediaPartTypes(ctx)) + if err != nil { + return &mediaCodecError{err: err} + } + writer.Header().Set("Content-Type", media) + return encoder +} + +func mediaEncoder(writer io.Writer, media string, parts map[string]string) (Encoder, string, error) { + base, params, err := mime.ParseMediaType(media) + if err != nil { + return nil, "", loom.UnsupportedMediaTypeError(media) + } + switch base { + case "application/octet-stream": + return mediaRawEncoder(writer, media, false), media, nil + case "application/x-www-form-urlencoded": + return EncodingFunc(func(value any) error { + values, err := EncodeFormValues(value) + if err != nil { + return err + } + return writeMediaBytes(writer, []byte(values.Encode())) + }), media, nil + case "multipart/form-data": + multipartWriter := multipart.NewWriter(writer) + if boundary := params["boundary"]; boundary != "" { + if err := multipartWriter.SetBoundary(boundary); err != nil { + return nil, "", err + } + } + params["boundary"] = multipartWriter.Boundary() + return EncodingFunc(func(value any) error { + values, err := EncodeFormValues(value) + if err != nil { + return err + } + for _, name := range slices.Sorted(maps.Keys(values)) { + for _, field := range values[name] { + partType := parts[name] + if partType == "" { + partType = "text/plain" + } + header := textproto.MIMEHeader{} + header.Set("Content-Disposition", mime.FormatMediaType("form-data", map[string]string{"name": name})) + header.Set("Content-Type", partType) + part, err := multipartWriter.CreatePart(header) + if err != nil { + return err + } + if err := writeMediaBytes(part, []byte(field)); err != nil { + return err + } + } + } + return multipartWriter.Close() + }), mime.FormatMediaType(base, params), nil + } + switch httpcodec.RequestContentType(media).Kind { + case httpcodec.JSON: + return newJSONResponseEncoder(writer), media, nil + case httpcodec.Text: + return mediaRawEncoder(writer, media, true), media, nil + default: + return nil, "", loom.UnsupportedMediaTypeError(media) + } +} + +func mediaRawEncoder(writer io.Writer, media string, text bool) Encoder { + return EncodingFunc(func(input any) error { + value := reflect.ValueOf(input) + for value.Kind() == reflect.Pointer { + if value.IsNil() { + return fmt.Errorf("cannot encode nil as %s", media) + } + value = value.Elem() + } + switch { + case text && value.Kind() == reflect.String: + return writeMediaBytes(writer, []byte(value.String())) + case value.Kind() == reflect.Slice && value.Type().Elem().Kind() == reflect.Uint8: + return writeMediaBytes(writer, value.Bytes()) + default: + return fmt.Errorf("cannot encode %T as %s", input, media) + } + }) +} + +func writeMediaBytes(writer io.Writer, data []byte) error { + written, err := writer.Write(data) + if err != nil { + return err + } + if written != len(data) { + return io.ErrShortWrite + } + return nil +} + +// A selected form representation supplies an object even when all of its +// optional fields are absent. Keep this allocation separate from ordinary +// form decoding, where an absent root may still be represented by nil. +func decodeMediaFormValues(values url.Values, target any) error { + value := reflect.ValueOf(target) + if value.Kind() != reflect.Pointer || value.IsNil() { + return fmt.Errorf("media form decode target must be a non-nil pointer") + } + for value.Kind() == reflect.Pointer { + if value.IsNil() { + value.Set(reflect.New(value.Type().Elem())) + } + value = value.Elem() + } + return DecodeFormValues(values, target) +} diff --git a/http/media_contract.go b/http/media_contract.go new file mode 100644 index 000000000..f2f4070a4 --- /dev/null +++ b/http/media_contract.go @@ -0,0 +1,133 @@ +package http + +import ( + "context" + "fmt" + "io" + "net/http" + + "github.com/CaliLuke/loom/internal/httpcodec" + loom "github.com/CaliLuke/loom/pkg" +) + +type mediaContractKey struct{} +type mediaPartsKey struct{} + +// SelectMediaType returns the most specific matching media contract index. +// Missing, malformed, wildcard and unmatched Content-Type values return a +// service error suitable for an HTTP 415 response. Contract parameters must +// match; additional incoming parameters are allowed. +func SelectMediaType(contentType string, contracts []string) (int, error) { + index, err := httpcodec.SelectMediaContract(contracts, contentType) + if err != nil { + return -1, loom.UnsupportedMediaTypeError(contentType) + } + return index, nil +} + +// SelectRequestMediaType selects the request contract. An optional absent body +// needs no Content-Type; a nonempty body must always identify its representation. +func SelectRequestMediaType(request *http.Request, contracts []string, optional bool) (int, error) { + if optional && request.Header.Get("Content-Type") == "" { + if request.Body == nil { + return -1, io.EOF + } + var first [1]byte + _, err := io.ReadFull(request.Body, first[:]) + if err == io.EOF { + return -1, io.EOF + } + if err != nil { + return -1, err + } + } + return SelectMediaType(request.Header.Get("Content-Type"), contracts) +} + +// EncodeMediaRequest publishes one selected typed representation through the +// supplied factory. A mapped Content-Type may refine the selection but must +// resolve to the same branch. Ranges require that concrete mapped value. +func EncodeMediaRequest(request *http.Request, index int, contracts []string, value any, parts map[string]string, factory func(*http.Request) Encoder) error { + media, err := outgoingMediaType(request.Header.Get("Content-Type"), index, contracts) + if err != nil { + return err + } + request.Header.Set("Content-Type", media) + // Preserve the request identity: encoder factories install its Body/GetBody. + ctx := context.WithValue(request.Context(), mediaContractKey{}, true) + ctx = context.WithValue(ctx, mediaPartsKey{}, parts) + *request = *request.WithContext(ctx) + return factory(request).Encode(value) +} + +// DecodeMediaRequest invokes the supplied factory for an already selected typed +// branch. The branch's null contract replaces the enclosing union's null policy. +func DecodeMediaRequest(request *http.Request, target any, allowsNull bool, parts map[string]string, factory func(*http.Request) Decoder) error { + ctx := context.WithValue(request.Context(), mediaContractKey{}, true) + ctx = context.WithValue(ctx, nonNullableBodyKey{}, !allowsNull) + ctx = context.WithValue(ctx, mediaPartsKey{}, parts) + return factory(request.WithContext(ctx)).Decode(target) +} + +// MediaResponseEncoder selects a response representation before status commitment. +// Its returned encoder writes the selected value and ignores its Encode argument. +// The service chooses the variant; this function does not negotiate Accept. +func MediaResponseEncoder(ctx context.Context, writer http.ResponseWriter, index int, contracts []string, value any, parts map[string]string, factory func(context.Context, http.ResponseWriter) Encoder) (Encoder, error) { + media, err := outgoingMediaType(writer.Header().Get("Content-Type"), index, contracts) + if err != nil { + return nil, err + } + ctx = context.WithValue(ctx, ContentTypeKey, media) + ctx = context.WithValue(ctx, mediaContractKey{}, true) + ctx = context.WithValue(ctx, mediaPartsKey{}, parts) + writer.Header().Set("Content-Type", media) + encoder := factory(ctx, writer) + if failure, ok := encoder.(*mediaCodecError); ok { + return nil, failure.err + } + return EncodingFunc(func(any) error { + return encoder.Encode(value) + }), nil +} + +// DecodeMediaResponse invokes the supplied factory for a selected response +// branch. It preserves the response body and metadata and retains request context. +func DecodeMediaResponse(response *http.Response, target any, allowsNull bool, parts map[string]string, factory func(*http.Response) Decoder) error { + copy := *response + request := response.Request + if request == nil { + request = new(http.Request) + } + ctx := context.WithValue(request.Context(), mediaContractKey{}, true) + ctx = context.WithValue(ctx, nonNullableBodyKey{}, !allowsNull) + ctx = context.WithValue(ctx, mediaPartsKey{}, parts) + copy.Request = request.WithContext(ctx) + return factory(©).Decode(target) +} + +func outgoingMediaType(media string, index int, contracts []string) (string, error) { + if index < 0 || index >= len(contracts) { + return "", fmt.Errorf("HTTP media body has no selected variant") + } + if media == "" { + media = contracts[index] + } + selected, err := httpcodec.SelectMediaContract(contracts, media) + if err != nil { + return "", err + } + if selected != index { + return "", fmt.Errorf("Content-Type %q selects a different body variant", media) + } + return media, nil +} + +func mediaContractContext(ctx context.Context) bool { + selected, _ := ctx.Value(mediaContractKey{}).(bool) + return selected +} + +func mediaPartTypes(ctx context.Context) map[string]string { + parts, _ := ctx.Value(mediaPartsKey{}).(map[string]string) + return parts +} diff --git a/http/media_contract_test.go b/http/media_contract_test.go new file mode 100644 index 000000000..bd1e9a057 --- /dev/null +++ b/http/media_contract_test.go @@ -0,0 +1,209 @@ +package http + +import ( + "bytes" + "context" + "io" + "mime" + "mime/multipart" + stdhttp "net/http" + "net/http/httptest" + "testing" + + "github.com/stretchr/testify/require" +) + +func TestMediaContractFactories(t *testing.T) { + contracts := []string{"application/json", "text/plain", "*/*"} + for _, tc := range []struct { + name, media string + index int + value any + rejected bool + }{ + {"json", "", 0, 42, false}, {"text", "", 1, "hello", false}, + {"range", "application/octet-stream", 2, []byte("bytes"), false}, + {"mismatch", "text/plain", 0, 42, true}, {"missing range", "", 2, []byte("bytes"), true}, + } { + t.Run(tc.name, func(t *testing.T) { + request := httptest.NewRequest("POST", "/", nil) + if tc.media != "" { + request.Header.Set("Content-Type", tc.media) + } + called := false + err := EncodeMediaRequest(request, tc.index, contracts, tc.value, nil, func(r *stdhttp.Request) Encoder { + called = true + return EncodingFunc(func(value any) error { + require.Equal(t, tc.value, value) + return nil + }) + }) + if tc.rejected { + require.Error(t, err) + require.False(t, called) + return + } + require.NoError(t, err) + require.True(t, called) + selected, err := SelectMediaType(request.Header.Get("Content-Type"), contracts) + require.NoError(t, err) + require.Equal(t, tc.index, selected) + }) + } +} + +func TestMediaContractBuiltinCodecs(t *testing.T) { + contracts := []string{"application/json", "text/plain", "application/octet-stream"} + for _, tc := range []struct { + name string + index int + value any + target func() any + }{ + {"json", 0, 42, func() any { + return new(int) + }}, + {"text", 1, "hello", func() any { + return new(string) + }}, + {"binary", 2, []byte{0, 255}, func() any { + return new([]byte) + }}, + } { + t.Run(tc.name, func(t *testing.T) { + request := httptest.NewRequest("POST", "/", nil) + require.NoError(t, EncodeMediaRequest(request, tc.index, contracts, tc.value, nil, RequestEncoder)) + data, err := io.ReadAll(request.Body) + require.NoError(t, err) + incoming := httptest.NewRequest("POST", "/", bytes.NewReader(data)) + incoming.Header.Set("Content-Type", request.Header.Get("Content-Type")) + target := tc.target() + require.NoError(t, DecodeMediaRequest(incoming, target, false, nil, RequestDecoder)) + writer := httptest.NewRecorder() + encoder, err := MediaResponseEncoder(context.Background(), writer, tc.index, contracts, tc.value, nil, ResponseEncoder) + require.NoError(t, err) + writer.WriteHeader(201) + require.NoError(t, encoder.Encode(nil)) + response := writer.Result() + target = tc.target() + require.NoError(t, DecodeMediaResponse(response, target, false, nil, ResponseDecoder)) + switch got := target.(type) { + case *int: + require.Equal(t, tc.value, *got) + case *string: + require.Equal(t, tc.value, *got) + case *[]byte: + require.Equal(t, tc.value, *got) + } + }) + } +} + +func TestMediaJSONNullAndMalformed(t *testing.T) { + for _, tc := range []struct { + name, body string + nullable, valid bool + }{ + {"zero", "0", false, true}, {"null rejected", "null", false, false}, + {"null allowed", "null", true, true}, {"empty", "", false, false}, + {"whitespace", " \n\t", false, false}, {"malformed", "!", false, false}, + {"truncated", "{", false, false}, + } { + t.Run(tc.name, func(t *testing.T) { + request := httptest.NewRequest("POST", "/", bytes.NewBufferString(tc.body)) + request.Header.Set("Content-Type", "application/json") + var value *int + err := DecodeMediaRequest(request, &value, tc.nullable, nil, RequestDecoder) + require.Equal(t, tc.valid, err == nil, "%v", err) + response := &stdhttp.Response{Header: stdhttp.Header{"Content-Type": {"application/json"}}, Body: io.NopCloser(bytes.NewBufferString(tc.body))} + value = nil + err = DecodeMediaResponse(response, &value, tc.nullable, nil, ResponseDecoder) + require.Equal(t, tc.valid, err == nil, "%v", err) + }) + } +} + +func TestMediaResponseFactorySelection(t *testing.T) { + for _, tc := range []struct { + name, header string + index int + rejected bool + }{ + {"exact", "", 0, false}, {"range", "image/png", 1, false}, + {"different branch", "image/png", 0, true}, {"unselected", "", -1, true}, + } { + t.Run(tc.name, func(t *testing.T) { + writer := httptest.NewRecorder() + writer.Header().Set("Content-Type", tc.header) + called := false + encoder, err := MediaResponseEncoder(context.Background(), writer, tc.index, []string{"text/plain", "*/*"}, "selected", nil, func(ctx context.Context, w stdhttp.ResponseWriter) Encoder { + called = true + require.Equal(t, w.Header().Get("Content-Type"), ctx.Value(ContentTypeKey)) + require.Empty(t, writer.Body.String()) + return EncodingFunc(func(value any) error { + require.Equal(t, "selected", value) + return nil + }) + }) + require.Equal(t, !tc.rejected, called) + if tc.rejected { + require.Error(t, err) + require.Nil(t, encoder) + } else { + require.NoError(t, err) + require.NoError(t, encoder.Encode("ignored")) + } + }) + } +} + +func TestMediaMultipartPartTypes(t *testing.T) { + value := struct { + Data []byte `form:"data"` + }{Data: []byte{0, 255}} + request := httptest.NewRequest("POST", "/", nil) + require.NoError(t, EncodeMediaRequest(request, 0, []string{"multipart/form-data"}, value, map[string]string{"data": "image/png"}, RequestEncoder)) + _, params, err := mime.ParseMediaType(request.Header.Get("Content-Type")) + require.NoError(t, err) + reader := multipart.NewReader(request.Body, params["boundary"]) + part, err := reader.NextPart() + require.NoError(t, err) + require.Equal(t, "data", part.FormName()) + require.Equal(t, "image/png", part.Header.Get("Content-Type")) + data, err := io.ReadAll(part) + require.NoError(t, err) + require.Equal(t, value.Data, data) + require.NoError(t, part.Close()) +} + +func TestMediaCustomCodecsHaveNoFallback(t *testing.T) { + for _, media := range []string{"application/xml", "application/gob", "image/png"} { + t.Run(media, func(t *testing.T) { + request := httptest.NewRequest("POST", "/", nil) + require.Error(t, EncodeMediaRequest(request, 0, []string{media}, "value", nil, RequestEncoder)) + request = httptest.NewRequest("POST", "/", bytes.NewBufferString(`"value"`)) + request.Header.Set("Content-Type", media) + var value string + require.Error(t, DecodeMediaRequest(request, &value, false, nil, RequestDecoder)) + writer := httptest.NewRecorder() + _, err := MediaResponseEncoder(context.Background(), writer, 0, []string{media}, "value", nil, ResponseEncoder) + require.Error(t, err) + require.Empty(t, writer.Body.String()) + }) + } +} + +func TestMediaEmptyFormRetainsObject(t *testing.T) { + type form struct { + Name *string `form:"name"` + } + request := httptest.NewRequest("POST", "/", nil) + request.Header.Set("Content-Type", "application/x-www-form-urlencoded") + var decoded *form + require.NoError(t, DecodeMediaRequest(request, &decoded, false, nil, RequestDecoder)) + require.Equal(t, &form{}, decoded) + response := &stdhttp.Response{Header: request.Header, Body: stdhttp.NoBody} + decoded = nil + require.NoError(t, DecodeMediaResponse(response, &decoded, false, nil, ResponseDecoder)) + require.Equal(t, &form{}, decoded) +} diff --git a/internal/httpcodec/media_contract.go b/internal/httpcodec/media_contract.go new file mode 100644 index 000000000..72e30c2d7 --- /dev/null +++ b/internal/httpcodec/media_contract.go @@ -0,0 +1,110 @@ +package httpcodec + +import ( + "fmt" + "mime" + "strings" +) + +type mediaContract struct { + base string + major string + params map[string]string + rank int +} + +// ValidateMediaContracts rejects malformed, duplicate and equally specific +// overlapping contracts. Exact types outrank subtype ranges, which outrank */*. +func ValidateMediaContracts(contracts []string) error { + parsed := make([]mediaContract, len(contracts)) + for i, value := range contracts { + item, err := parseMediaContract(value, true) + if err != nil { + return err + } + for j := range i { + other := parsed[j] + if item.base == other.base && len(item.params) == len(other.params) && compatibleMediaParameters(item.params, other.params) { + return fmt.Errorf("ambiguous media contracts %q and %q", contracts[j], value) + } + } + parsed[i] = item + } + return nil +} + +// SelectMediaContract selects the most specific matching contract for a concrete +// Content-Type. Parameters in the contract must match the received parameters; +// additional received parameters do not prevent a match. Missing, malformed, +// ambiguous and unmatched content types return an error. +func SelectMediaContract(contracts []string, contentType string) (int, error) { + received, err := parseMediaContract(contentType, false) + if err != nil { + return -1, err + } + selected, bestRank, bestParams := -1, -1, -1 + for i, value := range contracts { + candidate, err := parseMediaContract(value, true) + if err != nil { + return -1, err + } + if candidate.base != received.base && candidate.base != "*/*" && candidate.base != received.major+"/*" { + continue + } + matches := true + for key, value := range candidate.params { + if receivedValue, exists := received.params[key]; !exists || receivedValue != value { + matches = false + break + } + } + if !matches { + continue + } + if candidate.rank < bestRank || candidate.rank == bestRank && len(candidate.params) < bestParams { + continue + } + if candidate.rank == bestRank && len(candidate.params) == bestParams { + return -1, fmt.Errorf("ambiguous media contract for %q", contentType) + } + selected, bestRank, bestParams = i, candidate.rank, len(candidate.params) + } + if selected < 0 { + return -1, fmt.Errorf("no media contract for %q", contentType) + } + return selected, nil +} + +func parseMediaContract(value string, allowRange bool) (mediaContract, error) { + base, params, err := mime.ParseMediaType(value) + major, minor, ok := strings.Cut(base, "/") + if err != nil || !ok || major == "" || minor == "" || strings.Contains(minor, "/") { + return mediaContract{}, fmt.Errorf("invalid media type %q", value) + } + if charset, exists := params["charset"]; exists { + params["charset"] = strings.ToLower(charset) + } + rank := 3 + if strings.Contains(base, "*") { + switch { + case !allowRange: + return mediaContract{}, fmt.Errorf("Content-Type must be concrete: %q", value) + case base == "*/*": + rank = 1 + case major != "*" && !strings.Contains(major, "*") && minor == "*": + rank = 2 + default: + return mediaContract{}, fmt.Errorf("invalid media range %q", value) + } + } + return mediaContract{base: base, major: major, params: params, rank: rank}, nil +} + +func compatibleMediaParameters(a, b map[string]string) bool { + for key, value := range a { + if other, exists := b[key]; exists && other != value { + return false + } + } + return true +} diff --git a/internal/httpcodec/media_contract_test.go b/internal/httpcodec/media_contract_test.go new file mode 100644 index 000000000..700295030 --- /dev/null +++ b/internal/httpcodec/media_contract_test.go @@ -0,0 +1,72 @@ +package httpcodec + +import ( + "testing" + + "github.com/stretchr/testify/require" +) + +func TestSelectMediaContract(t *testing.T) { + contracts := []string{"*/*", "text/*", "text/plain", "application/json", "application/json; profile=v2"} + for _, tc := range []struct { + name, header string + want int + rejected bool + }{ + {"exact", "text/plain; charset=utf-8", 2, false}, + {"subtype", "text/html", 1, false}, + {"fallback", "image/png", 0, false}, + {"json", "APPLICATION/JSON", 3, false}, + {"parameters", "application/json; profile=v2; charset=utf-8", 4, false}, + {"absent", "", 0, true}, + {"wildcard", "text/*", 0, true}, + {"malformed", "invalid", 0, true}, + } { + t.Run(tc.name, func(t *testing.T) { + got, err := SelectMediaContract(contracts, tc.header) + if tc.rejected { + require.Error(t, err) + return + } + require.NoError(t, err) + require.Equal(t, tc.want, got) + }) + } + _, err := SelectMediaContract([]string{"text/plain"}, "application/json") + require.Error(t, err) + _, err = SelectMediaContract([]string{"text/plain; a=1", "text/plain; b=2"}, "text/plain; a=1; b=2") + require.Error(t, err) +} + +func TestValidateMediaContracts(t *testing.T) { + for _, tc := range []struct { + name string + contracts []string + rejected bool + }{ + {"overlap ordered by specificity", []string{"*/*", "text/*", "text/plain"}, false}, + {"parameter specificity", []string{"application/json", "application/json; profile=v2"}, false}, + {"duplicates", []string{"text/plain", "TEXT/PLAIN"}, true}, + {"ambiguous parameters", []string{"text/plain; a=1", "text/plain; b=2"}, true}, + {"different parameters", []string{"text/plain; profile=v1", "text/plain; profile=v2"}, false}, + {"bad range", []string{"*/json"}, true}, + {"partial wildcard", []string{"application/*+json"}, true}, + } { + t.Run(tc.name, func(t *testing.T) { + err := ValidateMediaContracts(tc.contracts) + if tc.rejected { + require.Error(t, err) + } else { + require.NoError(t, err) + } + }) + } +} + +func TestMediaContractEmptyParameterRequiresPresence(t *testing.T) { + _, err := SelectMediaContract([]string{`application/json; profile=""`}, "application/json") + require.Error(t, err) + index, err := SelectMediaContract([]string{`application/json; charset=UTF-8`}, "application/json; charset=utf-8") + require.NoError(t, err) + require.Zero(t, index) +} diff --git a/internal/httpcodec/tla/media_contracts/MediaContracts.tla b/internal/httpcodec/tla/media_contracts/MediaContracts.tla new file mode 100644 index 000000000..845fc17b3 --- /dev/null +++ b/internal/httpcodec/tla/media_contracts/MediaContracts.tla @@ -0,0 +1,20 @@ +---------------------- MODULE MediaContracts ---------------------- +EXTENDS Naturals, FiniteSets, Sequences +CONSTANT Collapse +Contracts == <<"*/*", "text/*", "text/plain", "application/json">> +Headers == {"text/plain", "text/html", "application/json", "image/png"} +Rank(i, h) == IF Contracts[i] = h THEN 3 ELSE + IF Contracts[i] = "text/*" /\ h \in {"text/plain", "text/html"} THEN 2 ELSE + IF Contracts[i] = "*/*" THEN 1 ELSE 0 +Best(h) == CHOOSE i \in 1..Len(Contracts): \A j \in 1..Len(Contracts): Rank(i,h) >= Rank(j,h) +VARIABLE header, selected, phase +vars == <> +Init == header = "" /\ selected = 0 /\ phase = "idle" +Receive == phase = "idle" /\ \E h \in Headers: + header' = h /\ selected' = (IF Collapse THEN 1 ELSE Best(h)) /\ phase' = "decoded" +Reply == phase = "decoded" /\ phase' = "published" /\ UNCHANGED <> +Next == Receive \/ Reply +Specificity == phase # "idle" => selected = Best(header) +Concrete == phase = "published" => header \in Headers +Spec == Init /\ [][Next]_vars +================================================================== diff --git a/internal/httpcodec/tla/media_contracts/PhysicalLayouts.tla b/internal/httpcodec/tla/media_contracts/PhysicalLayouts.tla new file mode 100644 index 000000000..e1c978333 --- /dev/null +++ b/internal/httpcodec/tla/media_contracts/PhysicalLayouts.tla @@ -0,0 +1,17 @@ +---------------------- MODULE PhysicalLayouts ---------------------- +EXTENDS Naturals +CONSTANT IgnoreNative +Kinds == {"json", "form", "nested-json"} +Declared(k) == k # "form" +VARIABLE kind, inherited, converted, phase +vars == <> +Init == kind = "json" /\ inherited = TRUE /\ converted = TRUE /\ phase = "idle" +Select == phase = "idle" /\ \E k \in Kinds: + kind' = k /\ inherited' = TRUE /\ converted' = TRUE /\ phase' = "selected" +Convert == phase = "selected" /\ phase' = "converted" + /\ converted' = (IF IgnoreNative THEN inherited \/ Declared(kind) ELSE Declared(kind)) + /\ UNCHANGED <> +Next == Select \/ Convert +Correspondence == phase = "converted" => converted = Declared(kind) +Spec == Init /\ [][Next]_vars +==================================================================== diff --git a/internal/httpcodec/tla/media_contracts/README.md b/internal/httpcodec/tla/media_contracts/README.md new file mode 100644 index 000000000..00b50811e --- /dev/null +++ b/internal/httpcodec/tla/media_contracts/README.md @@ -0,0 +1,83 @@ +# Media contract selection + +Loom preserves a media map as typed union variants. Incoming Content-Type selects +one branch before decoding. Exact types outrank subtype ranges, which outrank +`*/*`; matching parameter constraints refine equal type ranges. Equally specific +overlapping contracts are rejected. Missing, malformed, wildcard and unmatched +incoming Content-Type values are rejected for media unions. + +Outgoing selection belongs to the service/client caller through the typed union +variant. Exact variants provide their authored media type. A range variant needs +an explicit concrete content type. The selected concrete type must resolve back +to the selected variant, so a broad branch cannot bypass a more specific schema. +No Accept negotiation is added. Schema/example/encoding ownership stays with each +variant. Ordinary single-media endpoints preserve their current behavior. + +Alternatives considered: application-owned raw codecs keep per-media contracts +outside generated validation and leave repeated dispatch glue; choosing a response +from Accept would add negotiation policy independent of the returned typed value. +Typed variants keep the wire choice explicit and reuse existing union validation. + +The bounded model explores exact, subtype and wildcard overlap with four headers. +`checked.cfg` preserves specificity through decode and publication (9 states). +`collapsed.cfg` deliberately chooses the first contract and fails Specificity. +This control models a rejected design, not the current importer's explicit rejection. +It does not prove MIME parsing, parameter admission, codec correctness or generated +Go behavior; direct tests and generated exchanges must cover those seams. + +Run explicitly with TLC (not a build, hook or CI gate): + +```sh +java -cp /path/to/tla2tools.jar tlc2.TLC -config checked.cfg MediaContracts.tla +java -cp /path/to/tla2tools.jar tlc2.TLC -config collapsed.cfg MediaContracts.tla +``` + +## Physical branch layouts + +`PhysicalLayouts.tla` models a JSON union containing a native form branch and a +nested JSON field. `inherited-layout.cfg` reproduces the old conversion rule: +a recorded false presence policy cannot override the enclosing true policy. +TLC finds a form declaration/conversion mismatch. `native-layout.cfg` honors the +recorded policy in both directions and checks all 7 reachable states. This is a +bounded correspondence check, not proof of generated Go layout. + +The implementation keeps media-form roots and aliases physically separate in +`transportir.mediaFormCarrier`, records their native layout in +`representation.Layout.Record`, and honors explicit true and false layouts in +`codegen.transformPhysicalContext`. CLI JSON inputs remove the wire form-root +marker from their detached copy. Generated forms exercise optional strings, +collections and JSON map fields; generated JSON unions and ordinary form bodies +remain regression cases. + +```sh +java -cp /path/to/tla2tools.jar tlc2.TLC -config native-layout.cfg PhysicalLayouts.tla +java -cp /path/to/tla2tools.jar tlc2.TLC -config inherited-layout.cfg PhysicalLayouts.tla +``` + +## Validation scope + +The parent for this ticket is `62d12653b`; validation uses Go 1.27.3. The bounded generated-output comparison +uses `PayloadBodyUnionValidateDSL`, `PayloadFormBodyObjectOptionalDSL`, +`ExplicitViewDSL`, and `StreamingPartialExamplesDSL`. Generated Go and OpenAPI +files are byte-identical at parent and current source; `gen/loom.json` changes +only its design digest because the expression model gains media mappings. Each +app compiles. Two independent generations of `MultipleMediaContractsDSL` produce +identical bytes and its generated app compiles. The checked-in HTTP ticktock, +JSON-RPC ticktock and mixedtick fixtures are regenerated; only their design +fingerprints change. The HTTP and JSON-RPC ticktock exact-tree/build tests pass. + +The new specimen covers equal JSON schemas with different examples, distinct +text/binary/form/multipart schemas, range publication, optional bodies, nullable +JSON branches, per-part types, declared error bodies, custom error formatter +replacement, and enclosing example projection. Import rejection tests cover +incompatible schemas (including nested nullable values through references), part +codecs, field names and array serialization. Required and optional empty forms +retain the selected branch. Mapped Content-Type checks use the semantic source +attribute for requests, results and errors; wildcard parameter values do not +turn an exact media key into a range. Direct tests +cover parameter specificity, rejected mappings and codec boundaries. Mounted +client/server tests cover all four directions, validation, malformed/null/empty +inputs, status/problem responses and absence of service calls on rejection. +OpenAPI JSON/YAML are parsed with libopenapi and linted with the pinned Vacuum +rules for 3.1 and 3.2. The full exported-design corpus is outside this ticket's +acceptance scope. diff --git a/internal/httpcodec/tla/media_contracts/checked.cfg b/internal/httpcodec/tla/media_contracts/checked.cfg new file mode 100644 index 000000000..87d7ba5f4 --- /dev/null +++ b/internal/httpcodec/tla/media_contracts/checked.cfg @@ -0,0 +1,4 @@ +CONSTANT Collapse = FALSE +SPECIFICATION Spec +INVARIANTS Specificity Concrete +CHECK_DEADLOCK FALSE diff --git a/internal/httpcodec/tla/media_contracts/collapsed.cfg b/internal/httpcodec/tla/media_contracts/collapsed.cfg new file mode 100644 index 000000000..638b60651 --- /dev/null +++ b/internal/httpcodec/tla/media_contracts/collapsed.cfg @@ -0,0 +1,4 @@ +CONSTANT Collapse = TRUE +SPECIFICATION Spec +INVARIANTS Specificity Concrete +CHECK_DEADLOCK FALSE diff --git a/internal/httpcodec/tla/media_contracts/inherited-layout.cfg b/internal/httpcodec/tla/media_contracts/inherited-layout.cfg new file mode 100644 index 000000000..7ddad0102 --- /dev/null +++ b/internal/httpcodec/tla/media_contracts/inherited-layout.cfg @@ -0,0 +1,4 @@ +CONSTANT IgnoreNative = TRUE +SPECIFICATION Spec +INVARIANT Correspondence +CHECK_DEADLOCK FALSE diff --git a/internal/httpcodec/tla/media_contracts/native-layout.cfg b/internal/httpcodec/tla/media_contracts/native-layout.cfg new file mode 100644 index 000000000..b7d129e54 --- /dev/null +++ b/internal/httpcodec/tla/media_contracts/native-layout.cfg @@ -0,0 +1,4 @@ +CONSTANT IgnoreNative = FALSE +SPECIFICATION Spec +INVARIANT Correspondence +CHECK_DEADLOCK FALSE diff --git a/internal/httpencoding/multipart.go b/internal/httpencoding/multipart.go new file mode 100644 index 000000000..568303b26 --- /dev/null +++ b/internal/httpencoding/multipart.go @@ -0,0 +1,35 @@ +package httpencoding + +import ( + "fmt" + "mime" + "strings" + + "github.com/CaliLuke/loom/internal/httpcodec" +) + +// MultipartContentType resolves a field's part media type and verifies that an +// authored override agrees with the native text, raw byte, or JSON field codec. +func MultipartContentType(shape Shape, binary bool, authored string) (string, error) { + media := "text/plain" + if binary { + media = "application/octet-stream" + } else if shape == Object || shape == Map { + media = "application/json" + } + if authored == "" { + return media, nil + } + base, _, err := mime.ParseMediaType(authored) + if err != nil { + return "", fmt.Errorf("invalid multipart content type %q", authored) + } + if !strings.Contains(base, "/") || strings.Contains(base, "*") { + return "", fmt.Errorf("multipart content type %q must be a concrete type/subtype", authored) + } + kind := httpcodec.RequestContentType(base).Kind + if !binary && (media == "application/json" && kind != httpcodec.JSON || media == "text/plain" && kind != httpcodec.Text) { + return "", fmt.Errorf("multipart content type %q disagrees with its generated codec", authored) + } + return authored, nil +} diff --git a/internal/httpencoding/multipart_test.go b/internal/httpencoding/multipart_test.go new file mode 100644 index 000000000..1463e5526 --- /dev/null +++ b/internal/httpencoding/multipart_test.go @@ -0,0 +1,29 @@ +package httpencoding + +import ( + "testing" + + "github.com/stretchr/testify/require" +) + +func TestMultipartContentTypeRequiresConcreteMedia(t *testing.T) { + for _, tc := range []struct { + media string + valid bool + }{ + {"image/png", true}, + {`image/png; profile="*"`, true}, + {"image/*", false}, + {"*/*", false}, + {"image", false}, + } { + t.Run(tc.media, func(t *testing.T) { + _, err := MultipartContentType(Scalar, true, tc.media) + if tc.valid { + require.NoError(t, err) + } else { + require.ErrorContains(t, err, "concrete") + } + }) + } +} diff --git a/internal/openapiimport/analyze.go b/internal/openapiimport/analyze.go index 061d06647..3eb6648ff 100644 --- a/internal/openapiimport/analyze.go +++ b/internal/openapiimport/analyze.go @@ -513,9 +513,9 @@ func (a *analyzer) content(content *orderedmap.Map[string, *v3.MediaType], path if orderedmap.Len(content) == 0 { return "", nil, nil } - if orderedmap.Len(content) != 1 { - a.unsupported("multiple-media-types", path, "exactly one media type is supported") - return "", nil, nil + if orderedmap.Len(content) > 1 { + _, schema, _ := a.requestContent(content, path) + return "", schema, nil } for contentType, media := range content.FromOldest() { if media == nil { diff --git a/internal/openapiimport/analyze_examples.go b/internal/openapiimport/analyze_examples.go index e7ff91b8c..468e19277 100644 --- a/internal/openapiimport/analyze_examples.go +++ b/internal/openapiimport/analyze_examples.go @@ -41,6 +41,9 @@ func (a *analyzer) mediaExamples(media *v3.MediaType, schema *Schema, path strin examples = append(examples, example) } } + for index := range examples { + examples[index].Scope = "media" + } return examples } @@ -137,12 +140,13 @@ func (a *analyzer) normalizedExample(source *base.Example, schema *Schema, path func (a *analyzer) schemaExamples(schema *Schema, source *base.Schema, path string) { if source.Example != nil { if value, ok := a.exampleValue(source.Example, schema, path+"/example"); ok { - schema.Examples = append(schema.Examples, Example{Value: value}) + schema.Examples = append(schema.Examples, Example{Value: value, Scope: "schema"}) } } for index, node := range source.Examples { if value, ok := a.exampleValue(node, schema, fmt.Sprintf("%s/examples/%d", path, index)); ok { schema.Examples = append(schema.Examples, Example{ + Scope: "schema", Name: fmt.Sprintf("example-%d", index+1), Value: value, }) diff --git a/internal/openapiimport/analyze_request_content.go b/internal/openapiimport/analyze_request_content.go index 04ed713e2..bf7be09e4 100644 --- a/internal/openapiimport/analyze_request_content.go +++ b/internal/openapiimport/analyze_request_content.go @@ -1,8 +1,6 @@ package openapiimport import ( - "reflect" - v3 "github.com/CaliLuke/libopenapi/datamodel/high/v3" "github.com/CaliLuke/libopenapi/orderedmap" "github.com/pb33f/go-yaml" @@ -16,39 +14,40 @@ func (a *analyzer) requestContent( return nil, nil, nil } contentTypes := make([]string, 0, orderedmap.Len(content)) - var sharedSchema *Schema - var sharedExamples []Example + branches := make([]*Schema, 0, orderedmap.Len(content)) + var singleExamples []Example for contentType, media := range content.FromOldest() { - first := len(contentTypes) == 0 contentTypes = append(contentTypes, contentType) + mediaPath := path + "/" + escapeJSONPointer(contentType) var schema *Schema var examples []Example - mediaPath := path + "/" + escapeJSONPointer(contentType) if media != nil { a.mediaTypeParserGapDiagnostics(media, mediaPath) if media.ItemSchema != nil { a.unsupported("media-item-schema", mediaPath+"/itemSchema", "item schemas are not in the strict import subset") } - if orderedmap.Len(media.Encoding) > 0 && !isFormMediaType(contentType) || orderedmap.Len(media.ItemEncoding) > 0 { + if orderedmap.Len(media.Encoding) > 0 && !isFormMediaType(contentType) && !isMultipartMediaType(contentType) || orderedmap.Len(media.ItemEncoding) > 0 { a.unsupported("media-encoding", mediaPath, "media encodings are not in the strict import subset") } schema = a.schema(media.Schema, mediaPath+"/schema") examples = a.mediaExamples(media, schema, mediaPath) + if len(contentTypes) == 1 { + singleExamples = examples + } + if orderedmap.Len(content) > 1 { + schema = schemaWithExamples(schema, examples) + if schema != nil { + schema.MediaEncoding = a.mediaEncoding(media, mediaPath) + } + } a.unsupportedExtensions(mediaPath, media.Extensions) } - if first { - sharedSchema = schema - sharedExamples = examples - continue - } - if !reflect.DeepEqual(schema, sharedSchema) { - a.unsupported("request-media-schema", mediaPath+"/schema", "request media types must use the same schema") - } - if !reflect.DeepEqual(examples, sharedExamples) { - a.unsupported("request-media-examples", mediaPath, "request media types must use the same examples") - } + branches = append(branches, schema) + } + if len(branches) == 1 { + return contentTypes, branches[0], singleExamples } - return contentTypes, sharedSchema, sharedExamples + return contentTypes, &Schema{OneOf: branches, MediaTypes: contentTypes}, nil } // mediaTypeParserGapDiagnostics inspects official OpenAPI 3.2 fields that the diff --git a/internal/openapiimport/analyze_schema_promotion.go b/internal/openapiimport/analyze_schema_promotion.go index d4bf8ec20..64eae6949 100644 --- a/internal/openapiimport/analyze_schema_promotion.go +++ b/internal/openapiimport/analyze_schema_promotion.go @@ -114,6 +114,17 @@ func (a *analyzer) prepareSchemaObjects( path+"/additionalProperties", objectMapValue, used) } for index, branch := range schema.OneOf { + if branch == nil { + continue + } + if len(schema.MediaTypes) > 0 { + componentName := uniqueComponentName(name+"Media", used) + a.prepareSchemaObjects(document, branch, componentName, + fmt.Sprintf("%s/content/%s/schema", path, escapeJSONPointer(schema.MediaTypes[index])), objectDeclaration, used) + document.Components.Schemas = append(document.Components.Schemas, NamedSchema{Name: componentName, Schema: branch}) + schema.OneOf[index] = &Schema{Ref: "#/components/schemas/" + componentName} + continue + } branchName := name + "Variant" if len(branch.Required) > 0 { branchName = name + codegen.Goify(branch.Required[0], true) diff --git a/internal/openapiimport/analyze_test.go b/internal/openapiimport/analyze_test.go index 253cc635b..ef6d385ab 100644 --- a/internal/openapiimport/analyze_test.go +++ b/internal/openapiimport/analyze_test.go @@ -8,8 +8,9 @@ import ( "reflect" "testing" - "github.com/CaliLuke/loom/internal/openapiref" "github.com/stretchr/testify/require" + + "github.com/CaliLuke/loom/internal/openapiref" ) func TestAnalyzeJSONYAMLParity(t *testing.T) { @@ -321,7 +322,7 @@ components: }, document.Operations[0].RequestBody.ContentTypes) } -func TestAnalyzeRejectsIncompatibleRequestMediaTypes(t *testing.T) { +func TestAnalyzeDistinctRequestMediaAdmission(t *testing.T) { tests := []struct { name string media string @@ -334,7 +335,6 @@ func TestAnalyzeRejectsIncompatibleRequestMediaTypes(t *testing.T) { schema: {type: string} multipart/form-data: schema: {type: object, properties: {value: {type: string}}}`, - wantCodes: []string{"request-media-schema"}, }, { name: "different examples", @@ -342,10 +342,9 @@ func TestAnalyzeRejectsIncompatibleRequestMediaTypes(t *testing.T) { application/json: example: first schema: {type: string} - application/x-www-form-urlencoded: + application/vnd.loom+json: example: second schema: {type: string}`, - wantCodes: []string{"request-media-examples"}, }, { name: "multipart encoding", @@ -356,16 +355,14 @@ func TestAnalyzeRejectsIncompatibleRequestMediaTypes(t *testing.T) { schema: {type: object, properties: {value: {type: string}}} encoding: value: {contentType: text/plain}`, - wantCodes: []string{"media-encoding"}, }, { - name: "unsupported media type", + name: "text media type", media: ` application/json: schema: {type: string} text/plain: schema: {type: string}`, - wantCodes: []string{"media-type"}, }, { name: "form media require an object schema", @@ -374,7 +371,7 @@ func TestAnalyzeRejectsIncompatibleRequestMediaTypes(t *testing.T) { schema: {type: string} application/x-www-form-urlencoded: schema: {type: string}`, - wantCodes: []string{"render-plan"}, + wantCodes: []string{"media-schema"}, }, } for _, test := range tests { @@ -390,6 +387,9 @@ paths: `) _, diagnostics, err := Analyze(source) require.NoError(t, err) + if len(test.wantCodes) == 0 { + require.Empty(t, diagnostics) + } for _, code := range test.wantCodes { requireDiagnosticCode(t, diagnostics, code) } diff --git a/internal/openapiimport/issues_299_303_test.go b/internal/openapiimport/issues_299_303_test.go index 7b86e06c7..6ac7dff17 100644 --- a/internal/openapiimport/issues_299_303_test.go +++ b/internal/openapiimport/issues_299_303_test.go @@ -401,6 +401,21 @@ components: ApiKey: {type: apiKey, in: header, name: X-API-Key} `) + _, diagnostics, err := Analyze(source) + require.NoError(t, err) + require.Len(t, diagnostics, 2) + for _, diagnostic := range diagnostics { + require.Equal(t, "media-schema", diagnostic.Code) + require.Contains(t, diagnostic.Path, "/requestBody/content/") + } + + // Multi-media form maps no longer fall back to one shared raw body. + // Keep the other four single-media contracts in the generated round trip. + multiStart := strings.Index(string(source), " /multi/{id}:") + multiEnd := strings.Index(string(source), " /multipart-required:") + require.GreaterOrEqual(t, multiStart, 0) + require.Greater(t, multiEnd, multiStart) + source = []byte(string(source[:multiStart]) + string(source[multiEnd:])) document, diagnostics, err := Analyze(source) require.NoError(t, err) require.Empty(t, diagnostics) @@ -413,16 +428,15 @@ components: `Attribute("body3", ImportedConfig)`, `Required("id", "body3")`, `Body("body3")`, - `OpenAPIRequestBodyTypes(ImportedConfig, []string{"application/json", "application/x-www-form-urlencoded", "multipart/form-data"}, false)`, `OpenAPIRequestBodyTypes(ImportedConfig, []string{"multipart/form-data"}, false)`, "MultipartRequest()", } { require.Contains(t, design, expected) } - requireRenderedDesignEvaluates(t, rendered, 5) + requireRenderedDesignEvaluates(t, rendered, 4) contract := readGeneratedOpenAPIContract(t, requireRenderedDesignGenerates(t, rendered)) - for _, path := range []string{"/required", "/envelope/{id}", "/multi/{id}", "/multipart-required", "/multipart-optional"} { + for _, path := range []string{"/required", "/envelope/{id}", "/multipart-required", "/multipart-optional"} { operation := operationFromImportedSpec(t, contract, path, "patch") require.NotNil(t, operation["requestBody"], path) } @@ -432,9 +446,6 @@ components: envelope := patchRequestBodyFromImportedSpec(t, contract, "/envelope/{id}") require.Equal(t, true, envelope["required"]) require.Equal(t, "#/components/schemas/Config", envelope["content"].(map[string]any)["application/x-www-form-urlencoded"].(map[string]any)["schema"].(map[string]any)["$ref"]) - multi := patchRequestBodyFromImportedSpec(t, contract, "/multi/{id}") - require.NotEqual(t, true, multi["required"]) - require.ElementsMatch(t, []string{"application/json", "application/x-www-form-urlencoded", "multipart/form-data"}, mapKeysAny(multi["content"].(map[string]any))) requiredMultipart := patchRequestBodyFromImportedSpec(t, contract, "/multipart-required") require.Equal(t, true, requiredMultipart["required"]) optionalMultipart := patchRequestBodyFromImportedSpec(t, contract, "/multipart-optional") @@ -458,14 +469,6 @@ func patchRequestBodyFromImportedSpec(t *testing.T, contract map[string]any, pat return body } -func mapKeysAny(values map[string]any) []string { - keys := make([]string, 0, len(values)) - for key := range values { - keys = append(keys, key) - } - return keys -} - func TestDynamicFormMapEnvelopeRejectsGeneratedBodyLocalCollision(t *testing.T) { source := []byte(`openapi: 3.1.1 info: {title: Collision, version: "1"} diff --git a/internal/openapiimport/media_contract.go b/internal/openapiimport/media_contract.go new file mode 100644 index 000000000..086ed3f25 --- /dev/null +++ b/internal/openapiimport/media_contract.go @@ -0,0 +1,225 @@ +package openapiimport + +import ( + "fmt" + "maps" + "mime" + "slices" + "strconv" + "strings" + + v3 "github.com/CaliLuke/libopenapi/datamodel/high/v3" + "github.com/CaliLuke/libopenapi/orderedmap" + + "github.com/CaliLuke/loom/internal/httpcodec" + "github.com/CaliLuke/loom/internal/httpencoding" + "github.com/CaliLuke/loom/internal/openapiref" +) + +func (a *analyzer) mediaEncoding(media *v3.MediaType, path string) map[string]httpencoding.Serialization { + result := make(map[string]httpencoding.Serialization, orderedmap.Len(media.Encoding)) + for name, encoding := range media.Encoding.FromOldest() { + if encoding == nil { + continue + } + if orderedmap.Len(encoding.Headers) > 0 { + a.unsupported("media-encoding-headers", path+"/encoding/"+escapeJSONPointer(name)+"/headers", "custom part headers are not in the strict import subset") + } + value := httpencoding.Serialization{MediaType: encoding.ContentType, Style: encoding.Style, Explode: encoding.Explode} + if encoding.AllowReserved || encoding.GoLow() != nil && !encoding.GoLow().AllowReserved.IsEmpty() { + value.AllowReserved = &encoding.AllowReserved + } + result[name] = value + } + return result +} + +func (p *documentPlanner) mediaContracts(schema *Schema, path string) { + if len(schema.MediaTypes) != len(schema.OneOf) { + p.unsupported("media-contracts", path, "each media type requires one schema branch") + return + } + if err := httpcodec.ValidateMediaContracts(schema.MediaTypes); err != nil { + p.unsupported("media-contracts", path, err.Error()) + } + for index, media := range schema.MediaTypes { + branch := schema.OneOf[index] + mediaPath := path + "/content/" + escapeJSONPointer(media) + if branch == nil { + p.unsupported("content-schema", mediaPath+"/schema", "content type has no schema") + continue + } + p.mediaBranchSchema(branch, media, mediaPath) + role := schemaEncodedValue + if isMultipartMediaType(media) { + role = schemaMultipartBody + } else if !isJSONMediaType(media) && !isFormMediaType(media) { + role = schemaRawBody + } + p.schemaEncoding(branch, mediaPath+"/schema", role, make(map[schemaEncodingOccurrence]bool)) + p.schema(branch, mediaPath+"/schema", objectField) + } +} + +func (r *renderer) mediaTypeMappings(schema *Schema) error { + for index, media := range schema.MediaTypes { + branch := schema.OneOf[index] + name, err := openapiref.Name(branch.Ref, "#/components/schemas/") + if err != nil { + return fmt.Errorf("render media contract %q: %w", media, err) + } + r.line("MediaType(%q, %q)", media, name) + } + return nil +} + +func (r *renderer) mediaEncodingMetadata(schema *Schema) { + for _, name := range slices.Sorted(maps.Keys(schema.MediaEncoding)) { + encoding := schema.MediaEncoding[name] + prefix := "openapi:encoding:" + name + ":" + if encoding.MediaType != "" { + r.line("Meta(%q, %q)", prefix+"contentType", encoding.MediaType) + } + if encoding.Style != "" { + r.line("Meta(%q, %q)", prefix+"style", encoding.Style) + } + if encoding.Explode != nil { + r.line("Meta(%q, %q)", prefix+"explode", strconv.FormatBool(*encoding.Explode)) + } + if encoding.AllowReserved != nil { + r.line("Meta(%q, %q)", prefix+"allowReserved", strconv.FormatBool(*encoding.AllowReserved)) + } + } +} + +func mediaRangeSchema(schema *Schema) bool { + if schema == nil { + return false + } + for _, media := range schema.MediaTypes { + if strings.Contains(strings.SplitN(media, ";", 2)[0], "*") { + return true + } + } + return false +} + +func withMediaContentTypeParameter(parameters []Parameter) []Parameter { + for _, parameter := range parameters { + if parameter.In == "header" && strings.EqualFold(parameter.Name, "Content-Type") { + return parameters + } + } + return append(slices.Clone(parameters), Parameter{Name: "Content-Type", In: "header", Schema: &Schema{Type: "string"}}) +} + +func withMediaContentTypeHeader(headers []NamedHeader) []NamedHeader { + for _, header := range headers { + if strings.EqualFold(header.Name, "Content-Type") { + return headers + } + } + return append(slices.Clone(headers), NamedHeader{Name: "Content-Type", Header: Header{Schema: &Schema{Type: "string"}}}) +} + +func (p *documentPlanner) mediaEncodingFor(schema *Schema) map[string]httpencoding.Serialization { + seen := make(map[*Schema]bool) + for schema != nil && !seen[schema] { + seen[schema] = true + if schema.MediaEncoding != nil { + return schema.MediaEncoding + } + if schema.Ref == "" { + return nil + } + name, err := openapiref.Name(schema.Ref, "#/components/schemas/") + if err != nil { + return nil + } + schema = p.schemas[name].Schema + } + return nil +} + +func (p *documentPlanner) mediaBranchSchema(branch *Schema, media, path string) { + resolved := p.urlSchema(branch) + if resolved == nil { + return + } + base, _, err := mime.ParseMediaType(media) + if err != nil { + return // The media map validator reports malformed keys. + } + if httpcodec.RequestContentType(base).Kind == httpcodec.Text && (resolved.Type != "string" || resolved.Nullable) { + p.unsupported("media-schema", path, "text media requires a non-nullable string or byte schema") + } + if base == "application/octet-stream" && (resolved.Type != "string" || resolved.Format != "binary" || resolved.Nullable) { + p.unsupported("media-schema", path, "octet-stream media requires a non-nullable binary string schema") + } + if !isFormMediaType(media) && !isMultipartMediaType(media) { + return + } + representation, representationErr := classifyObject(resolved) + if resolved.Type != "object" || representationErr != nil || representation != objectStructured || !p.mediaFormSchemaSupported(branch, make(map[*Schema]bool)) { + p.unsupported("media-schema", path, "form and multipart media require object schemas without nullable values or untagged unions") + return + } + properties := make(map[string]*Schema) + p.formProperties(branch, properties, make(map[*Schema]bool)) + encoding := p.mediaEncodingFor(branch) + for _, name := range slices.Sorted(maps.Keys(encoding)) { + if _, exists := properties[name]; !exists { + p.unsupported("media-encoding", path+"/encoding/"+escapeJSONPointer(name), "encoding must name a body property") + } + } + for _, name := range slices.Sorted(maps.Keys(properties)) { + property := p.urlSchema(properties[name]) + shape := p.urlShape(property) + value := encoding[name] + fieldPath := path + "/encoding/" + escapeJSONPointer(name) + if property == nil { + continue + } + if isMultipartMediaType(media) { + if _, err := httpencoding.MultipartContentType(shape, property.Type == "string" && stringRepresentation(property.Format).raw, value.MediaType); err != nil { + p.unsupported("media-encoding", fieldPath, err.Error()) + } + value.MediaType = "" + } + if err := httpencoding.Match(httpencoding.FormField, shape, value); err != nil { + p.unsupported("media-encoding", fieldPath, err.Error()) + } + } +} + +// mediaFormSchemaSupported mirrors the evaluated design's recursive form +// admission, including alias occurrences and nested JSON-valued fields. +func (p *documentPlanner) mediaFormSchemaSupported(schema *Schema, seen map[*Schema]bool) bool { + if schema == nil || seen[schema] { + return true + } + seen[schema] = true + if schema.Nullable || len(schema.OneOf) > 0 { + return false + } + if schema.Ref != "" { + name, err := openapiref.Name(schema.Ref, "#/components/schemas/") + if err == nil && !p.mediaFormSchemaSupported(p.schemas[name].Schema, seen) { + return false + } + } + for _, property := range schema.Properties { + if !p.mediaFormSchemaSupported(property.Schema, seen) { + return false + } + } + for _, base := range schema.Bases { + if !p.mediaFormSchemaSupported(base, seen) { + return false + } + } + if schema.AdditionalProperties != nil && !p.mediaFormSchemaSupported(schema.AdditionalProperties.Schema, seen) { + return false + } + return p.mediaFormSchemaSupported(schema.Items, seen) +} diff --git a/internal/openapiimport/media_contract_test.go b/internal/openapiimport/media_contract_test.go new file mode 100644 index 000000000..9e5e13ef1 --- /dev/null +++ b/internal/openapiimport/media_contract_test.go @@ -0,0 +1,94 @@ +package openapiimport + +import ( + "encoding/json/v2" + "testing" + + "github.com/stretchr/testify/require" +) + +func TestDistinctMediaContractsRoundTrip(t *testing.T) { + var source map[string]any + require.NoError(t, json.Unmarshal(readFixture(t, "supported.json"), &source)) + source["components"].(map[string]any)["schemas"].(map[string]any)["MediaFormData"] = map[string]any{ + "type": "object", "properties": map[string]any{"tags": map[string]any{"type": "array", "items": map[string]any{"type": "string"}}}, + } + content := map[string]any{ + "application/x-www-form-urlencoded": map[string]any{"schema": map[string]any{"$ref": "#/components/schemas/MediaFormData"}, "encoding": map[string]any{"tags": map[string]any{"style": "form", "explode": true}}}, + "multipart/form-data": map[string]any{"schema": map[string]any{"$ref": "#/components/schemas/MediaFormData"}, "encoding": map[string]any{"tags": map[string]any{"contentType": "text/plain"}}}, + "application/*": map[string]any{"schema": map[string]any{"type": "string", "format": "binary"}}, + "application/json": map[string]any{"schema": map[string]any{"type": "integer"}, "example": 12}, + "application/vnd.loom+json": map[string]any{"schema": map[string]any{"type": "integer"}, "example": 34}, + "text/plain": map[string]any{"schema": map[string]any{"type": "string"}, "example": "hello"}, + } + source["paths"].(map[string]any)["/representations"] = map[string]any{"post": map[string]any{ + "operationId": "exchangeRepresentations", + "requestBody": map[string]any{"required": true, "content": content}, + "responses": map[string]any{"200": map[string]any{"description": "Selected representation.", "content": content}, "400": map[string]any{"description": "Rejected representation.", "content": map[string]any{"application/json": content["application/json"], "text/plain": content["text/plain"]}}}, + }} + encoded, err := json.Marshal(source, json.Deterministic(true)) + require.NoError(t, err) + document, diagnostics, err := Analyze(encoded) + require.NoError(t, err) + require.Empty(t, diagnostics) + rendered, err := Render(document, Options{PackageName: "design"}) + require.NoError(t, err) + directory := requireRenderedDesignGenerates(t, rendered) + contract := readGeneratedOpenAPIContract(t, directory) + operation := operationFromImportedSpec(t, contract, "/representations", "post") + request := operation["requestBody"].(map[string]any)["content"].(map[string]any) + response := operation["responses"].(map[string]any)["200"].(map[string]any)["content"].(map[string]any) + for _, got := range []map[string]any{request, response} { + require.Len(t, got, 6) + form := got["application/x-www-form-urlencoded"].(map[string]any) + require.Equal(t, true, form["encoding"].(map[string]any)["tags"].(map[string]any)["explode"]) + multipart := got["multipart/form-data"].(map[string]any) + require.Equal(t, "text/plain", multipart["encoding"].(map[string]any)["tags"].(map[string]any)["contentType"]) + for media, expected := range map[string]any{"application/json": float64(12), "application/vnd.loom+json": float64(34), "text/plain": "hello"} { + require.Equal(t, expected, got[media].(map[string]any)["example"]) + } + } + errorContent := operation["responses"].(map[string]any)["400"].(map[string]any)["content"].(map[string]any) + require.Len(t, errorContent, 2) + require.Equal(t, "hello", errorContent["text/plain"].(map[string]any)["example"]) + requireIndependentRegeneration(t, directory) +} + +func TestDistinctMediaContractsRejectUnsupportedEncoding(t *testing.T) { + for _, tc := range []struct { + name, media string + schema map[string]any + encoding map[string]any + }{ + {"multipart explode", "multipart/form-data", map[string]any{"type": "object", "properties": map[string]any{"tags": map[string]any{"type": "array", "items": map[string]any{"type": "string"}}}}, map[string]any{"tags": map[string]any{"explode": false}}}, + {"unknown part", "multipart/form-data", map[string]any{"type": "object", "properties": map[string]any{"name": map[string]any{"type": "string"}}}, map[string]any{"missing": map[string]any{"contentType": "text/plain"}}}, + {"part range", "multipart/form-data", map[string]any{"type": "object", "properties": map[string]any{"data": map[string]any{"type": "string", "format": "binary"}}}, map[string]any{"data": map[string]any{"contentType": "image/*"}}}, + {"wrong part codec", "multipart/form-data", map[string]any{"type": "object", "properties": map[string]any{"name": map[string]any{"type": "string"}}}, map[string]any{"name": map[string]any{"contentType": "application/json"}}}, + {"non-text", "text/plain", map[string]any{"type": "integer"}, nil}, + {"non-binary", "application/octet-stream", map[string]any{"type": "integer"}, nil}, + {"nested nullable", "application/x-www-form-urlencoded", map[string]any{"type": "object", "properties": map[string]any{"nested": map[string]any{"type": "object", "properties": map[string]any{"value": map[string]any{"type": []string{"string", "null"}}}}}}, nil}, + {"nested union", "multipart/form-data", map[string]any{"type": "object", "properties": map[string]any{"nested": map[string]any{"type": "object", "properties": map[string]any{"value": map[string]any{"oneOf": []any{map[string]any{"type": "integer"}, map[string]any{"type": "string"}}}}}}}, nil}, + {"nested reference", "multipart/form-data", map[string]any{"type": "object", "properties": map[string]any{"nested": map[string]any{"type": "object", "properties": map[string]any{"value": map[string]any{"$ref": "#/components/schemas/NullableMediaValue"}}}}}, nil}, + {"nullable form", "application/x-www-form-urlencoded", map[string]any{"type": []string{"object", "null"}, "properties": map[string]any{"name": map[string]any{"type": "string"}}}, nil}, + } { + t.Run(tc.name, func(t *testing.T) { + var source map[string]any + require.NoError(t, json.Unmarshal(readFixture(t, "supported.json"), &source)) + source["components"].(map[string]any)["schemas"].(map[string]any)["MediaInvalid"] = tc.schema + source["components"].(map[string]any)["schemas"].(map[string]any)["NullableMediaValue"] = map[string]any{"type": []string{"string", "null"}} + source["paths"].(map[string]any)["/media-invalid"] = map[string]any{"post": map[string]any{ + "operationId": "invalidMedia", + "requestBody": map[string]any{"content": map[string]any{ + "application/json": map[string]any{"schema": map[string]any{"type": "integer"}}, + tc.media: map[string]any{"schema": map[string]any{"$ref": "#/components/schemas/MediaInvalid"}, "encoding": tc.encoding}, + }}, + "responses": map[string]any{"204": map[string]any{"description": "No content."}}, + }} + encoded, err := json.Marshal(source, json.Deterministic(true)) + require.NoError(t, err) + _, diagnostics, err := Analyze(encoded) + require.NoError(t, err) + require.NotEmpty(t, diagnostics) + }) + } +} diff --git a/internal/openapiimport/model.go b/internal/openapiimport/model.go index c3f63149e..a0795dc3e 100644 --- a/internal/openapiimport/model.go +++ b/internal/openapiimport/model.go @@ -259,7 +259,7 @@ type RequestBody struct { Description string // Required reports whether the request body is required. Required bool - // ContentTypes are the request media types that share Schema and Examples. + // ContentTypes are the authored request media types in source order. ContentTypes []string // Schema describes the request body. Schema *Schema @@ -348,7 +348,12 @@ type Schema struct { // LegacyNullable retains the OpenAPI 3.0 nullable keyword, including false. // Other dialects reject this keyword instead of applying OpenAPI 3.0 semantics. LegacyNullable *bool - // OneOf contains the branches of a supported untagged object union. + // MediaTypes maps OneOf branches to distinct HTTP content representations. + // When present, OneOf is a service union, not an authored JSON oneOf. + MediaTypes []string + // MediaEncoding retains this branch's per-property HTTP encoding declarations. + MediaEncoding map[string]httpencoding.Serialization + // OneOf contains the branches of a supported union. OneOf []*Schema // Format is the authored schema format. Format string @@ -413,6 +418,8 @@ type Schema struct { // Example is an example value retained from a schema or media type. type Example struct { + // Scope identifies schema or media ownership; empty retains shared DSL behavior. + Scope string // Name is the authored media example key or a deterministic schema example name. Name string // Summary is the authored media example summary. diff --git a/internal/openapiimport/plan.go b/internal/openapiimport/plan.go index bea72a4d5..be96f7f4d 100644 --- a/internal/openapiimport/plan.go +++ b/internal/openapiimport/plan.go @@ -165,7 +165,7 @@ func (p *documentPlanner) operation(operation *Operation, path string) { if !success && p.schemaContainsUntaggedOneOf(response.Response.Schema, make(map[string]struct{})) { p.unsupported("schema-oneof-location", responsePath+"/content/schema", "untagged oneOf is supported only in JSON request and success response bodies") } - if response.Response.Schema != nil && !isJSONMediaType(response.Response.ContentType) && + if response.Response.Schema != nil && len(response.Response.Schema.MediaTypes) == 0 && !isJSONMediaType(response.Response.ContentType) && responseMode == responseBodyEncoded { p.unsupported( "media-type", @@ -223,6 +223,10 @@ func (p *documentPlanner) requestBody(body *RequestBody, path string) { // resolveReusableDocument already reported this unresolved occurrence. return } + if body.Schema != nil && len(body.Schema.MediaTypes) > 0 { + p.mediaContracts(body.Schema, path) + return + } if len(body.ContentTypes) == 0 { p.content("", body.Schema, path, false, requestObjectRole(*body)) return @@ -257,7 +261,11 @@ func (p *documentPlanner) response(response Response, path string, allowNonJSON // resolveReusableDocument already reported this unresolved occurrence. return } - p.content(response.ContentType, response.Schema, path, allowNonJSON, responseObjectRole(response, allowNonJSON)) + if response.Schema != nil && len(response.Schema.MediaTypes) > 0 { + p.mediaContracts(response.Schema, path) + } else { + p.content(response.ContentType, response.Schema, path, allowNonJSON, responseObjectRole(response, allowNonJSON)) + } for _, named := range response.Headers { headerPath := path + "/headers/" + escapeJSONPointer(named.Name) if strings.Contains(named.Name, ":") { @@ -311,7 +319,7 @@ func (p *documentPlanner) schemaContainsUntaggedOneOf(schema *Schema, seen map[s if schema == nil { return false } - if len(schema.OneOf) > 0 { + if len(schema.OneOf) > 0 && len(schema.MediaTypes) == 0 { return true } if schema.Ref != "" { @@ -347,7 +355,7 @@ func operationResponseBodyMode(operation *Operation) responseBodyMode { successClass := successResponseClass(operation.Responses) for _, response := range operation.Responses { if !isSuccessResponseStatus(response.Status, successClass) || response.Response.Schema == nil || - isJSONMediaType(response.Response.ContentType) { + isJSONMediaType(response.Response.ContentType) || len(response.Response.Schema.MediaTypes) > 0 { continue } if (operation.Method == "GET" || operation.Method == "HEAD") && response.Status == "200" && @@ -398,7 +406,7 @@ func (p *documentPlanner) schema(schema *Schema, path string, role objectConstru if !p.schemaAdmission(schema, path, role) { return } - if schema != nil && len(schema.OneOf) > 0 { + if schema != nil && len(schema.OneOf) > 0 && len(schema.MediaTypes) == 0 { for index, branch := range schema.OneOf { if branch == nil || branch.Ref == "" { p.unsupported("schema-oneof-branch", fmt.Sprintf("%s/oneOf/%d", path, index), "untagged oneOf branches must reference concrete object or array components") diff --git a/internal/openapiimport/render.go b/internal/openapiimport/render.go index c4f4f3826..b34d183a1 100644 --- a/internal/openapiimport/render.go +++ b/internal/openapiimport/render.go @@ -260,7 +260,11 @@ func (r *renderer) planOperation(operation *Operation, path string) (operationPl if err != nil { return operationPlan{}, err } - parameters, err := r.parameters(operation.Parameters, path+"/parameters") + sourceParameters := operation.Parameters + if operation.RequestBody != nil && mediaRangeSchema(operation.RequestBody.Schema) { + sourceParameters = withMediaContentTypeParameter(sourceParameters) + } + parameters, err := r.parameters(sourceParameters, path+"/parameters") if err != nil { return operationPlan{}, err } @@ -528,6 +532,7 @@ func (r *renderer) errorTypeClones() error { func (r *renderer) responseType(call, name string, response renderedResponse, path string) error { responseSchema := schemaWithExamples(response.response.Schema, response.response.Examples) wrapUnconstrainedError := call == "Error" && r.effectiveUnconstrained(responseSchema) + wrapMediaError := call == "Error" && responseSchema != nil && len(responseSchema.MediaTypes) > 0 prefix := call + "(" if name != "" { prefix += strconv.Quote(name) + ", " @@ -536,7 +541,7 @@ func (r *renderer) responseType(call, name string, response renderedResponse, pa r.line("%s%s)", prefix, response.cloneErrorGoName) return nil } - if len(response.headers) == 0 && !wrapUnconstrainedError { + if len(response.headers) == 0 && !wrapUnconstrainedError && !wrapMediaError { if call == "Error" && response.response.Schema == nil { r.open("Error(%q, func()", name) r.line("Attribute(%q, String)", "message") diff --git a/internal/openapiimport/render_media.go b/internal/openapiimport/render_media.go index 2cd5aeb09..38fb28d43 100644 --- a/internal/openapiimport/render_media.go +++ b/internal/openapiimport/render_media.go @@ -58,7 +58,7 @@ func (r *renderer) requestBody(source *RequestBody, path string) (*renderedBody, return nil, fmt.Errorf("render OpenAPI design: %s has no content type", path) } if len(body.ContentTypes) > 1 { - return r.multipleRequestBody(body, path) + return &renderedBody{body: body, field: "body", mode: requestBodyJSON}, nil } contentType := body.ContentTypes[0] mode, err := requestBodyModeFor(contentType, path) @@ -68,21 +68,6 @@ func (r *renderer) requestBody(source *RequestBody, path string) (*renderedBody, return r.singleRequestBody(body, mode, path) } -func (r *renderer) multipleRequestBody(body RequestBody, path string) (*renderedBody, error) { - for _, contentType := range body.ContentTypes { - if !isMultipartMediaType(contentType) && !isFormMediaType(contentType) { - continue - } - if err := r.validateRequestTransportBodySchema( - body.Schema, - path+"/content/"+escapeJSONPointer(contentType)+"/schema", - ); err != nil { - return nil, err - } - } - return &renderedBody{body: body, mode: requestBodyRaw}, nil -} - func requestBodyModeFor(contentType, path string) (requestBodyMode, error) { switch { case contentType != "" && httpcodec.RequestContentType(contentType).Kind == httpcodec.JSON: @@ -219,7 +204,7 @@ func (r *renderer) renderedResponse(source StatusResponse, rawBody bool, path st if source.Response.Schema == nil && source.Response.ContentType != "" { return renderedResponse{}, fmt.Errorf("render OpenAPI design: %s has content type but no schema", path) } - if source.Response.Schema != nil && (source.Response.ContentType == "" || + if source.Response.Schema != nil && len(source.Response.Schema.MediaTypes) == 0 && (source.Response.ContentType == "" || !rawBody && !isJSONMediaType(source.Response.ContentType)) { return renderedResponse{}, fmt.Errorf( "render OpenAPI design: %s content type %q is not renderable", @@ -227,6 +212,9 @@ func (r *renderer) renderedResponse(source StatusResponse, rawBody bool, path st source.Response.ContentType, ) } + if mediaRangeSchema(source.Response.Schema) { + source.Response.Headers = withMediaContentTypeHeader(source.Response.Headers) + } used := make(map[string]int) headers := make([]renderedHeader, 0, len(source.Response.Headers)) for _, named := range source.Response.Headers { diff --git a/internal/openapiimport/render_request_body.go b/internal/openapiimport/render_request_body.go index cff2cfa58..680a532e0 100644 --- a/internal/openapiimport/render_request_body.go +++ b/internal/openapiimport/render_request_body.go @@ -62,6 +62,13 @@ func (r *renderer) operationRequestBodyHTTP(body *renderedBody, path string) err if body == nil { return nil } + if len(body.body.Schema.MediaTypes) > 0 { + if err := r.mediaTypeMappings(body.body.Schema); err != nil { + return err + } + r.line("Body(%q)", body.field) + return nil + } switch body.mode { case requestBodyMultipart: r.line("MultipartRequest()") diff --git a/internal/openapiimport/render_response.go b/internal/openapiimport/render_response.go index e56d604f3..f72d5e31f 100644 --- a/internal/openapiimport/render_response.go +++ b/internal/openapiimport/render_response.go @@ -72,7 +72,7 @@ func responseMappingPrefix(response renderedResponse, failure bool, path string) } func responseMappingNeedsBlock(response renderedResponse, failure bool) bool { - return len(response.response.referenceChain) > 0 || failure || response.response.Summary != "" || response.response.Description != "" || + return response.response.Schema != nil && len(response.response.Schema.MediaTypes) > 0 || len(response.response.referenceChain) > 0 || failure || response.response.Summary != "" || response.response.Description != "" || response.response.ContentType != "" || len(response.headers) > 0 || len(response.response.Extensions) > 0 } @@ -97,6 +97,11 @@ func (r *renderer) responseMappingMetadata(response renderedResponse, failure bo if err := r.emitExtensions("response", response.response.Extensions); err != nil { return err } + if response.response.Schema != nil && len(response.response.Schema.MediaTypes) > 0 { + if err := r.mediaTypeMappings(response.response.Schema); err != nil { + return err + } + } if response.response.ContentType != "" { r.line("ContentType(%q)", response.response.ContentType) } @@ -117,7 +122,8 @@ func (r *renderer) responseMappingRawBody(response renderedResponse, path string func (r *renderer) responseMappingBody(response renderedResponse, failure bool, path string) error { wrapUnconstrainedError := failure && r.effectiveUnconstrained(response.response.Schema) - if response.body != "" && (len(response.headers) > 0 || wrapUnconstrainedError) { + wrapMediaError := failure && response.response.Schema != nil && len(response.response.Schema.MediaTypes) > 0 + if response.body != "" && (len(response.headers) > 0 || wrapUnconstrainedError || wrapMediaError) { r.line("Body(%q)", response.body) } if len(response.headers) == 0 && !response.rawBody && response.response.Schema != nil { diff --git a/internal/openapiimport/render_schema.go b/internal/openapiimport/render_schema.go index d404309a1..7055a5b96 100644 --- a/internal/openapiimport/render_schema.go +++ b/internal/openapiimport/render_schema.go @@ -277,6 +277,7 @@ func (r *renderer) schemaBlock(schema *Schema, path string, errorType bool, desc } func (r *renderer) schemaAttributeBlock(schema *Schema, path string, description *string) error { + r.mediaEncodingMetadata(schema) if err := r.schemaRequired(schema, path); err != nil { return err } @@ -353,7 +354,7 @@ func (r *renderer) schemaAnnotations(schema *Schema, description *string) error } func (r *renderer) untaggedValidation(schema *Schema) { - if len(schema.OneOf) > 0 { + if len(schema.OneOf) > 0 && len(schema.MediaTypes) == 0 { r.line("Untagged()") } } @@ -404,7 +405,7 @@ func (r *renderer) hasSchemaBlock(schema *Schema) bool { if schema == nil { return false } - if schema.Title != "" || schema.Description != "" || len(schema.OneOf) > 0 || len(schema.Bases) > 0 || len(schema.Properties) > 0 || len(schema.Required) > 0 || len(schema.Enum) > 0 || + if len(schema.MediaEncoding) > 0 || schema.Title != "" || schema.Description != "" || len(schema.OneOf) > 0 || len(schema.Bases) > 0 || len(schema.Properties) > 0 || len(schema.Required) > 0 || len(schema.Enum) > 0 || schema.Pattern != "" || schema.Minimum != nil || schema.Maximum != nil || schema.ExclusiveMinimum != nil || schema.ExclusiveMaximum != nil || schema.MinLength != nil || schema.MaxLength != nil || schema.MinItems != nil || schema.MaxItems != nil || @@ -438,7 +439,7 @@ func (r *renderer) example(example Example, path string) error { if err != nil { return fmt.Errorf("render OpenAPI design: %s example: %w", path, err) } - structured := example.ComponentName != "" || example.DataValue || example.SerializedValue != "" + structured := example.Scope != "" || example.ComponentName != "" || example.DataValue || example.SerializedValue != "" if example.Name == "" && example.Summary == "" && example.Description == "" && !structured { r.line("Example(%s)", literal) return nil @@ -452,6 +453,9 @@ func (r *renderer) example(example Example, path string) error { return nil } r.open("Example(%q, func()", name) + if example.Scope != "" { + r.line("Meta(%q, %q)", "openapi:example:scope", example.Scope) + } if example.ComponentName != "" { r.line("Meta(%q, %q)", "openapi:component:example", example.ComponentName) } diff --git a/internal/openapiimport/render_test.go b/internal/openapiimport/render_test.go index 1165ccb95..7f3349e6d 100644 --- a/internal/openapiimport/render_test.go +++ b/internal/openapiimport/render_test.go @@ -224,62 +224,6 @@ components: }, operationFromImportedSpec(t, contract, "/optional", "get")["security"]) } -func TestRenderPreservesSchemaAndMediaExamples(t *testing.T) { - source := []byte(`openapi: 3.1.1 -info: {title: Examples, version: "1"} -paths: - /pets: - post: - operationId: createPet - requestBody: - required: true - content: - application/json: - example: {name: Fido} - schema: {$ref: '#/components/schemas/Pet'} - responses: - "200": - description: created - content: - application/json: - examples: - created: - summary: Created pet - description: A newly-created pet. - value: {name: Fido} - schema: {$ref: '#/components/schemas/Pet'} -components: - schemas: - Pet: - type: object - required: [name] - properties: - name: {type: string, example: Fido} - kind: {type: string, examples: [dog, cat]} -`) - - document, diagnostics, err := Analyze(source) - require.NoError(t, err) - require.Empty(t, diagnostics) - rendered, err := Render(document, Options{PackageName: "design"}) - require.NoError(t, err) - requireRenderedDesignEvaluates(t, rendered, 1) - - design := string(rendered) - for _, expected := range []string{ - `Example("Fido")`, - `Example("example-1", "dog")`, - `Example("example-2", "cat")`, - `Example(Val{"name": "Fido"})`, - `Example("created", func() {`, - `Meta("openapi:example:summary", "Created pet")`, - `Description("A newly-created pet.")`, - `Value(Val{"name": "Fido"})`, - } { - require.Contains(t, design, expected) - } -} - func TestRenderPreservesSchemaTitles(t *testing.T) { source := []byte(`openapi: 3.1.1 info: {title: Titles, version: "1"} @@ -430,9 +374,11 @@ components: rendered, err := Render(document, Options{PackageName: "design"}) require.NoError(t, err) design := string(rendered) - require.Contains(t, design, "SkipRequestBodyEncodeDecode()") - require.Contains(t, design, `OpenAPIRequestBodyTypes(ImportedItem, []string{"application/json", "application/x-www-form-urlencoded", "multipart/form-data"}, true`) - require.NotContains(t, design, `Body("body")`) + require.NotContains(t, design, "SkipRequestBodyEncodeDecode()") + require.Contains(t, design, `MediaType("application/json",`) + require.Contains(t, design, `MediaType("application/x-www-form-urlencoded",`) + require.Contains(t, design, `MediaType("multipart/form-data",`) + require.Contains(t, design, `Body("body")`) requireRenderedDesignEvaluates(t, rendered, 1) moduleDir := requireRenderedDesignGenerates(t, rendered) @@ -479,7 +425,7 @@ paths: require.Empty(t, diagnostics) rendered, err := Render(document, Options{PackageName: "design"}) require.NoError(t, err) - require.Contains(t, string(rendered), "OpenAPIRequestBodyTypes(func() {") + require.Contains(t, string(rendered), `MediaType("multipart/form-data",`) requireRenderedDesignEvaluates(t, rendered, 1) } @@ -550,121 +496,6 @@ components: } } -func TestRenderPreservesNullableSchemas(t *testing.T) { - source := []byte(`openapi: 3.1.1 -info: {title: Nullable, version: "1"} -paths: - /records: - post: - operationId: createRecord - requestBody: - required: true - content: - application/json: - schema: {$ref: '#/components/schemas/Record'} - responses: - "200": - description: record - content: - application/json: - schema: {$ref: '#/components/schemas/Record'} -components: - schemas: - Record: - type: object - required: [required_note, required_child] - examples: - - required_note: null - required_child: null - nullable_items: [value, null] - nullable_values: {value: null} - nullable_objects: [{name: loom}, null] - nullable_object_values: {value: null} - properties: - required_note: - type: [string, "null"] - examples: [null] - optional_count: {type: [integer, "null"]} - required_child: - type: [object, "null"] - properties: - name: {type: string} - optional_tags: - type: [array, "null"] - items: {type: string} - nullable_items: - type: array - items: {type: [string, "null"]} - nullable_values: - type: object - additionalProperties: {type: [integer, "null"]} - nullable_objects: - type: array - items: - anyOf: - - {$ref: '#/components/schemas/Child'} - - {type: "null"} - nullable_object_values: - type: object - additionalProperties: - anyOf: - - {$ref: '#/components/schemas/Child'} - - {type: "null"} - optional_label: {$ref: '#/components/schemas/NullableLabel'} - Child: - type: object - properties: - name: {type: string} - NullableLabel: - type: [string, "null"] -`) - - document, diagnostics, err := Analyze(source) - require.NoError(t, err) - require.Empty(t, diagnostics) - rendered, err := Render(document, Options{PackageName: "design"}) - require.NoError(t, err) - requireRenderedDesignEvaluates(t, rendered, 1) - moduleDir := requireRenderedDesignGenerates(t, rendered) - requireTypeNullAcceptance(t, source, moduleDir, "Record", []struct { - value any - valid bool - }{ - {map[string]any{"required_note": nil, "required_child": nil}, true}, - {map[string]any{"required_note": "note", "required_child": map[string]any{"name": "child"}, "optional_count": nil, "optional_tags": nil, "nullable_items": []any{"value", nil}, "nullable_values": map[string]any{"value": nil}}, true}, - {nil, false}, - {"record", false}, - {map[string]any{"required_note": nil}, false}, - {map[string]any{"required_note": 1, "required_child": nil}, false}, - {map[string]any{"required_note": nil, "required_child": "child"}, false}, - {map[string]any{"required_note": nil, "required_child": nil, "nullable_items": []any{1}}, false}, - }) - requireIndependentRegeneration(t, moduleDir) - design := string(rendered) - require.GreaterOrEqual(t, strings.Count(design, "Nullable()"), 5) - require.NotContains(t, design, `Meta("openapi:nullable"`) - require.NotContains(t, design, `Meta("struct:field:type", "loom.Nullable[`) - require.Contains(t, design, "ArrayOf(String, func() {\n\t\tNullable()") - require.Contains(t, design, "MapOf(String, Int, func() {\n\t\tElem(func() {\n\t\t\tNullable()") - require.Contains(t, design, "Null())") - require.Contains(t, design, "Attribute(\"optional_label\", ImportedNullableLabel, func() {\n\t\tNullable()") - require.Contains(t, design, `Example("example-1", Val{"nullable_items": []any{"value", nil}, "nullable_object_values": Val{"value": nil}, "nullable_objects": []any{Val{"name": "loom"}, nil}, "nullable_values": Val{"value": nil}, "required_child": nil, "required_note": nil})`) - require.Contains(t, design, "var ImportedNullableLabel = Type(\"NullableLabel\", String, func() {\n\tMeta(\"openapi:typename:canonical\", \"true\")\n\tNullable()") -} - -func TestExampleLiteralKeepsNestedNullAsGoNil(t *testing.T) { - literal, err := exampleLiteral(map[string]any{ - "items": []any{"value", nil}, - "value": nil, - }) - require.NoError(t, err) - require.Equal(t, `Val{"items": []any{"value", nil}, "value": nil}`, literal) - - literal, err = exampleLiteral(nil) - require.NoError(t, err) - require.Equal(t, "Null()", literal) -} - func requireRenderedDesignEvaluates(t *testing.T, source []byte, wantMethods int, checks ...string) { t.Helper() moduleDir := t.TempDir() diff --git a/internal/openapiimport/schema_examples_test.go b/internal/openapiimport/schema_examples_test.go new file mode 100644 index 000000000..37efc358c --- /dev/null +++ b/internal/openapiimport/schema_examples_test.go @@ -0,0 +1,262 @@ +package openapiimport + +import ( + "encoding/json/v2" + "strings" + "testing" + + "github.com/stretchr/testify/require" +) + +func TestSchemaExamplesSupportedFixtureRoundTrip(t *testing.T) { + var source map[string]any + require.NoError(t, json.Unmarshal(readFixture(t, "supported.json"), &source)) + schemas := source["components"].(map[string]any)["schemas"].(map[string]any) + pet := schemas["Pet"].(map[string]any) + name := pet["properties"].(map[string]any)["name"].(map[string]any) + name["type"] = []any{"string", "null"} + want := []any{nil, "red", "blue", "blue"} + name["examples"] = want + delete(name, "example") + operation := operationFromImportedSpec(t, source, "/pets/{pet_id}", "get") + parameter := operation["parameters"].([]any)[0].(map[string]any) + parameter["schema"].(map[string]any)["examples"] = []any{1, 2, 1} + source["paths"].(map[string]any)["/colors"] = map[string]any{ + "get": map[string]any{ + "operationId": "colors.get", + "responses": map[string]any{ + "200": map[string]any{ + "description": "Colors.", + "content": map[string]any{ + "application/json": map[string]any{ + "schema": map[string]any{"type": "string", "examples": []any{"red", "blue", "blue"}}, + "example": "green", + }, + }, + }, + }, + }, + } + encoded, err := json.Marshal(source, json.Deterministic(true)) + require.NoError(t, err) + document, diagnostics, err := Analyze(encoded) + require.NoError(t, err) + require.Empty(t, diagnostics) + rendered, err := Render(document, Options{PackageName: "design"}) + require.NoError(t, err) + directory := requireRenderedDesignGenerates(t, rendered) + contract := readGeneratedOpenAPIContract(t, directory) + gotPet := contract["components"].(map[string]any)["schemas"].(map[string]any)["Pet"].(map[string]any) + gotName := gotPet["properties"].(map[string]any)["name"].(map[string]any) + require.Equal(t, want, gotName["examples"]) + require.NotContains(t, gotName, "example") + gotOperation := operationFromImportedSpec(t, contract, "/pets/{pet_id}", "get") + foundParameter := false + for _, raw := range gotOperation["parameters"].([]any) { + parameter := raw.(map[string]any) + if parameter["name"] == "limit" { + foundParameter = true + schema := parameter["schema"].(map[string]any) + require.Equal(t, []any{float64(1), float64(2), float64(1)}, schema["examples"]) + } + } + require.True(t, foundParameter) + requireIndependentRegeneration(t, directory) + // Generated document identity and servers are separate unsupported import + // surfaces; the schema examples must survive strict schema analysis. + delete(contract, "$self") + delete(contract, "jsonSchemaDialect") + delete(contract, "servers") + generated, err := json.Marshal(contract, json.Deterministic(true)) + require.NoError(t, err) + second, diagnostics, err := Analyze(generated) + require.NoError(t, err) + require.Empty(t, diagnostics) + for _, operation := range second.Operations { + if operation.Path == "/colors" { + response := operation.Responses[0].Response + require.Len(t, response.Schema.Examples, 3) + require.Equal(t, "red", response.Schema.Examples[0].Value) + require.Equal(t, "blue", response.Schema.Examples[1].Value) + require.Equal(t, "blue", response.Schema.Examples[2].Value) + require.Len(t, response.Examples, 1) + require.Equal(t, "green", response.Examples[0].Value) + return + } + } + t.Error("missing colors operation after second import") +} + +func TestRenderPreservesSchemaAndMediaExamples(t *testing.T) { + source := []byte(`openapi: 3.1.1 +info: {title: Examples, version: "1"} +paths: + /pets: + post: + operationId: createPet + requestBody: + required: true + content: + application/json: + example: {name: Fido} + schema: {$ref: '#/components/schemas/Pet'} + responses: + "200": + description: created + content: + application/json: + examples: + created: + summary: Created pet + description: A newly-created pet. + value: {name: Fido} + schema: {$ref: '#/components/schemas/Pet'} +components: + schemas: + Pet: + type: object + required: [name] + properties: + name: {type: string, example: Fido} + kind: {type: string, examples: [dog, cat]} +`) + + document, diagnostics, err := Analyze(source) + require.NoError(t, err) + require.Empty(t, diagnostics) + rendered, err := Render(document, Options{PackageName: "design"}) + require.NoError(t, err) + requireRenderedDesignEvaluates(t, rendered, 1) + + design := string(rendered) + for _, expected := range []string{ + `Value("Fido")`, + `Example("example-1", func() {`, + `Value("dog")`, + `Example("example-2", func() {`, + `Value("cat")`, + `Meta("openapi:example:scope", "schema")`, + `Meta("openapi:example:scope", "media")`, + `Example("created", func() {`, + `Meta("openapi:example:summary", "Created pet")`, + `Description("A newly-created pet.")`, + `Value(Val{"name": "Fido"})`, + } { + require.Contains(t, design, expected) + } +} + +func TestRenderPreservesNullableSchemas(t *testing.T) { + source := []byte(`openapi: 3.1.1 +info: {title: Nullable, version: "1"} +paths: + /records: + post: + operationId: createRecord + requestBody: + required: true + content: + application/json: + schema: {$ref: '#/components/schemas/Record'} + responses: + "200": + description: record + content: + application/json: + schema: {$ref: '#/components/schemas/Record'} +components: + schemas: + Record: + type: object + required: [required_note, required_child] + examples: + - required_note: null + required_child: null + nullable_items: [value, null] + nullable_values: {value: null} + nullable_objects: [{name: loom}, null] + nullable_object_values: {value: null} + properties: + required_note: + type: [string, "null"] + examples: [null] + optional_count: {type: [integer, "null"]} + required_child: + type: [object, "null"] + properties: + name: {type: string} + optional_tags: + type: [array, "null"] + items: {type: string} + nullable_items: + type: array + items: {type: [string, "null"]} + nullable_values: + type: object + additionalProperties: {type: [integer, "null"]} + nullable_objects: + type: array + items: + anyOf: + - {$ref: '#/components/schemas/Child'} + - {type: "null"} + nullable_object_values: + type: object + additionalProperties: + anyOf: + - {$ref: '#/components/schemas/Child'} + - {type: "null"} + optional_label: {$ref: '#/components/schemas/NullableLabel'} + Child: + type: object + properties: + name: {type: string} + NullableLabel: + type: [string, "null"] +`) + + document, diagnostics, err := Analyze(source) + require.NoError(t, err) + require.Empty(t, diagnostics) + rendered, err := Render(document, Options{PackageName: "design"}) + require.NoError(t, err) + requireRenderedDesignEvaluates(t, rendered, 1) + moduleDir := requireRenderedDesignGenerates(t, rendered) + requireTypeNullAcceptance(t, source, moduleDir, "Record", []struct { + value any + valid bool + }{ + {map[string]any{"required_note": nil, "required_child": nil}, true}, + {map[string]any{"required_note": "note", "required_child": map[string]any{"name": "child"}, "optional_count": nil, "optional_tags": nil, "nullable_items": []any{"value", nil}, "nullable_values": map[string]any{"value": nil}}, true}, + {nil, false}, + {"record", false}, + {map[string]any{"required_note": nil}, false}, + {map[string]any{"required_note": 1, "required_child": nil}, false}, + {map[string]any{"required_note": nil, "required_child": "child"}, false}, + {map[string]any{"required_note": nil, "required_child": nil, "nullable_items": []any{1}}, false}, + }) + requireIndependentRegeneration(t, moduleDir) + design := string(rendered) + require.GreaterOrEqual(t, strings.Count(design, "Nullable()"), 5) + require.NotContains(t, design, `Meta("openapi:nullable"`) + require.NotContains(t, design, `Meta("struct:field:type", "loom.Nullable[`) + require.Contains(t, design, "ArrayOf(String, func() {\n\t\tNullable()") + require.Contains(t, design, "MapOf(String, Int, func() {\n\t\tElem(func() {\n\t\t\tNullable()") + require.Contains(t, design, "Null())") + require.Contains(t, design, "Attribute(\"optional_label\", ImportedNullableLabel, func() {\n\t\tNullable()") + require.Contains(t, design, `Value(Val{"nullable_items": []any{"value", nil}, "nullable_object_values": Val{"value": nil}, "nullable_objects": []any{Val{"name": "loom"}, nil}, "nullable_values": Val{"value": nil}, "required_child": nil, "required_note": nil})`) + require.Contains(t, design, "var ImportedNullableLabel = Type(\"NullableLabel\", String, func() {\n\tMeta(\"openapi:typename:canonical\", \"true\")\n\tNullable()") +} + +func TestExampleLiteralKeepsNestedNullAsGoNil(t *testing.T) { + literal, err := exampleLiteral(map[string]any{ + "items": []any{"value", nil}, + "value": nil, + }) + require.NoError(t, err) + require.Equal(t, `Val{"items": []any{"value", nil}, "value": nil}`, literal) + + literal, err = exampleLiteral(nil) + require.NoError(t, err) + require.Equal(t, "Null()", literal) +} diff --git a/jsonrpc/integration_tests/fixtures/mixedtick/gen/loom.json b/jsonrpc/integration_tests/fixtures/mixedtick/gen/loom.json index c13766b2b..716ec04f3 100644 --- a/jsonrpc/integration_tests/fixtures/mixedtick/gen/loom.json +++ b/jsonrpc/integration_tests/fixtures/mixedtick/gen/loom.json @@ -1,5 +1,5 @@ { "loom_version": "(devel)", "design_package": "example.com/mixedtick/design", - "design_digest": "f9a7f4b1f10eaa0e8a4beee77c76552f9cad19093c3de5f8ced1f103795a1788" + "design_digest": "feb456cf34cf969ed7c306d408c06f7904a0c2706e12072ead5e89f281389369" } diff --git a/jsonrpc/integration_tests/fixtures/ticktock/gen/loom.json b/jsonrpc/integration_tests/fixtures/ticktock/gen/loom.json index e945ef957..a1d00ab41 100644 --- a/jsonrpc/integration_tests/fixtures/ticktock/gen/loom.json +++ b/jsonrpc/integration_tests/fixtures/ticktock/gen/loom.json @@ -1,5 +1,5 @@ { "loom_version": "(devel)", "design_package": "example.com/ticktock/design", - "design_digest": "d2670ad6167054834d8a1ccda36f7bba2f7bf8690023f9479a874a0ab8c1ac38" + "design_digest": "65a3d8c18cb724d8d08073619b0a5000e22633c945ee03a6a8a02391b937744c" }