Solid models with metadata - #328
Open
parrangoiz wants to merge 70 commits into
Open
parrangoiz wants to merge 70 commits into
parrangoiz wants to merge 70 commits into
Conversation
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Keep layer-based deduplication and connected-component splitting as separate passes. Rehash CC subgroups from final contents, update registry and split bookkeeping directly, and assert disjoint terminal and ground PGs.
Make SourceStack the sole owner of source layers and level z coordinates. Remove StackLevels, route rendering and serialization through the stack, and expose the simplified layer_z, thickness, and sourcelayer APIs.
Let Julia infer layer and level dictionary types independently while retaining unitful/unitless validation. Remove manual layer reconstruction and coordinate conversions, and update SolidModelTarget for the relaxed stack parameters.
Keep operation validation in compile_layer_ops and inline the remaining placed-layer checks at the render call site.
Traverse placed metadata once, then pass the collected records into registry construction for validation and registration.
Document the project bang-name rule and rename experimental helpers that mutate the solid model or schematic logging state.
Replace the flat deferred-interface list and separate provenance dictionary with a MetaDiGraph of PG and interface vertices. Cache Gmsh membership and boundary queries per PG, derive Tag and metadata relationships from the graph, reject duplicate destinations, simplify retention, and expand interface execution coverage.
Have Tag and terminal discovery return the metadata needed by later passes, including registered metal connected components and their entity tags. Store interface layer provenance on the deferred graph and derive it during serialization, keep deduplication and CC splitting outside serialization, simplify internal helper names, expand coverage and render-flow comments, and document commit-message guidance.
Rename the layer registry alias, consolidate compiler and interface helpers, and extract locator discovery into locators.jl. Remove obsolete registry and artwork files, colocate schematic rendering and finalization code, simplify locator coordinates and entity matching, add unmatched-locator diagnostics, update tests, and document private-helper naming.
Load SolidModelsExperimental as a top-level DeviceLayout module after its SolidModels and schematic dependencies, and fold the former schematic-integration file into the module entrypoint. Update module references and filenames, narrow the exported API, replace write_metadata with direct JSON output, and refresh tests, documentation, and examples.
Replace the specialized working-schematic reconstruction with a direct deepcopy and consistently name the disposable value sch_copy. Preserve the build and placement-prefix isolation needed for deterministic, non-mutating renders, and update the placement-copy regression coverage.
Add renderer-scoped component geometry caching and recovery so schematic solid-model renders can log failed components, substitute empty geometry, and apply strictness per invocation. Preserve experimental placement prefixing on a private schematic copy, flatten once before rendering, share strictness checks, and make logfile cleanup tolerate already-closed streams while leaving direct low-level renders unchanged.
Build a reverse entity-to-physical-group index for terminal serialization. Cache locator bounding boxes and use per-layer and terminal RTrees to reduce repeated Gmsh containment queries while preserving exact isInside validation.
Avoid redundant reference deep copies and process flattened entity-metadata occurrences directly during schematic rendering. Keep locator metadata intact, enforce unique lumped-port physical-group identities, and remove wording-sensitive exception assertions from the experimental tests.
Process flattened metadata occurrences directly, enforce unique lumped-port identities, simplify locator records, validate model PG registration before terminal discovery, and streamline metadata mapping and tests.
Use concise metadata names throughout the experimental pipeline, centralize physical-group content hashing, and keep operation destinations outside the hash helpers. Preserve schema field names and validate the refactored compiler and rendering paths.
Rename metadata and operation naming helpers, separate hash generation from caller-owned prefixes, and quiet Gmsh across the experimental solid-model tests.
Expose temporary registry and operation printers for compiler debugging, exercise interior-solid flushing through compile_ops, and derive expected PG names from compiler outputs rather than hardcoded naming details.
Make repeated translations execute with unique destination PGs, adopt PG-level copy defaults, and honor copy=false across destination layers. Centralize source-layer validation, standardize added error messages to lowercase, and update compiler coverage.
Replace tuple-based layer operations with immutable operation types and multiple-dispatch compilation. Add grouped boolean operands, typed options, concise constructors, operation documentation, and migrate experimental tests and examples while retaining PG-level tuple output internally.
Simplify compiler state names and access, compile typed operations directly through dispatch, restore operand-level operation hashing, add single-tool Difference syntax, and centralize locator-role checks.
Move optional registry and operation inspection utilities out of the production experimental API and into the compiler test item, where they remain available for debugging.
Standardize public docstrings, document exported role and operation semantics, and clarify metadata, stack, rendering, and source-lifetime behavior. Convert documentation for internal compiler and finalization helpers to comments and correct the conceptual Fuse and Heal guidance.
Move artwork rendering into a dedicated source file, colocate extrusion and exterior-boundary helpers with the compiler, and remove the redundant eager metadata traversal while retaining lazy source-layer validation.
Add OCC-backed Intersect with cross-product PG semantics, explicit-removal lowering, collision rejection, and runtime pruning of unrealized records. Rename operation commands to verbs, remove experimental exports, and update documentation, examples, and tests to use explicit imports.
Exclude source-removal policy from Cut output hashes, reject repeated generated identities, and preserve stateful and explicit composition behavior with focused coverage. Standardize typed-operation error prefixes and operation hash keywords, and document current repeated-operation semantics.
Append one disjoint fused PG when the destination exists outside the source set, while retaining replacement semantics when the destination is an explicit source. Reject repeated generated identities and expand coverage for append, repeat, stateful, chain, and dimension behavior.
Add focused compiler tests for independent collision rejection, stateful in-place reapplication, absorbed-removal collisions, and explicit prefix-preserving Heal chains.
Treat aliased object or tool layers as replaced OCC inputs, reject self-intersections, and preserve explicit removal of the resulting destination. Expand compiler coverage and document the stateful repetition behavior.
Reject repeated and aliased GetInterface destinations, preserve explicit deferred chains, and validate that every deferred input PG identity and dimension remains registered through execution. Update documentation and compiler coverage for the stricter contract.
Reject appending boundary selections to existing unrelated layers while preserving new destinations, in-place dimensional composition, explicit chains, and multi-PG sources. Remove unsafe geometric precedence subtraction and document caller responsibility for disjoint source boundaries.
Reject GetBoundary appends to existing unrelated layers while preserving in-place and chained extraction. Add context-sensitive Translate copy defaults, an in-place shorthand, collision rejection, and explicit copy/move validation with expanded repetition and composition coverage.
Add context-sensitive Translate copy defaults and an in-place shorthand, reject ambiguous out-of-place moves and generated collisions, and add one-layer Revolve construction with repeat and chain coverage. Simplify unary lowering now that all generated collisions are errors.
Reject self-pairing SetPeriodic commands at construction and document the distinct-layer requirement with focused compiler coverage.
Exercise compiled layer operations through the experimental renderer on compact analytical fixtures. Verify realized physical-group cardinality, measures, bounds, source lifetime, deferred interfaces, clipping, and periodic relationships while retaining exhaustive unit-level compiler checks.
Move registry/model consistency and deferred-interface edge cases into the public-render compiler integration suite. Cover duplicate, self, and mixed-dimensional interfaces with realized geometry measures and bounds while removing redundant standalone test items.
Move lumped-port validation and renderer side-effect checks into focused test items, rely on the end-to-end render for schema validation, and remove redundant metadata serialization assertions and the static metadata fixture.
Exercise unitless rendering through the sparse intersection compiler integration fixture and keep renderer warning policy in a focused Strictness modes test. Remove the former ad hoc unitless metadata and file-output assertions.
Physical groups are now opaque geometric partitions identified only by name, dimension, and layer membership. User-facing identity is applied after geometry construction by point locators rather than carried on the geometry itself. - Replace `EntityMeta`/`Role` (entitymeta.jl) with `LayerRef` and the `LocatorMeta` hierarchy `Terminal`, `Ground`, `Tag`, `LumpedPort`, `WavePort` (metadata.jl). Locators are placed as `Locator` point entities and render to no geometry. - Resolve locators in two stages: `resolve_locators` reads positions and port directions from the flat geometry; after rendering, `select!` produces `Selection`s (entity tags plus the locators that chose them) for tag and port surfaces and for metal connected components. - Rewrite deduplication to partition entities by exact PG membership so that every entity belongs to exactly one PG, as Palace requires. This single pass also isolates locator selections and connected components, replacing the layer-signature dedup, the CC split pass, and the temporary selection layers. It runs for all dimensions. - Remove `EntityMeta` from `PGRecord`; source PGs are named after their layer. Interface, boundary, and cut operations no longer promise to preserve or collapse source identities. - Serialize `tagged`, `ports`, `terminals`, and `ground` sections from selections; drop `physical_groups[*].entity_meta` and add `ports` to the schema. - Prefix locator names with the graph node id on a private copy of the schematic during `render!`. - Add variable-naming rules to AGENTS.md; update docs, example, and tests.
The visualization-only PG remap produced a non-Palace-compatible model in which entities could belong to several PGs. Drop it and its end-to-end test tail; visualization will be addressed separately.
The z-range heuristic for overlapping 3D source layers was advisory only and had no test coverage. Overlapping volumes surface as OCC fragmentation failures regardless.
Merge _prune_unrealized_pgs! and _check_pgs_registered into sync_registry!, which prunes unrealized records and errors on unregistered model PGs in one pass. Move it, retained_physical_groups, and deduplicate_pgs! to compiler.jl alongside the registry types they operate on, dropping their underscores. Remove the hasgroup/haskey guards in locators.jl and serialization.jl that the synced registry makes redundant.
execute_deferred_interfaces! becomes realize_interfaces!, pairing with defer_interface! and the registry's realized/unrealized vocabulary. _prefix_placement_names! becomes _qualify_locator_names!, since it qualifies component-local locator names with their graph node id; drop the redundant LayerRef and Ground methods of its helper.
Extrusions are no longer prepended from SourceLayer.thickness; the operation list is the complete construction sequence. - Extrude(source), Extrude(source, dz), Extrude(source; to_level, offset), and Extrude(destination, source, ...) mirror Revolve, accept 1D and 2D sources, and consume the source in place. Source-stack layers build to their declared thickness or level span; explicit distances must agree with the declaration, and check_declared_thicknesses fails rendering if a declared thickness is never extruded. Generated layers require dz or to_level; to_level is resolved from the source geometry when the operation executes and recorded on the destination LayerState. - Hollow(layer) replaces a 3D layer by its shell and, after fragmentation, removes the enclosed volumes and any face left bounding no volume. It runs before interfaces are realized so voided interiors do not contribute to volume boundaries. Replaces the deferred interior-solid subtraction (_flush_interior_solids!, intsol) and EXTBND_MISC. - SourceLayer drops contour_only and keep_interior and renames height to offset; shells are GetBoundary(:x, :x); Extrude(:x). - LayerState gains dz; metadata reports it as thickness for generated layers, and the schema key height becomes offset. - Out-of-place Heal names its result by content hash instead of requiring a layer-prefixed source PG name.
Source physical groups are keyed by layer, so until rendering every
layer holds exactly one PG named after it. The compiler now assumes this
instead of fanning operations out over per-identity PGs.
- LayerState.pgs is a Vector{String}; PGRecord is removed. Each
_compile! lowers one input PG to one output PG named after the
destination via _pg and _create!. A destination must be new or one of
the operation's own inputs; append modes, content-addressed names
(ophash), and their collision checks are gone.
- Heal is removed: it lowered identically to a one-source Fuse, whose
one-layer form now documents in-place healing. Translate loses its
copy field, since in-place moves and a distinct destination copies.
- Deferred interfaces are a Vector{DeferredInterface} instead of a
MetaDiGraph; realization runs in compile order, and the experimental
module no longer imports Graphs or MetaGraphs.
- File-private types are underscored: _CompilerState, _HOLLOWED.
- Compiler unit tests are rewritten for the single-PG semantics, and
repeated-operation-semantics.md reduces to the destination rule.
Julia 1.10 and 1.11 reject assigning a testitem-level variable whose name was already resolved as an import, which failed CI when a local `layer` followed calls to DeviceLayout's `layer`. Rename test variables that shadow exported functions (`layer`, `geometry`, `graph`, `component`, `cell`) throughout the file, and update the unused `inspect_registry` helper for the string-only layer registry.
parrangoiz
force-pushed
the
arrangop/new-sim-interface
branch
from
September 30, 2026 17:47
7071464 to
3fb234f
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR introduces a new solid model rendering pipeline that enables marking special objects in a model -- such as terminals or ports -- and annotating them with metadata. The marked objects are isolated into dedicated physical groups (PGs) and the metadata is returned by
render!in a structured format that can later be used to programmatically generate Palace configurations. By design, the metadata is purely semantic and geometric, and does not include "physics configuration" such as material properties or port impedances. This enables a clean split between CAD+mesh generation and downstream configuration of any Palace simulation from a single source mesh.To enable this, I've added a few new concepts to the public API (all encapsulated in an opt-in module):
SourceLayer. This is a natural extension ofSemanticMeta: it has a name and a level, but also includes a material class (metal, dielectric, or null), a level, and a thickness. It corresponds naturally to a grouping of objects that can be added or removed by a single fabrication step, e.g. when depositing a film or etching pockets. Each source layer can be configured to appear or not appear in its eventual solid model or GDS destination (for the latter, an explicitGDSMetacan be passed). The "source" qualifier refers to the fact that these are the layers into which we'llplace!entities -- subsequent CAD kernel operations can create new ones, which we refer to as "generated" layers.SourceStack. The set of allSourceLayers in a model, keyed by their name, plus a map of level indices to z-coordinates.LayerRef. A simpleSymbolwrapper (and subtype ofDeviceLayout.Meta) used to signal whichSourceLayerwe're placing an entity into.Locator. A point-likeGeometryEntitythat can beplace!d to mark special objects for selection. The entity or set of entities selected is that which contains the point.LocatorMeta. An abstract type for all locator metadata. For now, its concrete types areTerminal,Ground,LumpedPort,WavePort, andTag. The type ofLocatorMetaused toplace!aLocatordetermines the type of selection that is created. In the case ofTerminalandGround, the full connected component of metallic entities is selected.Extrude,Cut,Fuse,Intersect,GetBoundary,GetInterface,Translate,Revolve,Hollow,Remove,RestrictTo, andSetPeriodic. These operate on layers (source or generated) and are lowered to PG-level operations by a simple compiler, which keeps track of layer dimensions, absorbs adjacentRemoveoperations ( simple peephole optimization to make the OCC kernel work more efficiently), and tracks operations that can only be executed after fragmentation (GetInterfaceandHollow). While simple, down the line I expect this compiler to prove useful for simplifying the interface to the OCC kernel and perform more optimizations.SolidModelTarget. ASourceStack+ a set of operations, providing the complete recipe for turning a schematic into a solid model. Fed directly to the renderer viarender!(solid_model, schematic, target). I greedily used the same name for the type that we use in the existing pipeline, as it's encapsulated in the module and not exported, but open to changing it.Under the hood, the thing that makes it possible to create and track selections while offering a simple layer-based interface is a deduplication pass that ensures that every entity in the model belongs to one and only one PG, as this is a critical requirement by Palace and the MFEM mesh data structures. The pass partitions the model into as many PGs as needed while keeping track of their layer memberships.
Changes to existing code
In addition to this new functionality, this PR makes a few minor (and non-breaking) modifications to existing code:
src/schematics/components/components.jl—geometry(comp)can now access a task-local context, providing a cache and an error-recovery callback so that rendering withstrict=:nocan continue past a failing component. Fixes #287.src/schematics/solidmodels.jl—render!now runs inside that context and atry/finallyaround the logfilesrc/schematics/schematics.jl—close_logfiletolerates an already-closed stream so cleanup doesn't mask an earlier failure.src/solidmodels/render.jl—_render_orchestrator!has a new keywordpreflattened=false; the newpipeline flattens the coordinate system once itself (after qualifying locator names by graph node) and passes the result through.
Project.toml— addsJSONandJSONSchemaas test-only dependencies (schema validation of thereturned metadata).
docs/make.jlregisters the module and a concept page.Testing
test/test_solidmodels_experimental.jl: compiler unit tests per operation (lowering,lifetime absorption, rejection cases), Gmsh-backed integration tests with measured
areas/volumes for each operation family including hollowing and interfaces,
locator/port/tag resolution, deduplication invariants (dims 1–3), registry/model
consistency, strictness modes, and a schematic end-to-end test that validates the metadata
against the schema.
Limitations / follow-ups
paths are not yet encoded.
ustrip(STP_UNIT, ::Float64)limitation inextrude_z!).Miscellaneous
AGENTS.mdwith a few style conventions. Feel free to reject it or modify it, although it'd probably be useful to have such a file for the project anyway, even if we end up using a different version.TODO
P.S. sorry for the messy commit history... I took a meandering path to get here. Maybe we just squash when we merge?