Initial new parser implementation - #5751
Open
Robert Johnson (LTA-Thinking) wants to merge 66 commits into
Open
Conversation
…terate - BaseSqlParser: Fix column selection in chain mode to select target resource IDs (RefResourceTypeId/RefResourceSurrogateId) instead of source IDs when ChainLevel > 0 - ChainedSqlParser: Fix source join for multi-level chains, rewrite final CTE assembly to properly join back through ref CTE - ReversedChainSqlParser: Complete rewrite using builder pattern, fix column naming (flip Ref columns for reverse direction), remove IsHistory/IsDeleted check on ReferenceSearchParam table - IncludeSqlParser: Fix JoinMultiLine -> InnerJoin, support iterate includes without Row column dependency - RevIncludeSqlParser: Simplify to match IncludeSqlParser structure, fix IsHistory/IsDeleted check on correct table alias - ParserUtil: Add Row=0 to UNION ALL includes, add includeRow param for iterate dependency unions - ParserOptions: Add IsIterateInclude flag - SqlServerSearchService: Fix async method without await warning Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Fix SortSqlParser.CreateSortCte: remove references to IsMatch/IsPartial/Row columns that don't exist in the source CTE at that point - Fix ORDER BY ORDER BY duplication: CreateOrderByClause no longer includes the 'ORDER BY' prefix since .OrderBy() adds it - Fix Count CTE ordering: use SortValue for ROW_NUMBER() and ORDER BY when a sort CTE exists, instead of always using ResourceTypeId/ResourceSurrogateId - Fix sort pagination: preserve ResourceSurrogateId from continuation token as tiebreaker in sort CTE to prevent duplicate results across pages - Fix special sort (_lastUpdated) pagination: don't clear ContinuationToken when sort is a special parameter (it's needed by base CTE for filtering) - Fix _include/_revinclude without other search params: generate base system CTE when only include parameters are present (prevents null return) - Use hasSortCte flag for reliable SortValue column detection Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Fix ascending sort pagination: set SortQuerySecondPhase=true when continuation token has a SortValue for ascending non-_lastUpdated sorts, preventing phase 1 (missing=true) from re-running on subsequent pages. - Skip two-phase sort logic when a continuation token is already present (not the first page). - Fix UNION ALL column order mismatch when sort + include are combined: use explicit column list to ensure count CTE and include CTE columns align positionally. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Implement NotReferencedSqlParser that generates NOT EXISTS subqueries against the ReferenceSearchParam table to find resources that are not referenced by other resources. Supports: - *:* (wildcard all - no references from any resource/param) - <ResourceType>:* (wildcard param - no refs from specific type) - <ResourceType>:<param> (specific resource type and search param) Invalid formats (no colon separator) are gracefully skipped, letting the SearchOptionsFactory handle the warning. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add ResultCteName property to ParserOptions so chain/reverse-chain
parsers communicate the actual CTE name they produce. Previously,
outer parsers assumed a fixed CTE naming pattern (cte{N}chain{level})
that broke when the inner parser was itself a chain or reverse chain,
which produces cte{N}chain{parent}_search instead.
Updated ChainedSqlParser, ReversedChainSqlParser, and BaseSqlParser
to set/read ResultCteName, fixing:
- Nested reverse chain (_has:...:_has:...)
- Reverse chain with chained search param (_has:...:device.manufacturer)
- Forward chain into reverse chain (link:Patient._has:...)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The CompartmentSqlParser was not applying the continuation token filter, causing paginated compartment searches to always return the same results and never reach the end. Add ResourceSurrogateId and ResourceTypeId filters when a continuation token is present. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Replace Expression tree construction with QueryParams-based approach. Patient search parameters are added directly as query params, and Coverage parameters are converted to _has:Coverage:beneficiary:<param> reverse chain format. This allows the new SQL parser to handle -match without any parser-side changes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When multiple chain or reverse chain searches use the same reference parameter (e.g., _has:Coverage:beneficiary:identifier=xxx and _has:Coverage:beneficiary:type=yyy), the expensive ReferenceSearchParam JOIN Resource CTE is now generated once and reused by all leaf searches in the group. Add ChainSearchGroup class that groups chain parameters by their first-level reference parameter key. Update ChainedSqlParser and ReversedChainSqlParser to accept a sharedRefCteName parameter. Update ParseMultiple to group parameters before generating CTEs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When multiple chains share a ref CTE, each produces its own walk-back CTE independently. Previously only the last walk-back fed into the final query, silently discarding earlier results. Now an intersection CTE is generated after all entries in a shared group, INNER JOINing all walk-back CTEs to enforce AND semantics. Single-chain groups are unaffected (no intersection CTE added). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Create SmartCompartmentSqlParser that generates UNION-based CTEs for: 1. Resources in user's compartment (via ReferenceSearchParam) 2. User's own resource 3. Universal resources (Location, Organization, Practitioner, etc.) - Add _smartCompartmentType/_smartCompartmentId to QueryParams in SearchOptionsFactory when AccessControlContext has a SMART compartment - Add _fhirScopeAllowedTypes to QueryParams for fine-grained access control resource type restrictions - SearchParameterSqlParser handles new params: generates SMART compartment CTE as base filter, applies allowed type restrictions Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Use Microsoft NuGet proxy for restore Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Keep NuGet feeds in local config Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Support environment-specific NuGet sources Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Robert Johnson (LTA-Thinking)
added
Enhancement
Standardize all search parsers to output ResourceTypeId before ResourceSurrogateId, matching the order used by IncludeSqlParser, RevIncludeSqlParser, and ParserUtil.AddUnionCte. The positional mismatch caused UNION ALL to swap columns, breaking all include results. Also remove dead expression-tree code that prevented compilation of SearchIncludeImpl. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
ReferenceSqlParser now applies the type modifier (e.g., :Practitioner) when filtering by reference target type. SearchParameterSqlParser now validates total parameter count and throws RequestNotValidException when exceeding 2048 values. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When multiple _has: parameters share the same resource type and reference parameter (e.g., MemberMatch coverage conditions), combine them into a single CTE with multiple JOINs instead of N sequential CTEs. This matches the old stored procedure behavior where all conditions were ANDed in one query, allowing SQL Server to use index intersection. Falls back to sequential chaining for entries with :missing/:not modifiers or non-standard parsers. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds 151 unit tests across 17 test files covering: - Infrastructure: SqlQueryBuilder, QueryStringParser, ChainSearchGroup, ParserUtil - Base parsers: Token, String, DateTime, Number, Quantity, Reference, Uri - Composite parsers: TokenString, TokenToken, TokenDateTime, TokenQuantity, TokenNumberNumber, ReferenceToken - Special parsers: Id, LastUpdated, System, Sort Tests verify SQL generation, WHERE clause building, parameter parsing, CTE construction, and edge cases (escaping, modifiers, empty values). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+444
to
+452
| if (allowedTypes.Any()) | ||
| { | ||
| searchOptions.QueryParams["_fhirScopeAllowedTypes"] = allowedTypes; | ||
| } | ||
| else | ||
| { | ||
| // No resource types allowed — block all queries | ||
| searchOptions.QueryParams["_fhirScopeAllowedTypes"] = new List<string> { "none" }; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Removes the Expression based SQL parser and replaces it with one that goes directly from query parameters to SQL queries.
Related issues
Addresses User Story 199220
Testing
Existing E2E tests are utilized heavily to ensure feature parity.
FHIR Team Checklist
Semver Change (docs)
Patch|Skip|Feature|Breaking (reason)