RestIngestor refactor - #2130
Draft
aestoltm wants to merge 22 commits into
Draft
Conversation
…for a single nested level currently)
…ult key per field map entry. Also fixed a few possible bugs
aestoltm
added
enhancement
jpwhite4
reviewed
Mar 10, 2026
| } // verifyTransformDirective() | ||
|
|
||
| /* ------------------------------------------------------------------------------------------ | ||
| * Parses PHP obj/array and retrieves desired field. Supports a wildcard, "?", for the last item in an array. |
Member
There was a problem hiding this comment.
What is the use case for adding support for the 'last item in array' wildcard?
Contributor
Author
There was a problem hiding this comment.
Originally was using this to get the latest conversion factors from the cider api (last element). However, the cider pipeline is going to create a history of the conversion factors so I do not actually use it anymore. I can't think of any other cases so we probably can remove it.
Member
There was a problem hiding this comment.
I think it is better to not add it for now. If we need a capability in future then should be easy to add it in based on this work.
aestoltm
marked this pull request as draft
April 2, 2026 20:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The
RestIngestorwill now extend theStructuredFileIngestorand save the JSON results to a tmp directory (e.g./www/scratch/rest-ingestor/) before being ingested through aDirectoryScannerdata endpointMotivation and Context
Large refactor to
RestIngestor.Tests performed
Tested with CiDER API (Operations API)
Checklist: