Feat/binary encodings and modularization#38
Merged
Conversation
…I generated and will be manually reviewed as time permits
…prove overall doc coverage of streamableresource.py
…_/from_swejson_dict, to_/from_geojson_dict, and to_/from_csapi_dict methods across System/Datastream/ControlStream and their underlying pydantic resource/schema models for round-tripping CS API server JSON, deprecated the older System.from_system_resource and Datastream.from_resource factories, and fixed three latent bugs (Node._client_session initialization, TimeUtils.time_to_iso UTC handling, ObservationOMJSONInline alias direction) exposed by the new tests.
…endency to main publish workflow so we are less likely to publish a broken build
…ing the workflow to execute on pushes
…used with underlying object representations that are needed for actual wire serialization. Update docs to reflect this.
update related docs
…dd networked tests to verify that commands can be sent based off source schemas and maintain coherence across the wire.
… the same process.
…y of the base consysapi object by subclassing based on request type
… the properties were not honored due to only accepting the default value for the type property
… account for this. Add a SchemaFetchWarning to alert users when this fails so it doesn't get missed.
…unions and forcing rebuilds on the models to fix a forward ref to AnyComponent
…uf and flatbuffers
Work-in-progress snapshot pushed to local Forgejo for backup/sharing. Includes csapi4py module pruning, swe+binary/protobuf work, resource split, and test updates.
Add a NATS.io pub/sub transport as a drop-in twin of the MQTT path, targeting the osh-nats-dev consys-nats server, plus a working FlexBuffers-based swe+flatbuffers codec. NATS transport: - NatsCommClient (csapi4py/nats.py): synchronous interface matching MQTTCommClient, backed by an asyncio loop on a background thread; qos/retain accepted-and-ignored for parity; clean shutdown. - Node(enable_nats, nats_port, nats_token) mirrors enable_mqtt; resources select the active transport via get_comm_client(). - Nested-under-systems subjects (StreamableResource.get_nats_subject), matching ConSysApiNatsConnector.getResourceUri exactly (verified live). - PROACTIVE format handling: PULL subscribes to a format wildcard (:data.*), reads the concrete format per-message from the delivered subject (nats_content_type_from_subject), and decodes via decode_observation(raw, obs_format=...). Publish keeps the exact format. - nats-py added as a core dependency. swe+flatbuffers: - OSH's wire format is schemaless length-prefixed FlexBuffers, not compiled FlatBuffers, so the flatc vector-of-union limitation never applied. SWEFlatBuffersCodec now encodes/decodes real frames. - Fix: swe-flatbuffers was missing from MQTT_TOPIC_FORMAT_TOKENS. Tests: test_nats_subjects.py (subjects, wildcard, format extraction, transport dispatch), rewritten test_swe_flatbuffers.py for the real codec. Docs: tutorial, API ref, spec-deviations #3 (resolved) and #5.
…lity
Older OSH servers predate the CS API Part 3 topic scheme. Add an opt-in
legacy mode that reverts MQTT topic construction to the pre-Part-3 form:
a leading slash, no ':data' suffix, and no format subtopic
(e.g. /api/datastreams/{id}/observations vs
api/datastreams/{id}/observations:data/<token>).
- APIHelper gains a `legacy_topics` field and a per-call `legacy=` override
on get_mqtt_topic(); in legacy mode data_topic/format are ignored.
- Node(mqtt_legacy_topics=True) flows the flag into its APIHelper.
- Resources are unaware — they call get_mqtt_topic as before and get the
legacy strings transparently, so publish and subscribe both use them.
- Legacy mode affects MQTT topics only; NATS subjects are unaffected.
Tests: TestLegacyTopics covers legacy strings for datastream/controlstream/
status/system, init_mqtt wiring, off-by-default, and per-call override.
Docs: tutorial note by the MQTT section.
…to tutorial Explain that OSHConnect is a client requiring a running OSH node with the Connected Systems API service (and the CS API MQTT module + a broker for real-time streaming), linking the official OpenSensorHub docs rather than duplicating setup instructions. Add a table mapping the information to gather from a node to the corresponding Node() constructor parameters, with a worked example.
Collaborator
Author
|
We have investigated ourselves and found no wrongdoing |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.