Repository navigation
Conversation
| // newer state of the other would delete a live fault from a client's view. | ||
| auto supersede_key = [](const QueuedEvent & queued) { | ||
| return queued.peer + '\0' + queued.event.fault.fault_code; | ||
| const auto & sources = queued.event.fault.reporting_sources; |
There was a problem hiding this comment.
on_peer_eventreads onlyfault_codeback out of a relayed payload, neverreporting_sources, so for every relayed entry owneris empty here and the key collapses to (peer, code): a peer's two records of one code still coalesce into one, and a lagging client loses one owner's last update uncounted, which is exactly what this commit removes for local events. Readfault.reporting_sources(orsource_id) back into queued.event.faultinon_peer_eventand cover it with a relayed variant ofAnotherOwnersEventsDoNotSupersedeARecordsLastUpdate`.
| // shows first, then the muted ones, then the cleared or healed ones, so | ||
| // neither a muted record nor one a source cleared turns the record a client | ||
| // read off the list into an ambiguous address. | ||
| auto result = fault_mgr->list_faults("", /*include_prefailed=*/true, /*include_confirmed=*/true, |
There was a problem hiding this comment.
Every per-code GET and DELETE now pulls the whole store (all statuses, muted included) through ListFaults before the one GetFault it used to make, and the plugin clear branch and the recording download repeat the same listing, so with cleared records retained indefinitely each request costs O(history) on routes a stream consumer hits once per event. Call get_fault_with_env(fault_code, "") first, which the manager answers only when the code names exactly one record, scope-check that record, and take the full listing only when that call does not succeed: the tiers only matter with two or more records, so nothing observable changes.
| auto held = fault_mgr->list_faults("", /*include_prefailed=*/true, /*include_confirmed=*/true, | ||
| /*include_cleared=*/true, /*include_healed=*/true, | ||
| /*include_muted=*/true, /*include_clusters=*/false); | ||
| if (!held.success) { |
There was a problem hiding this comment.
Plugin DELETE now returns 503 when ListFaults fails, but plugin GET still falls through to the provider in that case. On a plugin-only box (UDS DTCs, no fault_manager node) read works and clear of the same DTC 503s. Make both paths agree: fall through with an empty owner here, or 503 in get_fault too.
| make_error(409, ERR_AMBIGUOUS_FAULT, "Fault code addresses several records in this entity", | ||
| json{{"details", | ||
| "Several sources this entity owns report this fault code, and each is its own record. " | ||
| "parameters.owners names them. Address one through the route of the app that owns it, " |
There was a problem hiding this comment.
The hint points at /apps/{app_id}/faults/{fault_code}, but owners are reporting sources: a node FQN for a ROS app, and a bare component id for an external component, which has no app route. If an external component and one of its apps report the same code, the component's own record cannot be read or cleared by code at all. Return each owner's entity route (build_source_entity_map already resolves it), or let the caller pick the owner.
| auto held = fault_mgr->list_faults("", /*include_prefailed=*/true, /*include_confirmed=*/true, | ||
| /*include_cleared=*/true, /*include_healed=*/true, /*include_muted=*/true, | ||
| /*include_clusters=*/false); | ||
| const json listing = held.success ? held.data : json::object(); |
There was a problem hiding this comment.
A failed list_faults becomes an empty listing, so requested_records and candidate_owners are empty and a bag this entity owns answers 404 "not found for this entity". Return classify_fault_failure (503) on !held.success, like the fault routes, and declare it on the download routes.
| // shows first, then the muted ones, then the cleared or healed ones, so | ||
| // neither a muted record nor one a source cleared turns the record a client | ||
| // read off the list into an ambiguous address. | ||
| auto result = fault_mgr->list_faults("", /*include_prefailed=*/true, /*include_confirmed=*/true, |
There was a problem hiding this comment.
Every per-code GET/DELETE now lists the whole store (all entities, cleared and healed history, muted) and then does a GetFault, both under the transport's executor mutex. Cost grows with retained history. List per scope source (list_faults(source)) or add a fault_code filter to ListFaults. Same full read in the plugin clear (997), download (494) and descriptor listing (376).
| } | ||
| } | ||
| if (lock_mgr) { | ||
| auto it = source_to_entity.find(owner); |
There was a problem hiding this comment.
Exact find(owner) misses path-child sources, which the per-entity scope gives to the parent app. A record owned by /ns/app/child is cleared through a lock on /ns/app, contrary to the new locking.rst line. Match with the same rule as source_matches_scope (exact or strict path child).
| { | ||
| auto faults_result = fault_mgr->list_faults("", /*include_prefailed=*/true, /*include_confirmed=*/true, | ||
| /*include_cleared=*/true, /*include_healed=*/true, | ||
| /*include_muted=*/false, /*include_clusters=*/false); |
There was a problem hiding this comment.
include_muted=false drops muted records from faults, so a muted record's legacy rows (created_at_ns == 0) get dated 0 although download serves them. Use include_muted=true, as download does.
| /// The reporting source that owns this record. Together with `fault_code` it | ||
| /// addresses the record on the per-record routes, and it is the single entry | ||
| /// of `reporting_sources`. | ||
| std::optional<std::string> source_id; |
There was a problem hiding this comment.
A list item now has top-level source_id = owner and x-medkit.source_id = namespace path, while the detail calls the owner x-medkit.owner to avoid exactly that clash. Use owner here too.
8fba96c to
547c411
Compare
72b93ed to
b2e3c29
Compare
The gateway carries the record owner through every per-record call and resolves a per-entity fault route to the record the entity owns, answering 409 when the code names several records in its scope. Bulk-data recording downloads, plugin clears, fault triggers, log triggers, entity locks and the fault event stream act on one record. Sub-path sources belong to the entity that owns the path. Plugin API version 8 adds FaultProvider::clear_fault_record.
The rosbag listing reads the fault list to find the owners in scope, so the fake transport now lists one confirmed record per fault it holds. The listing test also checks that an unreadable fault list answers 503, and the REST reference says the listing reads the fault list first.
The status route answered only for the node map's current component id. After the plugin re-derives that id from the device, /components lists the previous id until the next discovery refresh, and the route answered 404 for it. The route now also accepts the ids the last two introspect() calls returned. A component the plugin does not expose still gets 404.
shutdown() joins the plugin's fault-client executor and drops its clients with their pending requests. The test now holds a clear in flight, shuts the plugin down and destroys it, then has the stub answer: the plugin's sink receives nothing. A refusal answered while the plugin lives, and a control request answered after the dropped reply, show the harness sees a reply that is delivered.
Two sources reporting one code are two records only when the fault manager keeps that code per_source. The README now says so where it explains per-node records and per-record recordings.
The freeze-frame overlap tests hold the standing-fault catch-up until a second subscriber on the capture's own feed has seen the confirm, which the feed delivers after the capture has queued it, and wait for the live frame instead of a fixed delay. The logs trigger tests wait for the gateways to acknowledge the published entries, pace the real entries on the entity listings, and close each negative check with an entry logged after the others: a trigger delivers in log order, so once that entry arrives nothing earlier can still reach it.
An app shows the record of its source and the records of its sub-sources as one entry, and a component, area or function every record of the code under it. An entry of several records is their merge: the most advanced status, the highest severity, the summed occurrence count, the earliest first and latest last occurrence, the union of reporting sources, and the description and environment data of the confirmed record that occurred last. x-medkit.records names each record of an entry, its app, source and status. A shared record names no owner anywhere and looks as it did before records had owners. GET and DELETE of a code act on every record of it under the entity, and the DELETE clears each with its own owner; neither answers 409 any more. A per-entity clear that cannot clear a record answers 500 with failed_codes. The global fault list shows one item per record. The gateway's fault data carries the record owner from Fault.owner, "*" for a shared record, and addresses the record with it. The entity frames and the global clear's lock check still go by the record's reporting source.
list_entity_faults fills Fault.owner from the gateway's fault data, as the fault manager sets it: the reporting source of a per_source record, "*" for a shared one. A peer gateway's item names a per_source record's owner as source_id, which is used when no owner is given.
A fault list item has no top-level source_id. SOVD keeps vendor fields inside x-medkit, so a per_source record names its app and source only in x-medkit.records. FaultListItem drops the field and the published schema follows. The detail of one per_source record still names its owner as x-medkit.owner, now taken from its single x-medkit.records entry. sovd_service_interface reads the owner of a peer gateway's item from that entry too. The README fault list examples show the wire shape: items, x-medkit, reporting_sources and a fault status.
…reference rclcpp's AnyServiceCallback::set() stores a callable whose argument list is not SharedPtrCallback's by assigning it to its callback variant. A callable of (const std::shared_ptr<Request> &, const std::shared_ptr<Response> &) converts to SharedPtrCallback and to no other alternative. The Humble, Jazzy and Lyrical headers share that code, and the fault manager registers its services the same way. The handlers take the same references.
The bands are uint16_t, so the comparison has no NaN case and the simplified form clang-tidy asks for is equivalent.
…a constructor The guard deletes its copy and move constructors, so under C++20 it is no aggregate and the brace initialisation finds no constructor. Lyrical's toolchain compiles the gateway tests as C++20 and stopped there. A constructor taking the three references keeps the guard and what the test pins unchanged on every distro.
…mble Humble's rcl_interfaces declares the Log severity constants as byte, so rclpy holds them as bytes and refuses them for the uint8 level field; later distros declare them uint8. Every test publishing a /rosout entry failed on its first publish on Humble. The constants now pass through a helper that returns the integer on every distro.
Each demo node logs one INFO line at startup. Whether the gateway gets it depends on whether its /rosout subscription matched the node first, and when it does the line is an entry of the app, so the restored trigger fires for it and writes one more marker than the test counts. The nodes now run at --log-level warn and publish nothing, so the test's entries are the only entries of each app.
A json returned where the function declares tl::expected<json, ...> or std::optional<json> converts either through that type's converting constructor or through json's own conversion operator. gcc 11 on Humble warns under -Wconversion about the choice it makes. Constructing the return type explicitly leaves one candidate, so the build prints no warning from these lines; the value returned is the same.
…itten The marker count was read as soon as two markers were listed, so a marker written after that read went unseen. The test now waits for the marker of the last entry it logs: one notifier worker dispatches entries in log order and writes an entry's markers before the next, so after that marker no trigger writes another. It then requires exactly three markers under the root app, none under the namespaced app, and no delivered event for an entry the test did not log.
A trigger's event stream wakes only for an event, the trigger's removal or its 15 s keepalive. Closing the client side blocked until the next keepalive, once per stream, and the gateway's shutdown then waited for the same wake-up: about 45 s of the test's 61 s. The test now removes both triggers, waits until the gateway has ended each stream, and requires that it does.
Closing the /faults/stream response in test_08 blocked until the next keepalive, 30 s by default. The gateway of this test sends one every second, the minimum of sse.keepalive_interval_sec.
Seven fixed spins (14.5 s a run) stood in for "sweeps have run" and "nothing more arrives". The test now runs the orphan sweep and restore replay itself, the code the refresh timer runs, and closes each negative check with a later entry: a trigger delivers in log order, so once that entry arrives nothing logged before it can still reach the trigger. The checks are exact: each trigger's events are the entries in log order, one marker per firing.
test_08 read frames of the code test_077 had already raised under both owners, so a frame for owner-b arrived even when test_08 never reported from owner-b. It now reports a per_source code nothing else raises, so every frame it counts comes from its own reports.
…r 1 s The stalled fault manager case waits out fault_manager.service_timeout_sec on every call it makes, 5 s each at the default. The test sets 1 s, scaled by the sanitizer time factor.
The scope notification and rosbag listing tests primed their report client and not their clear client, and the markers test made a new report client for its one report. Each client now gets one answer from answer_once_each before the test sends what it checks; the rosbag listing test's handshake report goes. The markers report is still sent once, since a second CRITICAL report would fire the faults trigger again.
… lifecycle client Each GetState and ChangeState call made a new client and sent one request, and a reply can be lost while the lifecycle node has not matched that client. GetState is now sent until answered. A ChangeState client first gets an answer to the default request, transition 0, which a running node refuses without changing state, and then sends its one transition.
b2e3c29 to
8a5276b
Compare
Pull Request
Summary
The gateway side of per-source fault records. #699 lets a fault code keep one record per reporting source
(
per_source: trueon the code); this PR shows those records through the SOVD fault routes. In every entity a faultcode appears once:
<app fqn>/<suffix>)show as one entry.
highest severity, the summed occurrences, the earliest and latest occurrence, and
x-medkit.recordswith eachrecord's App, source and status. A merge of one record is that record unchanged.
GET {entity}/faults/{code}returns that entry, with the environment data of the record confirmed last.DELETE {entity}/faults/{code}clears every record of the code under the entity.GET /api/v1/faultslists one item per record.DELETE {entity}/faultsanswers 500 withfailed_codeswhen a record could not be cleared. Before, it answered 204and logged a warning.
Also in this PR:
of its records' sources.
FaultProvidergainsclear_fault_record(entity_id, fault_code, owner); its default callsclear_fault(entity_id, fault_code). Plugin source written against version 7 compiles unchanged; a pre-builtversion 7 binary is rejected by the loader.
sovd_service_interface: each listed Fault carries the owner of its record.entity fires.
Stacked on #699. Review that one first.
Issue
Type
Testing
src/ros2_medkit_integration_tests/test/features/test_faults_owner_identity.test.pyruns a real gateway and faultmanager: a shared code reported by two Apps, a
per_sourcecode from two Apps and from two sub-sources of one App,and the entity, global and per-code routes for each.
Checklist