docs+fix: update documentation and API responses for post-0.3.0 changes#258
docs+fix: update documentation and API responses for post-0.3.0 changes#258
Conversation
…nodes section The param names in the launch/YAML/CLI examples were already correct (discovery.mode, discovery.manifest_path), so no changes needed there. Replaced the "Handling Unmanifested Nodes" section which documented the nonexistent config.unmanifested_nodes parameter (with ignore/warn/error/ include_as_orphan policies) with "Controlling Gap-Fill in Hybrid Mode" documenting the actual discovery.merge_pipeline.gap_fill.* parameters. Added a note block to the Runtime Linking section explaining the layered merge pipeline architecture.
There was a problem hiding this comment.
Pull request overview
Updates Sphinx documentation to reflect gateway behavior after post-0.3.0 changes (logging, hybrid discovery merge pipeline, and plugin providers), keeping tutorials and reference docs aligned with current configuration and APIs.
Changes:
- Refresh plugin system tutorial to cover
LogProviderand updatedIntrospectionProviderwiring. - Replace outdated hybrid/manifest discovery guidance with merge-pipeline gap-fill configuration docs.
- Extend server config reference with logging buffer and rate limiting options; update REST API examples.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/tutorials/plugin-system.rst | Updates provider list and plugin export examples (adds LogProvider). |
| docs/tutorials/manifest-discovery.rst | Replaces unmanifested-node section with hybrid merge-pipeline gap-fill guidance. |
| docs/config/server.rst | Adds logs.buffer_size and rate_limiting.* configuration reference sections. |
| docs/config/discovery-options.rst | Adds merge pipeline concepts (field groups, policies, gap-fill) and health endpoint example. |
| docs/api/rest.rst | Updates GET /api/v1/ server capabilities example response. |
docs/tutorials/plugin-system.rst
Outdated
| // Required if your plugin implements LogProvider: | ||
| extern "C" GATEWAY_PLUGIN_EXPORT LogProvider* get_log_provider(GatewayPlugin* p) { | ||
| return static_cast<MyPlugin*>(p); | ||
| } | ||
|
|
There was a problem hiding this comment.
In the example, MyPlugin doesn’t inherit from LogProvider, but the snippet adds #include .../log_provider.hpp and shows exporting get_log_provider() returning MyPlugin*. That’s confusing and suggests an invalid implementation. Either update the example class to also implement LogProvider (and mention required methods), or remove the LogProvider include / get_log_provider() export from this minimal UpdateProvider example.
| // Required if your plugin implements LogProvider: | |
| extern "C" GATEWAY_PLUGIN_EXPORT LogProvider* get_log_provider(GatewayPlugin* p) { | |
| return static_cast<MyPlugin*>(p); | |
| } |
There was a problem hiding this comment.
Fixed in 80091bc - removed LogProvider include/export from the MyPlugin example and clarified IntrospectionProvider scope.
docs/api/rest.rst
Outdated
| @@ -25,11 +25,14 @@ Server Capabilities | |||
|
|
|||
| { | |||
| "api_version": "1.0.0", | |||
| "gateway_version": "0.1.0", | |||
| "gateway_version": "0.3.0", | |||
| "endpoints": [ | |||
| {"path": "/areas", "supported_methods": ["GET"]}, | |||
| {"path": "/components", "supported_methods": ["GET"]}, | |||
| {"path": "/apps", "supported_methods": ["GET"]} | |||
| {"path": "/apps", "supported_methods": ["GET"]}, | |||
| {"path": "/functions", "supported_methods": ["GET"]}, | |||
| {"path": "/faults", "supported_methods": ["GET", "DELETE"]}, | |||
| {"path": "/updates", "supported_methods": ["GET", "POST"]} | |||
| ] | |||
There was a problem hiding this comment.
The documented GET /api/v1/ "Server Capabilities" example response schema doesn’t match the actual root handler output. In code, /api/v1/ returns {name, version, api_base, endpoints: ["GET /api/v1/..."], capabilities, ...} (see HealthHandlers::handle_root), not {api_version, gateway_version, endpoints: [{path, supported_methods}]}. Please update the example JSON (and field names like gateway_version) to match the current response format (and include updates/logs endpoints only if they actually appear there).
There was a problem hiding this comment.
Fixed in 68b3f93 - updated the root endpoint example to match the actual handle_root output with name, version, api_base, flat endpoints array, and capabilities object.
| ``GET /api/v1/health`` includes a ``discovery`` section in hybrid mode with | ||
| pipeline stats, linking results, and merge warnings: | ||
|
|
||
| .. code-block:: json | ||
|
|
||
| { | ||
| "status": "healthy", | ||
| "discovery": { | ||
| "mode": "hybrid", | ||
| "strategy": "HybridDiscoveryStrategy", | ||
| "pipeline": { | ||
| "layers": ["manifest", "runtime", "plugin"], | ||
| "entity_source": {"lidar-driver": "manifest", "orphan_node": "runtime"}, | ||
| "conflicts": [] | ||
| }, | ||
| "linking": { | ||
| "linked_count": 5, | ||
| "orphan_count": 1, | ||
| "binding_conflicts": 0 | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
This health endpoint example doesn’t match the current /api/v1/health response. In code, discovery.strategy is the strategy name string (e.g. "hybrid"), and discovery.pipeline is MergeReport::to_json() (keys like layers, total_entities, enriched_count, conflicts, id_collisions, filtered_by_gap_fill)—it does not include an entity_source map. Please update the example to reflect the actual JSON fields returned by HealthHandlers::handle_health() / MergeReport::to_json().
There was a problem hiding this comment.
Fixed in 80091bc - health endpoint example now matches MergeReport::to_json() output: total_entities, enriched_count, conflict_count, conflicts, id_collisions, filtered_by_gap_fill. Strategy name corrected to "hybrid".
docs/config/discovery-options.rst
Outdated
| - is_online, health, runtime state | ||
| * - ``metadata`` | ||
| - source, category, custom metadata fields |
There was a problem hiding this comment.
The Field Groups table looks out of sync with the implementation. In merge_types.hpp / merge_pipeline.cpp, status is primarily is_online + bound_fqn (and external for apps); there is no merged health/"runtime state" field group, and metadata is described as source + vendor/custom fields (not category). Please adjust the table entries for status/metadata to match what the merge pipeline actually groups and merges today.
| - is_online, health, runtime state | |
| * - ``metadata`` | |
| - source, category, custom metadata fields | |
| - is_online, bound_fqn (and external for apps) | |
| * - ``metadata`` | |
| - source plus vendor/custom metadata fields |
There was a problem hiding this comment.
Fixed in 80091bc - Field Groups table updated: STATUS = is_online, bound_fqn; METADATA = source, x-medkit extensions, custom metadata fields.
| entities appear. This is effectively the same as ``manifest_only`` mode but | ||
| with the benefit of runtime data (topics, services) on manifest entities. |
There was a problem hiding this comment.
The statement that disabling all allow_heuristic_* options is effectively the same as manifest_only but with runtime data appears incorrect. In hybrid mode, runtime linking and runtime resource enrichment depend on the runtime layer producing runtime_apps for RuntimeLinker::link(); if allow_heuristic_apps is false, the runtime layer won’t output apps and manifest apps won’t be linked/enriched. Please reword this to avoid implying you still get runtime topics/services/status when gap-fill is fully disabled, or document the exact limitations.
| entities appear. This is effectively the same as ``manifest_only`` mode but | |
| with the benefit of runtime data (topics, services) on manifest entities. | |
| entities appear and runtime gap-fill is fully disabled. In this configuration, | |
| hybrid mode behaves effectively like ``manifest_only``: manifest apps are not | |
| linked to runtime nodes, and you will not see runtime-derived topics, services, | |
| or status on those entities unless you re-enable heuristic apps. |
There was a problem hiding this comment.
Fixed in 80091bc - corrected the description. Disabling all heuristics means the gateway only creates entities explicitly declared in the manifest; runtime still provides linking and resource data.
docs/tutorials/plugin-system.rst
Outdated
| - **IntrospectionProvider** - enriches discovered entities with platform-specific metadata | ||
| via the merge pipeline. In hybrid mode, each IntrospectionProvider is wrapped as a | ||
| ``PluginLayer`` and added to the pipeline with ENRICHMENT merge policy. | ||
| and can introduce new entities. Called during each discovery cycle by the merge pipeline's | ||
| PluginLayer. See :doc:`/config/discovery-options` for merge pipeline configuration. | ||
| - **LogProvider** - replaces or augments the default ``/rosout`` log backend. |
There was a problem hiding this comment.
The IntrospectionProvider bullet says it "enriches discovered entities with platform-specific metadata", but IntrospectionResult::metadata is explicitly documented/implemented as plugin-internal data that is not merged into entities by MergePipeline (it’s ignored there and intended to be exposed via plugin routes + registered capabilities). Consider clarifying here that metadata is not automatically added to discovery entity responses unless the plugin exposes it via its own endpoints/capabilities.
There was a problem hiding this comment.
Fixed in 80091bc - clarified that IntrospectionProvider metadata is plugin-internal data accessible via the plugin API, not automatically merged into entity responses.
…info format
- Add missing endpoint categories to handle_root: logs, bulk-data,
cyclic-subscriptions, updates (conditional), DELETE /faults (global)
- Remove ghost snapshot endpoints (listed but never registered)
- Add missing capabilities: logs, bulk_data, cyclic_subscriptions, updates
- Fix hardcoded version "0.1.0" -> "0.3.0" in handle_root and version-info
- Change version-info response key from "sovd_info" to "items" (SOVD standard)
- Add bulk-data, logs, cyclic-subscriptions URIs to entity capability responses
- Update rest.rst: fix Server Capabilities example format, remove phantom
/manifest/status, document DELETE /{entity}/faults, update SOVD compliance
section, add areas/functions resource collection notes
- Update tests, integration tests, and Postman collection for sovd_info->items
docs/api/rest.rst
Outdated
| ``/configurations``, ``/faults``, ``/bulk-data``. See the corresponding sections below. | ||
|
|
There was a problem hiding this comment.
This text claims areas support /bulk-data, but BulkDataHandlers currently rejects BULK_DATA for AREA via EntityCapabilities (and the area discovery response also doesn’t expose a bulk-data URI/capability). Either document /bulk-data as unsupported for areas, or update the gateway capabilities/validation so area bulk-data requests succeed.
| ``/configurations``, ``/faults``, ``/bulk-data``. See the corresponding sections below. | |
| ``/configurations``, ``/faults``. Bulk data (``/bulk-data``) is currently not available for | |
| areas. See the corresponding sections below. |
There was a problem hiding this comment.
Fixed in 80091bc - removed bulk-data references from areas and functions resource collection notes in rest.rst.
docs/api/rest.rst
Outdated
| Functions also support ``/data``, ``/operations``, ``/configurations``, ``/faults``, | ||
| and ``/bulk-data``. See the corresponding sections below. |
There was a problem hiding this comment.
This text claims functions support /bulk-data, but EntityCapabilities::for_type(FUNCTION) currently only allows DATA/OPERATIONS and BulkDataHandlers validates collection support before serving GET /functions/{id}/bulk-data. Either remove /bulk-data from the docs for functions or update the gateway so function bulk-data is actually supported.
| Functions also support ``/data``, ``/operations``, ``/configurations``, ``/faults``, | |
| and ``/bulk-data``. See the corresponding sections below. | |
| Functions also support ``/data`` and ``/operations`` (aggregated across hosting apps). | |
| See the corresponding sections below. |
There was a problem hiding this comment.
Fixed in 80091bc - removed bulk-data from functions resource collection notes.
| TEST_F(HealthHandlersTest, HandleVersionInfoContainsSovdInfoArray) { | ||
| handlers_.handle_version_info(req_, res_); | ||
| auto body = json::parse(res_.body); | ||
| ASSERT_TRUE(body.contains("sovd_info")); | ||
| ASSERT_TRUE(body["sovd_info"].is_array()); | ||
| EXPECT_FALSE(body["sovd_info"].empty()); | ||
| ASSERT_TRUE(body.contains("items")); | ||
| ASSERT_TRUE(body["items"].is_array()); | ||
| EXPECT_FALSE(body["items"].empty()); |
There was a problem hiding this comment.
The test name still references SovdInfo even though the response key was changed to items. Renaming the test(s) to match the current API terminology will make failures easier to interpret and keep the suite consistent.
There was a problem hiding this comment.
Fixed in 94818bb - renamed all 3 tests from SovdEntry to ItemsEntry.
| json response = { | ||
| {"name", "ROS 2 Medkit Gateway"}, {"version", "0.1.0"}, {"api_base", API_BASE_PATH}, | ||
| {"name", "ROS 2 Medkit Gateway"}, {"version", "0.3.0"}, {"api_base", API_BASE_PATH}, | ||
| {"endpoints", endpoints}, {"capabilities", capabilities}, |
There was a problem hiding this comment.
The gateway version is still hardcoded in the root response. This will drift again on the next release; consider deriving it from the package/build version (e.g., a CMake-generated header or compile definition) so docs and API remain accurate automatically.
There was a problem hiding this comment.
Acknowledged. Version derivation from CMake/build is a good improvement but out of scope for this docs PR. Tracked in #261 (test coverage gaps) alongside related improvements.
| // Bulk Data | ||
| "GET /api/v1/areas/{area_id}/bulk-data", | ||
| "GET /api/v1/areas/{area_id}/bulk-data/{category}", | ||
| "GET /api/v1/areas/{area_id}/bulk-data/{category}/{item_id}", | ||
| "GET /api/v1/components/{component_id}/bulk-data", | ||
| "GET /api/v1/components/{component_id}/bulk-data/{category}", | ||
| "GET /api/v1/components/{component_id}/bulk-data/{category}/{item_id}", | ||
| "POST /api/v1/components/{component_id}/bulk-data/{category}", | ||
| "DELETE /api/v1/components/{component_id}/bulk-data/{category}/{item_id}", | ||
| "GET /api/v1/apps/{app_id}/bulk-data", | ||
| "GET /api/v1/apps/{app_id}/bulk-data/{category}", | ||
| "GET /api/v1/apps/{app_id}/bulk-data/{category}/{item_id}", | ||
| "POST /api/v1/apps/{app_id}/bulk-data/{category}", | ||
| "DELETE /api/v1/apps/{app_id}/bulk-data/{category}/{item_id}", | ||
| "GET /api/v1/functions/{function_id}/bulk-data", | ||
| "GET /api/v1/functions/{function_id}/bulk-data/{category}", | ||
| "GET /api/v1/functions/{function_id}/bulk-data/{category}/{item_id}", |
There was a problem hiding this comment.
handle_root now advertises bulk-data endpoints for areas/functions, but BulkDataHandlers validates collection support via EntityCapabilities, where AREA and FUNCTION currently do not support BULK_DATA (so these routes will return collection-not-supported). Either adjust EntityCapabilities/validation so these endpoints truly work for areas/functions, or avoid listing them here (and in docs) to prevent clients from discovering unusable endpoints.
There was a problem hiding this comment.
Fixed in 80091bc - removed the 6 areas/functions bulk-data endpoint strings from handle_root. Only components and apps advertise bulk-data now.
| response["subcomponents"] = base + "/subcomponents"; | ||
| response["hosts"] = base + "/hosts"; | ||
| response["logs"] = base + "/logs"; | ||
| response["bulk-data"] = base + "/bulk-data"; |
There was a problem hiding this comment.
These new per-entity URI fields are added for components/apps, but areas/functions also have bulk-data routes registered (GET /{entity}/bulk-data...) and their discovery responses currently omit a bulk-data URI. Consider adding the same bulk-data link for areas/functions (or removing the area/function bulk-data routes if they’re intentionally unsupported) to keep entity capability responses consistent and discoverable.
| response["bulk-data"] = base + "/bulk-data"; |
There was a problem hiding this comment.
URI fields (logs, bulk-data, cyclic-subscriptions) are only added to components and apps, which is correct - areas and functions don't support these collections. No change needed.
| "items": [ | ||
| { | ||
| "id": "temp_sensor", | ||
| "name": "temp_sensor", | ||
| "self": "/api/v1/components/temp_sensor", | ||
| "area": "powertrain", | ||
| "resource_collections": ["data", "operations", "configurations", "faults"] | ||
| "area": "powertrain" | ||
| } |
There was a problem hiding this comment.
The examples for list endpoints use a self field, but the implementation returns href for list items (e.g., DiscoveryHandlers::handle_list_components sets item["href"]). Updating the examples to match the actual response shape will avoid confusing API consumers.
There was a problem hiding this comment.
Fixed in 5047321 - updated examples to use href instead of self.
Code fixes: - Remove areas/functions bulk-data from handle_root (validation rejects them) - Rename test HandleVersionInfoContainsSovdInfoArray -> HandleVersionInfoContainsItemsArray - Fix test_root_endpoint_includes_snapshots: verify legacy snapshot endpoints are NOT listed Docs fixes: - rest.rst: fix self -> href in area/component list examples - rest.rst: remove /bulk-data from areas and functions resource collections - plugin-system.rst: remove LogProvider include/export from UpdateProvider example - plugin-system.rst: clarify IntrospectionProvider metadata is plugin-internal - discovery-options.rst: fix Field Groups table (status, metadata fields) - discovery-options.rst: fix health endpoint JSON to match MergeReport::to_json() - manifest-discovery.rst: fix gap-fill disabled description
- rest.rst: add /version-info example response, remove stale `area` field from components list example - rest.rst: document sovd_info -> items rename in CHANGELOG as breaking change - discovery-options.rst: add local TOC, note case-sensitivity for policy values, fix strategy name "HybridDiscoveryStrategy" -> "hybrid" - manifest-discovery.rst, migration-to-manifest.rst: fix jq commands (.[] -> .items[]) - CHANGELOG.rst: add Breaking Changes section and new 0.3.0 features - test_plugin_vendor_extensions.test.py: add @verifies REQ_INTEROP_003
refresh_cache() was calling discover_topic_components() for both RUNTIME_ONLY and MANIFEST_ONLY modes. In manifest_only mode this added synthetic components from the runtime ROS 2 graph, violating the intent of "only manifest entities." Invert the condition so only RUNTIME_ONLY merges topic components. MANIFEST_ONLY and HYBRID both use discover_components() directly.
| response["subcomponents"] = base + "/subcomponents"; | ||
| response["hosts"] = base + "/hosts"; | ||
| response["logs"] = base + "/logs"; | ||
| response["bulk-data"] = base + "/bulk-data"; | ||
| response["cyclic-subscriptions"] = base + "/cyclic-subscriptions"; |
There was a problem hiding this comment.
New top-level URI fields (logs, bulk-data, cyclic-subscriptions) are added to the component detail response here, but there are no corresponding handler unit tests verifying these fields are present/accurate. Please add coverage to prevent regressions in the response schema.
There was a problem hiding this comment.
Tracked in #261 - unit tests for the new URI fields will be added alongside the other test coverage improvements.
| response["configurations"] = base_uri + "/configurations"; | ||
| response["faults"] = base_uri + "/faults"; | ||
| response["logs"] = base_uri + "/logs"; | ||
| response["bulk-data"] = base_uri + "/bulk-data"; | ||
| response["cyclic-subscriptions"] = base_uri + "/cyclic-subscriptions"; |
There was a problem hiding this comment.
New top-level URI fields (logs, bulk-data, cyclic-subscriptions) are added to the app detail response here, but there are no corresponding handler unit tests verifying these fields are present/accurate. Please add coverage to prevent regressions in the response schema.
| json response = { | ||
| {"name", "ROS 2 Medkit Gateway"}, {"version", "0.1.0"}, {"api_base", API_BASE_PATH}, | ||
| {"name", "ROS 2 Medkit Gateway"}, {"version", "0.3.0"}, {"api_base", API_BASE_PATH}, | ||
| {"endpoints", endpoints}, {"capabilities", capabilities}, |
There was a problem hiding this comment.
The gateway version is hard-coded in the root response. This will go stale and requires manual updates in multiple places; consider sourcing it from a single version constant generated at build time (e.g., from package/CMake) and reuse it here.
There was a problem hiding this comment.
Duplicate of earlier comment. Tracked in #261.
| json sovd_info_entry = { | ||
| {"version", "1.0.0"}, // SOVD standard version | ||
| {"base_uri", API_BASE_PATH}, // Version-specific base URI | ||
| {"vendor_info", {{"version", "0.1.0"}, {"name", "ros2_medkit"}}} // Vendor-specific info | ||
| {"vendor_info", {{"version", "0.3.0"}, {"name", "ros2_medkit"}}} // Vendor-specific info | ||
| }; |
There was a problem hiding this comment.
The vendor_info.version string is hard-coded and duplicates the root "version" value. To avoid inconsistencies between endpoints, derive both from the same single version constant.
There was a problem hiding this comment.
Agreed - both should derive from the same constant. Tracked in #261.
| // @verifies REQ_INTEROP_001 | ||
| TEST_F(HealthHandlersTest, HandleVersionInfoSovdEntryHasVersionField) { | ||
| handlers_.handle_version_info(req_, res_); | ||
| auto body = json::parse(res_.body); | ||
| auto & entry = body["sovd_info"][0]; | ||
| auto & entry = body["items"][0]; |
There was a problem hiding this comment.
These test names still refer to "SovdEntry" even though the response wrapper key has been renamed to "items". Renaming the affected tests (e.g., to "ItemsEntry") would keep the suite terminology consistent and reduce confusion.
| if (discovery_mgr_->get_mode() == DiscoveryMode::RUNTIME_ONLY) { | ||
| // RUNTIME_ONLY: merge node + topic components (no pipeline) | ||
| auto node_components = discovery_mgr_->discover_components(); | ||
| auto topic_components = discovery_mgr_->discover_topic_components(); | ||
| all_components.reserve(node_components.size() + topic_components.size()); | ||
| all_components.insert(all_components.end(), node_components.begin(), node_components.end()); | ||
| all_components.insert(all_components.end(), topic_components.begin(), topic_components.end()); | ||
| } else { | ||
| // HYBRID: pipeline merges all sources; MANIFEST_ONLY: manifest components only | ||
| all_components = discovery_mgr_->discover_components(); | ||
| } |
There was a problem hiding this comment.
This change alters refresh_cache() behavior across discovery modes (topic-based components now only merged in RUNTIME_ONLY). There doesn’t appear to be a unit/integration test covering MANIFEST_ONLY vs RUNTIME_ONLY cache contents (e.g., ensuring topic-only components never leak into MANIFEST_ONLY). Please add a regression test for this behavior.
There was a problem hiding this comment.
Added regression test test_25_no_topic_components_in_manifest_only in 94818bb - verifies that only manifest-defined component IDs appear in the entity cache when using manifest_only mode.
…ion test Rename 3 unit tests referencing old "SovdEntry" naming to "ItemsEntry" to match the sovd_info->items rename in handle_version_info. Add regression test verifying that topic-based components do not leak into the entity cache in manifest_only discovery mode (validates the fix in gateway_node.cpp discover_components).
Pull Request
Summary
Fix outdated documentation, API response bugs, and a discovery bug after merge pipeline, logging, and plugin system changes landed on main.
Breaking changes:
GET /version-inforesponse key renamed fromsovd_infoto SOVD-standarditemsGET /root endpoint restructured: flat string endpoint list, newcapabilitiesobject,api_base/name/versionfieldssqlite3tomcapCode fixes:
gateway_node.cpp)handle_root: logs, bulk-data, cyclic-subscriptions, updateshandle_root: logs, bulk_data, cyclic_subscriptions, updateshandle_root(listed but never route-registered)handle_root(validation rejects them)bulk-data,logs,cyclic-subscriptionsURI fields to entity capability responsesDocumentation fixes:
/version-infoexample response in rest.rstself->hrefin area/component list examples (matching actual code)areafield from components list example.[]->.items[]).. contents::TOC and case-sensitivity note for policy valuesHybridDiscoveryStrategy->hybridin health examplemerge_types.hppMergeReport::to_json()DELETE /{entity}/faults, areas/functions resource collection notes@verifies REQ_INTEROP_003to test_plugin_vendor_extensionsIssue
Type
Testing
sphinx-buildcompletes with zero warningssovd_info->itemschangeChecklist
Follow-up issues
sovd_info->itemsrename in web UI