[IsaacLab 3.0.0 Multi-Backend] Visual Logging for Headless Runs#6212
[IsaacLab 3.0.0 Multi-Backend] Visual Logging for Headless Runs#6212mataylor-nvidia wants to merge 2 commits into
Conversation
Greptile SummaryThis PR introduces
Confidence Score: 3/5The production wrapper and training-script wiring are correct, but the entire new test file is broken due to a parameter name mismatch and should not be merged as-is. The core source/isaaclab/test/test_reinforcement_learning_common.py — the Important Files Changed
Sequence Diagram%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant RL as RL Framework
participant CES as CaptureEnvSensors
participant RV as RecordVideo
participant IL as IsaacLab Env
RL->>CES: step(action)
CES->>RV: step(action)
RV->>IL: step(action)
IL-->>RV: obs, reward, ...
RV-->>CES: obs, reward, ...
CES->>CES: "_step_count += 1"
CES->>CES: _save_frame()
Note over CES: step_count % interval < frame_count?
CES->>IL: unwrapped.scene.sensors
IL-->>CES: sensor dict
loop each sensor / data_type
CES->>CES: _to_image_tensor(buffer, data_type)
alt tensorboard mode
CES->>CES: writer.add_images(tag, tensor, step)
else file mode
CES->>CES: save_images_to_file(tensor, path)
end
end
CES-->>RL: obs, reward, ...
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant RL as RL Framework
participant CES as CaptureEnvSensors
participant RV as RecordVideo
participant IL as IsaacLab Env
RL->>CES: step(action)
CES->>RV: step(action)
RV->>IL: step(action)
IL-->>RV: obs, reward, ...
RV-->>CES: obs, reward, ...
CES->>CES: "_step_count += 1"
CES->>CES: _save_frame()
Note over CES: step_count % interval < frame_count?
CES->>IL: unwrapped.scene.sensors
IL-->>CES: sensor dict
loop each sensor / data_type
CES->>CES: _to_image_tensor(buffer, data_type)
alt tensorboard mode
CES->>CES: writer.add_images(tag, tensor, step)
else file mode
CES->>CES: save_images_to_file(tensor, path)
end
end
CES-->>RL: obs, reward, ...
Reviews (1): Last reviewed commit: "visual sensor logs" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| os.makedirs(os.path.dirname(file_path), exist_ok=True) | ||
| save_images_to_file(image_tensor, file_path) | ||
|
|
||
| self._saved_frame_count += 1 |
There was a problem hiding this comment.
_saved_frame_count is incremented at the end of every _save_frame call that passes the schedule gate, even when no sensor has image data and nothing was actually written to disk. If an environment has no image sensors, the frame counter still advances on every scheduled step, producing discontinuous frame indices (frame_000000, frame_000002, …) the first time a sensor does become available. Consider incrementing inside the inner loop only when an image is actually written, or resetting the counter to only count real captures.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
1 participant
Description
REQ-12: Support capturing images from the visualizer when running headless such that we can capture the full or subset of environments during training to observe progress through frameworks such as tensorboard or wandb
This feature exports per step frames from simulation from full or subset of the environments.
Fixes # (OMPE-78691)
https://jirasw.nvidia.com/browse/OMPE-78691
Type of change
Screenshots
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there