Skip to content

camera_streamer: map SIPL sensor timestamps to sender CLOCK_MONOTONIC and add an AVS1 timing record - #1474

Open
gareth-morgan-nv wants to merge 7 commits into
gmorgan/CameraStreamingSenderfrom
gmorgan/CameraStreamingSenderTiming
Open

gareth-morgan-nv wants to merge 7 commits into
gmorgan/CameraStreamingSenderfrom
gmorgan/CameraStreamingSenderTiming

Conversation

@gareth-morgan-nv

Copy link
Copy Markdown
Contributor

Summary

Places the SIPL sensor timestamp on the sender's CLOCK_MONOTONIC with a checked mapping and an explicit uncertainty, and carries it to receivers in an optional AVS1 timing record. Frames whose timestamp fails the checks carry no capture time; nothing is replaced by a host-side stamp. The record is off by default.

What changed

  • SensorClockMap: pairs the platform counter with CLOCK_MONOTONIC once a second and converts each frame's counter timestamp from the difference. It refuses a capture after the frame was ready, an implausibly old one, an unreadable counter, an unknown frequency, or a step between two anchors, and it reports the reason when it changes. A mapped frame carries an uncertainty (the anchor's read bracket plus 500 ppm of the time between the anchor and the sensor event, the bound on how far CLOCK_MONOTONIC can be slewed against the counter) and the anchor's age as a separate freshness value.
  • SIPL start-of-frame is reported as a checked platform TSC in nanoseconds in the existing AVS1 sample and bundle TSC fields (the same domain the Argus start-of-frame uses), only while the check passes. CAMERA_SENDER_SIPL_TSC_PROBE=1 logs the raw SIPL timestamps beside the counter for repeating the measurement.
  • AVS1 timing extension (bundle flag 0x0008): one 24-byte record per sample after the sample descriptors, with the capture instant on CLOCK_MONOTONIC, the uncertainty, the anchor age, a VALID flag and the clock domain. A record without VALID is all zero; the encoder and parser reject the other combinations, unknown flags and nonzero reserved fields; values that do not fit 32 bits are left unqualified, never wrapped. WIRE_FORMAT.md documents it with a golden byte vector that a unit test compares against.
  • network.timing_extension (default false) turns the extension on. A receiver that predates it rejects those bundles, so keep it off until the receiver parses it.

Validation

  • clang-format-14 and pre-commit clean on the changed files (and pre-commit run --all-files clean on this head).
  • aarch64 cross-compile with -DWITH_SIPL=ON and OFF: syntax check and full link.
  • Unit tests, built and run by hand on a PC: sensor_clock (wrong units, capture after ready, implausible age, refresh schedule, a deliberate clock step, reset, unreadable counter, and the uncertainty over the anchor-to-event distance), avs1_wire (extension round trip, rejections, golden vector), bundle_assembler (extension fill, 32-bit overflow left unqualified).
  • Native build and unit tests on a Jetson Orin (R39.2.1): 17 of 17 camera-free tests, on a local build that also contained the clock-exchange PR.
  • Measured on the Orin with the SIPL backend: the start-of-frame value counts CNTVCT_EL0 ticks at 31.25 MHz, advances at the same rate as CLOCK_MONOTONIC (1.00368 s against 1.00369 s over 30 frames), leads the dequeue by about 22 ms, and the end-of-frame value is about 15.5 ms later.
  • A real SIPL stream with the extension on, read by an independent AVS1 parser: 897 of 897 samples with a VALID record, no sequence gaps, uncertainty median about 0.23 ms and maximum 0.5 ms, and after a forced pipeline rebuild 896 of 896 again with a new session id.
  • The same stream with the extension on through a receiver that parses it: 901 of 901 records valid, no sequence gaps, and a different session id after a receiver reconnect. That receiver is not part of this PR.
  • Agreement of the capture instant with an independent physical measurement: not measured.

Known limitations / follow-ups

  • The counter units and the start-of-frame offset were measured on one Orin and one camera; they are not taken from vendor documentation, and how the event relates to the physical exposure start and to tracking data is not established.
  • Only the SIPL backend fills the record; the Argus path does not emit the extension.
  • A receiver has to parse the extension before it is enabled; that is a change on the receiver side.

Test plan

  • Stream with network.timing_extension: true into a receiver that parses the extension and check the per-frame capture times.
  • CI.

🤖 Generated with Claude Code

gareth-morgan-nv and others added 7 commits November 10, 2026 23:18
…K_MONOTONIC

A sensor timestamp that counts the platform counter's ticks is placed on CLOCK_MONOTONIC from one anchor pair
(counter, CLOCK_MONOTONIC) that is refreshed on a schedule, never per frame. Every conversion is checked: a
capture time after the frame was ready or implausibly long before it, a counter that cannot be read, an unknown
frequency, or a clock step between anchors gives an invalid mapping with a reason; nothing is replaced by a
host-side stamp. The mapping carries the anchor's bracket as its uncertainty and the anchor's age, and reset()
drops the anchor when capture restarts. Unit tests inject the clock reader and cover wrong units, capture after
ready, an implausible age, the refresh schedule, a deliberate clock step, reset and an unreadable counter.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
…tamps

CAMERA_SENDER_SIPL_TSC_PROBE=1 logs about once a second the SIPL start-of-frame and end-of-frame capture
timestamps beside the platform counter, its frequency and CLOCK_MONOTONIC, so the timestamp's units and the
event it marks can be established from measurements instead of assumed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
… TSC in nanoseconds

The SIPL start-of-frame timestamp counts the platform counter's ticks (31.25 MHz here, measured against
CNTVCT_EL0 and CLOCK_MONOTONIC on the board: it leads the counter read at dequeue by about 22 ms, and the
end-of-frame value by 15 ms). SiplCapture places each frame's timestamp with SensorClockMap and, only while the
check passes, reports it as a TSC in nanoseconds in the same domain as the Argus start-of-frame timestamp; a frame
that fails the check carries no sensor time and the reason is logged when it changes. CaptureFrame also carries the
capture time on CLOCK_MONOTONIC with its uncertainty and anchor age when mapped. SiplSender writes the TSC into the
AVS1 sample and the bundle's TSC anchor (read with the wall clock) only for frames that carry one; a raw counter is
never written to a nanosecond field.

Source-built and cross-compiled; qualified against hardware by a run recorded separately.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
…IPL sender

TIMING_EXT (bundle flag 0x0008) adds one 24-byte record per sample after the sample descriptors, counted in
descriptor_bytes: the sensor's capture instant on the sender's CLOCK_MONOTONIC, an uncertainty bound, the age of the
mapping's anchor, a VALID flag and the clock domain. A record without VALID is all zero and makes no capture claim; the
encoder and parser reject the other combinations, unknown flags and nonzero reserved fields. FrameBundleAssembler
fills the records from the frame's checked capture mapping when constructed with the extension, and the SIPL sender
turns it on with network.timing_extension, which is off by default because a receiver that predates the flag rejects
those bundles. SensorClockMap's uncertainty now also grows by 500 ppm of the anchor's age, the bound on how far
CLOCK_MONOTONIC can be slewed against the counter. WIRE_FORMAT.md documents the record with a golden byte vector that
the unit test compares against.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
…extension

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
…ance, not the anchor's age

The counter and CLOCK_MONOTONIC can drift apart by at most the slew limit over the time the mapping extrapolates from its
anchor to the sensor event, in either direction. The uncertainty is now the anchor's read bracket plus 500 ppm of that
distance; it was 500 ppm of how old the anchor was when the frame was handled, which is zero for a frame mapped at a
fresh anchor even though the event precedes the anchor by tens of milliseconds. The anchor's age stays a separate
freshness value. WIRE_FORMAT.md and the README state the new bound.

The sensor clock unit tests still assert the old formula and are updated separately.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
…nt distance

The slew term of the mapping's uncertainty follows the distance between the
anchor and the sensor event, not the anchor's age. Update the refresh-schedule
and first-mapping expectations and cover a fresh anchor with an earlier event,
a delayed capture, an event after the anchor, a non-default slewPpm, an event at
the anchor, and the uncertainty reported for refused mappings.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
@coderabbitai

coderabbitai Bot commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

This branch had an error being deployed

1 failed deployment
dev — 918adac7 Deployed Nov 11, 2026 by gareth-morgan-nv via publish-wheel #6762
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants