feat(data): generalize lidar model_element to flexible shapes and dtypes#156
Open
janickm wants to merge 1 commit into
Open
feat(data): generalize lidar model_element to flexible shapes and dtypes#156janickm wants to merge 1 commit into
janickm wants to merge 1 commit into
Conversation
The per-ray model_element of the V4 LidarSensorComponent was previously fixed to shape (N, 2) and dtype uint16. Generalize it to support arbitrary per-ray model indexing / sampling coordinates while staying backwards-compatible (zarr storage is already self-describing for shape and dtype; only the writer-side validation was restrictive). New contract: - Shape: a leading per-ray dimension N followed by any trailing shape -- flat (N,), the legacy (N, 2) (row, col), or arbitrary (N, A, B, ...). - Dtype: any integer type (discrete model index) or any floating type up to float32 (continuous model sampling coordinate, e.g. float16 / float32); float64 (and wider) is rejected as too storage-heavy. This is groundwork for solid-state / semi-solid-state lidar models (NVIDIA#125), whose elements may need extra dimensions (e.g. zone, scan-direction / frame parity) or continuous sampling coordinates, without changing the on-disk data format. Existing (N, 2) uint16 data and all consumers (which index the leading two columns) are unaffected. Broaden the LidarSensorProtocol / V4 reader return type to NDArray[Any] and update docstrings accordingly. Add round-trip tests for flat (N,), legacy (N, 2) uint16, higher-dim (N, A, B), and float16 / float32 sampling coordinates, plus rejection of float64, 0-d, and mismatched leading dim.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Generalize the per-ray
model_elementof the V4LidarSensorComponentfrom a fixed(N, 2)uint16layout to flexible shapes and dtypes, in a backwards-compatible way.New contract for
model_element:Nfollowed by any trailing shape -- flat(N,), the legacy(N, 2)(row, col), or arbitrary(N, A, B, ...).float32(continuous model sampling coordinate, e.g.float16/float32);float64(and wider) is rejected as too storage-heavy.Why
This is groundwork for solid-state / semi-solid-state lidar model support (#125). Such sensors do not fit the current spinning
(row, col)grid: their elements may need extra dimensions (e.g. zone, scan-direction / frame parity) or continuous sampling coordinates into a to-be-derived model.The zarr storage backend is already self-describing for shape and dtype, so only the writer-side validation was restrictive -- relaxing it lets the abstract lidar model carry richer per-ray indices without changing the on-disk data format.
Backwards compatibility
(N, 2)uint16data reads/writes unchanged.model_element[:, 0]/[:, 1]), so they are unaffected.LidarSensorProtocol/ V4 reader return type is broadened toOptional[NDArray[Any]].Tests
Added round-trip coverage for: flat
(N,), legacy(N, 2)uint16, higher-dim(N, A, B), andfloat16/float32sampling coordinates; plus rejection offloat64, 0-d arrays, and mismatched leading dimension. The existingtest_reloadintegration test now also exercises a multi-columnmodel_element.All affected Bazel test targets pass:
//ncore/impl/data/v4:pytest_components_3_11//ncore/impl/data/v4:pytest_compat_3_11//ncore/impl/sensors:pytest_lidar_3_11,pytest_rolling_shutter_3_11//tools/data_converter:pytest_structured_lidar_model_3_11Formatter (
bazel run format) clean.