diff --git a/docs/cli/t4viz.md b/docs/cli/t4viz.md index 4e7fb837..e5e94f95 100644 --- a/docs/cli/t4viz.md +++ b/docs/cli/t4viz.md @@ -58,6 +58,8 @@ t4viz instance ... [OPTIONS] This command performs the same behavior with [`Tier4.render_pointcloud(...)`](../tutorials/render.md#rendering-pointcloud). +If a pointcloud `sample_data` record has `info_filename`, `t4viz pointcloud` loads the metainfo JSON automatically and uses `num_pts_feats` to parse extended point layouts. Without that file, it falls back to the standard 5-feature layout. + For options, run `t4viz pointcloud -h`. ```shell diff --git a/docs/schema/data.md b/docs/schema/data.md index ec0e9ace..b7a3772b 100644 --- a/docs/schema/data.md +++ b/docs/schema/data.md @@ -11,7 +11,9 @@ data/ ... ``` -Each file contains `(x, y, z, intensity, ring_idx(=-1))`, and location coordinates are given with respect to the ego vehicle coordinate system. +By default, each file contains `(x, y, z, intensity, ring_idx(=-1))` as `float32` values, and location coordinates are given with respect to the ego vehicle coordinate system. + +Some datasets can store additional `float32` features for each point, such as `return_type` or `timestamp`. In that case, `sample_data.info_filename` should point to a pointcloud metainfo JSON file that includes `num_pts_feats`, which describes the total number of `float32` fields in each point. Each file can be loaded using as follows: @@ -19,15 +21,42 @@ Each file can be loaded using as follows: # Using NumPy import numpy as np -def load_lidar_point_cloud(file_path) -> np.ndarray: - data = np.fromfile(file_path, dtype=np.float32) # (N*5,) - return data.reshape((-1, 5)) # (N, 5) +def load_lidar_point_cloud(file_path, num_pts_feats: int = 5) -> np.ndarray: + data = np.fromfile(file_path, dtype=np.float32) + return data.reshape((-1, num_pts_feats)) # Using t4-devkit from t4_devkit.dataclass import LidarPointCloud -def load_lidar_point_cloud_t4(file_path) -> LidarPointCloud: - return LidarPointCloud.from_file(file_path) +def load_lidar_point_cloud_t4(file_path, metainfo_path: str | None = None) -> LidarPointCloud: + return LidarPointCloud.from_file(file_path, metainfo_filepath=metainfo_path) +``` + +When `metainfo_filepath` is provided and the file exists, `t4-devkit` uses `num_pts_feats` from the metainfo JSON to reshape the binary pointcloud correctly. If the metainfo file is omitted or unavailable, `t4-devkit` falls back to the standard 5-feature layout. + +Current `LidarPointCloud` and `SegmentationPointCloud` readers keep the first 4 dimensions `(x, y, z, intensity)` after reshaping. Additional point features are used to determine the row width and are not exposed in the returned point matrix. + +Example pointcloud metainfo JSON: + +```json +{ + "stamp": { + "sec": 1710000000, + "nanosec": 123456789 + }, + "num_pts_feats": 7, + "sources": [ + { + "sensor_token": "", + "idx_begin": 0, + "length": 120000, + "stamp": { + "sec": 1710000000, + "nanosec": 123456789 + } + } + ] +} ``` ## Camera Image diff --git a/docs/schema/requirement.md b/docs/schema/requirement.md index 30bccc9f..251e84d5 100644 --- a/docs/schema/requirement.md +++ b/docs/schema/requirement.md @@ -66,9 +66,9 @@ ### External Table Reference -| ID | Name | Severity | Fixable | Description | -| -------- | --------------------------------- | -------- | ------- | ------------------------------------------ | -| `REF301` | `pointcloud-metainfo-token-check` | `ERROR` | `N/A` | `PointCloudMetainfo sensor tokens` exists. | +| ID | Name | Severity | Fixable | Description | +| -------- | --------------------------------- | -------- | ------- | ------------------------------------------------------------------------------------------------------------------- | +| `REF301` | `pointcloud-metainfo-token-check` | `ERROR` | `N/A` | `PointCloudMetainfo.sources[].sensor_token` exists for each metainfo file referenced by `SampleData.info_filename`. | ## Format (`FMT`) diff --git a/docs/schema/table.md b/docs/schema/table.md index 7deb593c..23cd3b8e 100644 --- a/docs/schema/table.md +++ b/docs/schema/table.md @@ -257,7 +257,7 @@ sample_data { "next": -- Foreign key to the `SampleData` table associated with the next data in the sequence. Empty string `""` if this is the last data. "prev": -- Foreign key to the `SampleData` table associated with the previous data in the sequence. Empty string `""` if this is the first data. "is_valid": -- Indicates whether this data is valid. Defaults to `true`. - "info_filename": -- Relative path to metadata-blob file. + "info_filename": -- Relative path to an optional metainfo JSON file. For pointcloud data, this can describe the point layout, including `num_pts_feats`. "autolabel_metadata": -- List of models used for autolabeling applied to this entire sample_data item (e.g., image or scan). } ``` diff --git a/docs/tutorials/render.md b/docs/tutorials/render.md index 0a736510..08ed9b6c 100644 --- a/docs/tutorials/render.md +++ b/docs/tutorials/render.md @@ -36,6 +36,8 @@ If you want to visualize annotation results, `Tier4` supports some rendering met >>> t4.render_pointcloud() ``` +If `SampleData.info_filename` points to a pointcloud metainfo JSON file, `Tier4.render_pointcloud()` loads it automatically. This allows rendering pointclouds with extended per-point features such as `return_type` or `timestamp`. When no metainfo file is available, the standard 5-feature layout is assumed. + ![Render PointCloud GIF](../assets/render_pointcloud.gif) ### Save Recording @@ -127,8 +129,12 @@ from t4_devkit.dataclass import LidarPointCloud from t4_devkit.viewer import PointCloudColorMode # Point cloud channel name >>> lidar_channel = "LIDAR_TOP" -# Load point cloud from file ->>> pointcloud = LidarPointCloud.from_file() +# Load point cloud from file. +# Pass metainfo_filepath when the binary stores more than 5 float32 values per point. +>>> pointcloud = LidarPointCloud.from_file( +... , +... metainfo_filepath=, +... ) >>> color_mode = PointCloudColorMode.DISTANCE >>> viewer.render_pointcloud(seconds, lidar_channel, pointcloud, color_mode) ``` @@ -142,8 +148,13 @@ from t4_devkit.dataclass import SegmentationPointCloud from t4_devkit.viewer import PointCloudColorMode # Point cloud channel name >>> lidar_channel = "LIDAR_TOP" -# Load point cloud and label from file ->>> pointcloud = SegmentationPointCloud.from_file("", "") +# Load point cloud and label from file. +# Pass metainfo_filepath when the pointcloud binary uses an extended feature layout. +>>> pointcloud = SegmentationPointCloud.from_file( +... "", +... "", +... metainfo_filepath="", +... ) >>> color_mode = PointCloudColorMode.SEGMENTATION >>> viewer.render_pointcloud(seconds, lidar_channel, pointcloud, color_mode) ```