This is an application that helps create high quality computer vision datasets by making full use of available iPhone sensors.
The recording optimizes consistent frame quality at the expense of regular framerate. Framerate is variable but all frames are keyframes to reduce artifacts during training. Samples are stored in a video/quicktime container and contain the following streams:
- Main video stream from back of the camera:
prores (LT) (apcs / 0x73637061), yuv422p10le(tv, smpte170m/bt709/bt709, progressive), 3840x2160, 378799 kb/s, 29.31 fps, 30 tbr, 600 tbn, 600 tbc (default) - Depth map: Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv), 256x144, 592 kb/s, 30 fps, 30 tbr, 600 tbn, 600 tbc (default)
- Computed saliency map:
Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv), 68x68, 446 kb/s, 29.31 fps, 30 tbr, 600 tbn, 600 tbc (default) - Realtime metadata:
Data: none (mebx / 0x7862656D), 147 kb/s (default) - Post analysis metadata:
Data: none (mebx / 0x7862656D), 17 kb/s (default)
Samples contain video metadata and timed metadata:
mdta/nl.delftsolutions.trashScan.targetPosition.world_space[video]: Position of the chosen subject in world space. Scale is roughly in meters.mdta/nl.delftsolutions.trashScan.camera.projection_matrix[video]: The projection matrix of the camera.mdta/nl.delftsolutions.trashScan.camera.intrinsics[video]: The intrinsic matrix of the camera.mdta/nl.delftsolutions.trashScan.camera.lens_specification[video]: The lens specification of the camera.mdta/com.apple.quicktime.location.ISO6709[video]: The very rough location of the recording.mdta/com.apple.quicktime.creationdate[video]: The date and time when the recording was started.mdta/com.apple.quicktime.keywords[video]: The classification the user made when taking the recording. Can be absent. (Trash with recycling code: 10)mdta/com.apple.quicktime.artwork[video]: A png of the recycling code on the subject. Can be absent.mdta/nl.delftsolutions.trashScan.targetPosition.camera_space[timed]: Position of the chosen subject in camera space.mdta/nl.delftsolutions.trashScan.camera.euler_rotation[timed]: Euler angles of the camera orientation.mdta/nl.delftsolutions.trashScan.camera.position[timed]: Position of the camera in world space.mdta/nl.delftsolutions.trashScan.camera.exposure_bias[timed]: Exposure bias of the camera.mdta/nl.delftsolutions.trashScan.camera.focal_length[timed]: Focal length of the camera.mdta/nl.delftsolutions.trashScan.camera.exposureTime[timed]: Exposure time of the camera.mdta/nl.delftsolutions.trashScan.camera.aperture_value[timed]: Aperture value of the camera.mdta/nl.delftsolutions.trashScan.camera.white_balance[timed]: White balance of the camera.mdta/nl.delftsolutions.trashScan.camera.shutter_speed[timed]: Shutter speed of the camera.mdta/nl.delftsolutions.trashScan.camera.brightness[timed]: Brightness modification done to the stored pixels.mdta/nl.delftsolutions.trashScan.camera.view_matrix[timed]: The view matrix of the camera.mdta/nl.delftsolutions.trashScan.barcode[timed, track5]: Detected barcodes in the frame (if any)mdta/nl.delftsolutions.trashScan.text_element[timed, track5]: Detected text in the frame (if any)mdta/nl.delftsolutions.trashScan.classification[timed, track5]: The result of the iPhone classifier on that frame