Skip to content

Auto-populate a new box or line with a segmentation polygon, head/tail points or a tighter box - #1941

Open
mattdawkins wants to merge 4 commits into
mainfrom
dev/auto-populate-mask-points
Open

mattdawkins wants to merge 4 commits into
mainfrom
dev/auto-populate-mask-points

Conversation

@mattdawkins

@mattdawkins mattdawkins commented Sep 18, 2026

Copy link
Copy Markdown
Member

Two new switches under New Annotation Settings (desktop only — they use the VIAME interactive segmentation service):

  • Auto-populate mask — after a brand-new box or head/tail line is drawn, the loaded interactive segmentation model is prompted (a box: its centre; a line: foreground points at 10/30/50/70/90% along it) and the polygon is stored on the detection.
  • Auto-populate points — for a box, head/tail are derived from that polygon by VIAME's polygon_keypoints (the same add_keypoints_from_mask hull-extremes code as the SAM2 head/tail utility pipeline). For a line, the box is tightened to the segmentation's bounds.

Either switch triggers the segmentation; only what is still missing on the detection is filled in (an existing polygon or line is left alone), so a user who keeps annotating is not overwritten. Hooked through a new onNewAnnotationGeometry callback in useModeManager that fires only for the first shape of a track being created.

  • Stereo — with Auto compute location in other camera on, once the new box or line maps to the other camera successfully the mapped copy gets the same mask/points pass there (segmented on that camera's image). A derived head/tail is ordered to follow the source camera's line, and the stereo length is refreshed when length updates are on.

Needs VIAME/VIAME#299 for the polygon_keypoints command. Helper functions (prompt construction, ring closing, bounds) are unit tested; the end-to-end flow was verified against the running service but not yet clicked through in the app.

🤖 Generated with Claude Code

With auto compute on the other camera, the mapped shape gets the same mask
and/or points pass once the transfer succeeds; a derived head/tail follows
the source camera's direction.
Lets the VIAME service keep a line-prompted mask in scale with the line.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant