Skip to content

feat(infra): captions in the hosted player and embed - #4187

Merged
richiemcilroy merged 23 commits into
mainfrom
feat/infra-player-captions
Nov 13, 2026
Merged

richiemcilroy merged 23 commits into
mainfrom
feat/infra-player-captions

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Stacked on #4171 (Safari start), which merges first. This rebases onto main after that, so do not merge it before #4171.

A video that has captions from #4134 now shows them in Cap's hosted player and embed. Viewers get a CC button and the c key, and an embed can start with them on with captions=1.

flowchart LR
  T[POST /v1/transcripts videoId] -->|#4134 writes| S[(captions/en.vtt in storage)]
  P[loadDeveloperVideoPlayer] -->|captions: language, label, url or null| V[CapPlayer]
  V -->|CC on: fetch once, blob URL| R["/api/developer/playback/:id/captions-en.vtt"]
  R -->|readDeveloperVideoText, 60 s cache, CORS *| S
Loading

What changes

  • Playback response: DeveloperVideoPlayer (kind: "playing") gains captions: { language, label, url }[] | null. It is null for a video with no completed transcript with speech. Built by developerVideoCaptionTracks (apps/web/lib/developer-video-captions.ts) from feat(infra): transcribe a stored video by id and write its captions #4134's developerVideoCaptions. A failed lookup logs and plays without captions.
  • Playback route: captions-<lang>.vtt (for example captions-en.vtt, captions-pt-br.vtt, captions-und.vtt) is served by the existing /api/developer/playback/[playbackId]/[file] route.
    • Same lookups as every other file: unknown or deleted video 404, paused app 402, and the same public, max-age=60, s-maxage=60 and CORS headers.
    • The VTT is read from storage and sent as the body, like the playlists, rather than redirected to a signed URL. A <track> loads in same-origin mode unless its <video> sets crossorigin, so it cannot follow a redirect to storage on another origin, and setting crossorigin would change how every media file loads. No storage URL or key is handed out, and no new bucket access is added.
    • The file name is checked against ^captions-([a-z]{2,3}(-[a-z0-9]{2,8})*)\.vtt$, so the key is always developer/<appId>/<videoId>/captions/<lang>.vtt for that video.
  • Player (apps/web/components/cap-player):
    • The CC button appears only when the video has captions. It has aria-pressed, the player's focus ring, and the c key. With more than one language it opens the same menu as speed and quality (Off plus each language).
    • Nothing is fetched until captions are turned on. The file is fetched once, handed to a <track> as a blob URL, and the track runs in hidden mode.
    • The player draws the active cue itself, in its own font and scrim, and lifts it above the control bar while the controls show. In picture in picture and the iPhone's own full screen, which cannot show page content, the track switches to showing so the browser draws it.
    • The same path runs with hls.js (Chrome, Firefox, Safari on a Mac) and with Safari's own HLS (iPhone, iPad).
    • If the file cannot be loaded, the button goes away and the video plays on.
    • hls.js's subtitle and in-band caption controllers are turned off (source.ts). Cap's streams have neither. Left on, they clear the cues of every text track when a manifest loads and switch tracks off when hls.js detaches.
  • Embed parameter: captions=1 (or true) starts with captions on. captions=0, absent, or any other value keeps them off. It applies to the embed only, like autoplay, muted, loop and controls. Documented in the REST docs' embed parameter table and the OpenAPI embedUrl description.
  • Console: the video page lists the caption languages under the video's facts, and its player has the CC button. The console has no embed code builder (the embed code is fixed), so there is no captions=1 tick. The note under Captions says to add captions=1 to the embed link.
  • Regular Cap share pages (/s/...) do not use cap-player and are untouched. cap-player is imported only by the Infra pages (/dev/...), the Infra console and the Infra landing film.

Screenshots

Captured on the Mac mini against production playback of a temporary Cap Infra video (uploaded, transcribed through #4134, then deleted). The cues are #4134's WebVTT writer output for that transcript.

Captions off Captions on (paused, controls up) Embed with captions=1, playing
Chrome Chrome captions off Chrome captions on Chrome embed captions=1
Firefox Firefox captions off Firefox captions on Firefox embed captions=1
Playwright WebKit Playwright WebKit captions off Playwright WebKit captions on Playwright WebKit embed captions=1
Safari 27.0.1 (hls.js) Safari 27.0.1 (hls.js) captions off Safari 27.0.1 (hls.js) captions on Safari 27.0.1 (hls.js) embed captions=1
Safari 27.0.1 (native HLS) Safari 27.0.1 (native HLS) captions off Safari 27.0.1 (native HLS) captions on Safari 27.0.1 (native HLS) embed captions=1
Two languages Small embed (390×220) Console video page
language menu small embed console video page

Browser proof

Chrome Firefox Playwright WebKit Safari 27.0.1
No captions: no button, c ignored ✓ ✓ ✓
Off by default, VTT not fetched ✓ ✓ ✓ ✓ both
Button on, cue shown while paused ✓ ✓ ✓ ✓ both
c off and on ✓ ✓ ✓ ✓ both
Cues follow playback ✓ ✓ ✓ ✓ both
captions=1 on from the start, captions=0 off ✓ ✓ ✓ ✓ both (captions=1)
controls=0&captions=1 shows cues ✓ ✓ ✓
Two languages: menu ✓ ✓ ✓
Missing file: button gone, video plays ✓ ✓ ✓
Focus ring (2 px outline) ✓ ✓ ✓
Share page ignores captions=1 ✓ ✓ ✓
Native HLS with a touch screen reported ✓
  • "Both" is Safari on a Mac (hls.js) and Safari told it has a touch screen, which takes the iPhone and iPad path (Safari's own HLS).
  • Safari's driver does not deliver pointer clicks on this machine, so in Safari the CC button was pressed from script and playback started muted. The c key went through the driver's keyboard input.
  • The pages ran from this branch's next dev with production playback for the video. The WebVTT came from a stand-in server on another origin that sends the route's exact headers, because production does not serve captions-*.vtt until this ships. That is the harder case for the player (cross-origin), and the route itself is covered by unit tests. A real-deployment check of captions-en.vtt belongs in the post-merge smoke.
  • iPhone full screen and picture in picture (where the browser draws the cues) were not exercised.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge after its stated prerequisite.

Summary

Adds captions to the hosted Infra player, embeds, and console.

  • The playback route gives the player caption tracks for playable videos.
  • The hosted player lets viewers turn captions on and choose a language.
  • Embeds can start with captions already on.
  • The Infra console lists caption languages and sends tracks to its player.

This review used the full PR because the merge-base checks did not pass. Checks were limited to source inspection; no tests or browser checks were run.

Reviews (11) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile

hls.js cleared the cues of every text track when a manifest loaded, which
briefly showed a caption track that had no source yet. Chrome then kept
that track in its error state after the source arrived. Turn off the
hls.js caption parts Cap's streams never use, and make each caption track
only once its file has loaded.
@sandchest-com

sandchest-com Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Sandchest

Sandchest reviewed 69a1290: nothing exploitable found in this change.

Sandchest reviews every change for vulnerabilities an attacker could use · 5 credits · Review details

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 1 potential issue (1 medium).

Open in Capy

@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
cap-status Ignored Ignored Preview Nov 13, 2026 10:02am UTC

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 07:11 Inactive
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 1 other finding in Capy.

Open in Capy

…tions

# Conflicts:
#	apps/web/app/(infra)/dashboard/infra/infra-data.ts
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 1 other finding in Capy.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 07:56 Inactive
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 1 other finding in Capy.

Open in Capy

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 2 other findings in Capy.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 08:33 Inactive
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 09:48 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 1 other finding in Capy.

Open in Capy

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 09:53 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 09:57 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 1 other finding in Capy.

Open in Capy

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@richiemcilroy
richiemcilroy merged commit 7e3d1d2 into main Nov 13, 2026
39 checks passed

This branch was successfully deployed

1 active and 1 inactive (outdated) deployments
Preview – cap-web — 69a12905 Deployed Nov 13, 2026 by vercel[bot]
Preview – cap-status — 7ebc9a45 Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant