Skip to content

perf(infra): finish a short transcript in one workflow step - #4147

Merged
richiemcilroy merged 15 commits into
mainfrom
perf/infra-transcript-fast-path
Nov 13, 2026
Merged

richiemcilroy merged 15 commits into
mainfrom
perf/infra-transcript-fast-path

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

A short transcript now completes in one workflow step instead of five, which should take about 3 seconds off every transcript up to 30 minutes long, from about 5.1 s to about 2.0 s for a short file. Billing, holds, limits and failure codes are unchanged.

#4134 (captions), #4143 (transcript abuse fixes) and #4150 (URL sources refused from their first bytes) have landed and main is merged in, so the diff is only this change: the workflow, one helper in lib/developer-transcript-run.ts and tests. The URL check from #4150 sits in the shared prepare body, so the fast step and the prepare step both run it.

Why it was slow

On production, a short transcript does about 1.4 s of work, spread over six steps. Each step waits about 0.6 s to be scheduled and queued, so most of the time to complete is overhead.

Production medians from 22 runs:

Step Wait before it Queue Work
claim 363 ms 179 ms 86 ms
prepare 618 ms 181 ms 152 ms
extract 591 ms 166 ms 565 ms
transcribe 605 ms 168 ms 392 ms
settle 581 ms 174 ms 162 ms
cleanup, after complete 576 ms 172 ms 118 ms

The shape

One new step, runTranscriptFastPath, runs claim, prepare, extract, transcribe and settle back to back in one function. It passes the extracted audio straight to transcription instead of reading it back from storage. Cleanup and the caption write stay separate steps after complete is written.

The fast step stops and reports where it stopped whenever the normal path would do something other than move on:

  • the hold has to wait for other transcripts, or the credit is short
  • the audio is longer than the hold covers, so the hold has to be raised
  • any stage fails or throws
  • the audio is longer than 30 minutes, or the step has already spent 2 minutes, before extraction or before transcription

The workflow then carries on from that stage with the existing steps, unchanged: the same sleeps and waits, the same raises, the same retries and the same failure handling. A transcript that needs none of this completes in the one step.

I chose this over starting the work from the create request with after(), the way videos do:

  • The overhead is per step. Folding five steps into one removes four waits and four queue delays, about 3.1 s.
  • after() would save only the first wait, about 0.5 s more. In exchange it would run downloads and speech calls in the API request's function, outside the runId fencing, with the workflow as a watchdog that could run the same work again.
  • Every stage folded in here was already safe to run again. Claim is fenced by runId, the hold is a no-op once held, extraction returns early when the audio exists, transcription reuses a stored result, and settle reports already settled. So if the runtime retries the whole fast step, nothing is charged, transcribed or written twice.

Step time limit

Workflow steps are deployed with the plan's maximum duration, 800 s on this plan.

  • Inside the fast step, transcription only runs for audio up to 30 minutes, and only if the step has spent less than 2 minutes. On production a 30 minute file spends 4.2 s in extract.
  • Extraction only runs in the fast step if getting the file took less than 2 minutes.
  • So beyond what getting the file costs today, the fast step adds at most a couple of minutes of work. Longer files keep their transcription in its own step, as now.

Skipping extraction for audio the speech service accepts: not done

The speech service accepts most audio and video containers, but it reports the length it transcribed only after the work is done. So its duration cannot be what the hold is checked against before we pay for the call.

Today the hold is safe because of extraction itself. It re-encodes the audio at a fixed bitrate and caps it at what the hold covers, so what the speech service receives is bounded and measured by us.

Skipping it would mean trusting our reading of the original file. Local tests show that reading can be far from what a decoder plays:

File Our MP3 frame count Container header Full decode After extraction
MP3 with random bytes spliced between frames 60.11 s 60.30 s 60.08 s not needed
MP3 followed by layer II frames 10.03 s 70.03 s 130.01 s 130.01 s

The second file would be held and charged for 10 seconds while the speech service transcribes 130. The best case saving was about 0.4 s of extract time on MP3 sources only, so the step stays.

Measured

In-process, on the scratch database, the local object store, local ffmpeg and a local stand-in for the speech service, with no step overhead, the work is the same in both versions. Median of 8 runs, 4 for 30 minutes:

Clip Before After
13 s 152 ms 138 ms
1 min 195 ms 182 ms
5 min 457 ms 447 ms
30 min 2,065 ms 2,050 ms

The small saving is the audio no longer being read back from storage. The win is the step overhead. Modelled with the production medians above, start to complete:

Length Steps before complete Modelled before Modelled after
up to 30 min 5 to 1 5.1 s 2.0 s
over 30 min 5 to 3 measured 40.6 to 61.0 s for 2 h about 1.6 s less

Applied to the production runs: 10 s files 3.5 to 5.3 s would become about 1.5 to 2.2 s, and 30 minutes 9.6 and 14.9 s about 6.5 and 11.8 s. The production matrix should be run again after merge.

Proof

What Result
Fast path tests (new file, cannot load on the code before this change) 18 pass
Workflow, holds, no speech, download, video and settle tests through the new path, unchanged apart from two delete tests all pass
Developer, infra and agent unit suites 2,251 pass
Real MySQL scratch database: transcripts suite and holds suite (the holds suite runs the workflow end to end) 46 of 46
Workflow bundle build builds, runTranscriptFastPath is registered, and the transcribe module in the flow bundle has no database code
Typecheck and Biome clean, apart from one known unrelated typecheck error

The fast path tests cover:

  • A short file completes in one call of the fast step: claimed, held, extracted, transcribed and charged once, with no audio read back from storage.
  • The whole workflow does nothing twice after the fast step: one speech call, one extraction, one write of the audio and of the result, one ledger row.
  • Running the fast step again after it finished, or after its settle failed, does not transcribe, write or charge twice.
  • A hold wait, a hold raise, audio over 30 minutes, and each 2 minute budget hand over at the right stage. The rest of the workflow finishes with the same charge.
  • Failure codes and messages are unchanged for an unusable file and for a speech service that keeps failing.

The existing tests still show:

  • The hold waits keep their exact sleep sequence.
  • A delete during the speech call is still charged, and a delete before it is free.
  • A video transcript still writes its captions.

Two delete tests changed. One triggered its delete on the audio read-back, which no longer happens, so it now triggers on the last storage call before the speech service. The other expected the workflow to throw when the speech call failed after a delete. Its transcription is now handed to the transcription step, which sees the delete and ends the transcript with nothing charged. That is what production already does once the runtime retries the step.

Not covered

  • The real workflow runtime was not run, because that needs a dev server or a full build. Step boundaries are shown by calling the fast step directly and by the built manifest, and the overhead is modelled from production rather than measured.
  • When a stage throws inside the fast step, the separate step tries it again with its own retries. So a speech service that keeps failing is called once more than before before the transcript fails.
  • The 30 minute and 2 minute limits are conservative. Raising them would save about 1.6 s more on long files.
  • Getting a file from a URL can take up to 15 minutes, which is longer than a step may run. That was already true of the prepare step before this change.

After review

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no new actionable issue was established.

Summary

Short transcripts now claim, prepare, extract, transcribe, and settle in one workflow step. Longer work and interrupted stages continue through the existing separate steps.

  • Short developer transcripts finish their main work in one workflow step.
  • Deleting a transcript keeps its files available while its run is processing.

Reviews (7) · Last reviewed commit: "Merge branch 'main' into perf/infra-tran..." · Reviewed by Greptile

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 2 potential issues (2 medium).

Open in Capy

@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
cap-status Skipped Skipped Nov 13, 2026 6:03am UTC

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 02:47 Inactive
Comment thread apps/web/content/docs/api/rest-api.mdx Resolved

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 02:53 Inactive
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 1 other finding in Capy.

Open in Capy

@sandchest-com

sandchest-com Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Sandchest

Sandchest reviewed 0d9ea0e: nothing exploitable found in this change.

Sandchest reviews every change for vulnerabilities an attacker could use · 12 credits · Review details

…-fast-path

# Conflicts:
#	apps/web/__tests__/unit/developer-transcript-docs.test.ts
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 05:42 Inactive
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 05:48 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 06:00 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 06:03 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@richiemcilroy
richiemcilroy merged commit 08e5619 into main Nov 13, 2026
38 checks passed

This branch was successfully deployed

1 active and 1 inactive deployments
Preview – cap-web — 0d9ea0e3 Deployed Nov 13, 2026 by vercel[bot]
Preview – cap-status — 0d9ea0e3 Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants