feat(ocr): show scan progress and support whole-image recognition - #350
Merged
Conversation
Recognition ran on a worker with nothing on screen to say the region had even been read, and a toast afterwards that gave no sense of where it came from. A band now sweeps the rectangle while the worker runs, and a short card beside it reports the outcome. The card carries an outcome and a character count, never the recognized text. `src/ocr` keeps screen contents out of application state entirely — the worker publishes to the clipboard inside one stack frame and reports back only a count — and painting a transcript here would put exactly what that invariant exists to exclude into the overlay's own UI. `Ctrl+A` now takes the whole displayed image for recognition as well as for capture. Reading a full screen of text should not need a drag across the whole output. Measure is the only purpose left out, having nothing to submit. Details worth stating: - The hand-over deadline is fixed when recognition settles, not recomputed per tick. A deadline derived from the current elapsed time moves along with it and is only ever met on an exact sweep boundary, which a frame clock does not hit: the band would sweep forever and the card would never appear. - The overlay starts only for a request the controller accepted, and carries its id. A refused submission produces no completion, and an overlay waiting on one would sweep until something dismissed it. - Under `[ui] reduced_motion` the region takes a static tint instead of a moving band, the result shows as soon as it arrives rather than waiting out an animation that never ran, and the overlay asks for no frames — one deadline expires the card instead of pinning a repaint for its whole life. - Dismissal takes a finished card, not a running sweep: the sweep is progress feedback for work still in flight, and dropping it would discard the result about to arrive. Pointer, touch, stylus and keyboard all dismiss, the stylus before its surface-specific routing so a press on the Review bar or a toolbar counts too. Both entry points share one submit transaction, so a fix to the dragged-region path cannot leave whole-image recognition behind.
Whole-image recognition is a keystroke with nothing on screen to suggest it, so `Ctrl+A` was findable only by reading the documentation. The gap was wider than one key. Recognition never went through the capture picker at all — `render_capture_picker` gates on capture and measure purposes, so OCR falls to its own overlay of a scrim and scan brackets — and every affordance the picker gained, the hint strip included, simply never reached it. The strip is now shared rather than copied, so the two selectors cannot drift apart in wording or position, and it honours the same `show_legend` setting. Its text is its own: recognition's selection policy sets `allow_square: false`, so advertising `Shift: square` would be a lie, and `Ctrl+A` reads everything rather than selecting everything. Dismissal and damage both came for free, which is the sign the seam was in the right place: `begin_selection` already clears the legend for any purpose, and `mark_region_dirty` already marks full damage for every purpose but measure, so the frame that dismisses the strip is the frame that clears it.
Recognition shipped without a chord, and the stated reason — `O` is the orange quick colour — read as "we could not find a key" rather than "it should not have one". Now that it has a scan overlay, a hint strip, whole-image `Ctrl+A`, a palette entry and a toolbar button, being keyless was the last thing making it feel optional. `Ctrl+Shift+X` for e**X**tract text. Every mnemonic was taken: `Ctrl+Shift+O` captures the active window, `Ctrl+Alt+O` opens the capture folder, `R` is spoken for in both modifier sets, and `Ctrl+Shift+T` returns to transparent. This keeps recognition in the `Ctrl+Shift+` block the rest of the capture family lives in, on a letter nothing else wants. Two candidates were rejected for reasons a free-letter search does not show: `Ctrl+Shift+U` is the IBus Unicode entry sequence, which collides with text input, and `Ctrl+Alt+T` opens a terminal nearly everywhere — the overlay holds keyboard focus and would win, quietly doing something else with a chord that deep in muscle memory. Additive, so nothing existing changes: anyone who already bound the action keeps their chord, and the new default stands down through the skipped-default path. The configurator's conflict fixture used `Ctrl+Shift+X` precisely because it was free, and its claimant count is exact. It moves to a chord that is still unclaimed and now asserts that up front, so the next default to take it fails there saying why.
Three leftovers from binding recognition to Ctrl+Shift+X. Two comments still said the action was unbound, and one of them argued against ever giving it a default. Both now describe the binding that exists. The configurator's conflict fixtures used Ctrl+Shift+X because nothing claimed it. One of them was more than untidy: `replace_removes_only_the_contested_binding` runs a replace that strips the contested chord from every claimant, so with a default on that chord it was silently rewriting `copy_text_from_screen` — a field the test never mentions and never asserts. It passed while exercising a polluted scenario. Every draft-based fixture moves to Ctrl+Shift+Q, which the conflict test now asserts is unclaimed so the next default to take it fails there with the reason. Chord parsing and formatting tests keep Ctrl+Shift+X: they take no draft and one of them is about that chord's canonical spelling. `render_region_legend` only forwarded to `draw_legend`, so the rename replaces the pair.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ctrl+Arecognize the whole displayed image without requiring a drag.Privacy and reliability
Documentation
Ctrl+Abehavior, progress indicator, result card, dismissal rules, and reduced-motion behavior.