Predict wide characters in the local echo overlay - #1410
Open
ventusff wants to merge 2 commits into
Open
Conversation
The prediction engine punted on any character with wcwidth != 1, so typing CJK text over a high-latency link always waited a full round trip for the echo, while ASCII was echoed locally. Predict width-2 characters the same way Emulator::print renders them: the cell under the cursor receives the character with its wide flag set, the overlapped neighbor cell is predicted blank, and the cursor advances two columns. A blank replacement is never judged incorrect by ConditionalOverlayCell::get_validity, so the overlapped-cell prediction cannot trigger false invalidations. Backspace and left/right arrows now move by the width of the character they cross. Wide characters within two columns of the right edge are not predicted, because terminals disagree on early-wrap behavior there.
Stop the server with SIGSTOP while a mixed ASCII/CJK string is typed with --predict=always, so anything that reaches the client screen can only have come from the prediction engine. The ASCII characters act as a control for the harness itself; the wide characters are the subject. A priming string is typed first, because predictions are not displayed until one of them has been confirmed, and it must not end in a carriage return, which starts a new tentative epoch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mosh's predictive local echo covers only single-width characters:
PredictionEngine::new_user_byte()gives up on anything withwcwidth != 1(the old/* XXX handle wide characters */). On a high-latency link, ASCII is echoed locally but every CJK character waits a full round trip for its echo. For users who type in Chinese, Japanese or Korean, this makes prediction — arguably mosh's best feature — inoperative for most of what they type.What this does
Width-2 characters are now predicted exactly the way
Emulator::print()will render them:Backspace and left/right arrows now move by the width of the character they cross (an active prediction is consulted first, then the framebuffer).
Why it is safe
Emulator::print().ConditionalOverlayCell::get_validity()never judges a blank replacement incorrect — that cell can lose credit, but cannot trigger a false invalidation.width - 1early, and downstream terminals disagree about that case, so those keystrokes just become tentative, as all wide characters did before.become_tentative().Test
src/tests/prediction-wide.teststops mosh-server with SIGSTOP while a mixed ASCII/CJK string is typed under--predict=always, so anything that reaches the client screen during that window can only have come from the prediction engine. The ASCII characters act as a control for the harness itself (they have always been predicted); the wide characters are the subject. A priming string is typed before the server is stopped, because predictions are not displayed until one of them has been confirmed — and it must not end in a carriage return, which starts a new tentative epoch.Verified in both directions: the test fails on master (
wide characters were not predicted) and passes with this change. The fullmake checkpasses (31 PASS, 2 XFAIL) on Linux/x86-64.Measurements
Typing into a fish shell over a ~190 ms RTT link (client on this branch, stock mosh 1.4.0 server), measuring from writing the bytes to the pty until the glyph appears in the terminal output, 12 keystrokes per run:
(The max in the "after" column is the first keystroke after a carriage return, which is tentative by design.)
Final screens were also compared against plain ssh after mixed CJK/ASCII editing — typing, backspacing over wide characters, arrow-key movement with insertion — and after force-wrapping a 64-CJK-character line in an 80-column window: identical in both cases.
Not covered
Prediction of combining characters is unchanged, and input arriving in reads larger than 100 bytes is still treated as a paste and not predicted, as before.