Skip to content

Predict wide characters in the local echo overlay - #1410

Open
ventusff wants to merge 2 commits into
mobile-shell:masterfrom
ventusff:wide-char-prediction
Open

Predict wide characters in the local echo overlay#1410
ventusff wants to merge 2 commits into
mobile-shell:masterfrom
ventusff:wide-char-prediction

Conversation

@ventusff

@ventusff ventusff commented Sep 1, 2026

Copy link
Copy Markdown

Mosh's predictive local echo covers only single-width characters: PredictionEngine::new_user_byte() gives up on anything with wcwidth != 1 (the old /* XXX handle wide characters */). On a high-latency link, ASCII is echoed locally but every CJK character waits a full round trip for its echo. For users who type in Chinese, Japanese or Korean, this makes prediction — arguably mosh's best feature — inoperative for most of what they type.

What this does

Width-2 characters are now predicted exactly the way Emulator::print() will render them:

  • the cell under the cursor receives the character with its wide flag set;
  • the overlapped neighbor cell is predicted blank, mirroring the emulator's erase of that cell;
  • the cursor advances two columns, and the insert shift moves the tail of the row by two columns.

Backspace and left/right arrows now move by the width of the character they cross (an active prediction is consulted first, then the framebuffer).

Why it is safe

  • The prediction has to agree with the emulator exactly, so the width computation is copied verbatim from Emulator::print().
  • The overlapped-cell prediction is blank, and ConditionalOverlayCell::get_validity() never judges a blank replacement incorrect — that cell can lose credit, but cannot trigger a false invalidation.
  • Wide characters within two columns of the right edge are not predicted: the emulator wraps a 2-cell character at column width - 1 early, and downstream terminals disagree about that case, so those keystrokes just become tentative, as all wide characters did before.
  • Combining characters (width 0) and other unprintables are unchanged: still become_tentative().

Test

src/tests/prediction-wide.test stops mosh-server with SIGSTOP while a mixed ASCII/CJK string is typed under --predict=always, so anything that reaches the client screen during that window can only have come from the prediction engine. The ASCII characters act as a control for the harness itself (they have always been predicted); the wide characters are the subject. A priming string is typed before the server is stopped, because predictions are not displayed until one of them has been confirmed — and it must not end in a carriage return, which starts a new tentative epoch.

Verified in both directions: the test fails on master (wide characters were not predicted) and passes with this change. The full make check passes (31 PASS, 2 XFAIL) on Linux/x86-64.

Measurements

Typing into a fish shell over a ~190 ms RTT link (client on this branch, stock mosh 1.4.0 server), measuring from writing the bytes to the pty until the glyph appears in the terminal output, 12 keystrokes per run:

before after
CJK character min 229 / median 269 / max 288 ms min 0 / median 0 / max 228 ms
ASCII (control) median 0 ms median 0 ms

(The max in the "after" column is the first keystroke after a carriage return, which is tentative by design.)

Final screens were also compared against plain ssh after mixed CJK/ASCII editing — typing, backspacing over wide characters, arrow-key movement with insertion — and after force-wrapping a 64-CJK-character line in an 80-column window: identical in both cases.

Not covered

Prediction of combining characters is unchanged, and input arriving in reads larger than 100 bytes is still treated as a paste and not predicted, as before.

The prediction engine punted on any character with wcwidth != 1, so
typing CJK text over a high-latency link always waited a full round
trip for the echo, while ASCII was echoed locally.

Predict width-2 characters the same way Emulator::print renders them:
the cell under the cursor receives the character with its wide flag
set, the overlapped neighbor cell is predicted blank, and the cursor
advances two columns.  A blank replacement is never judged incorrect
by ConditionalOverlayCell::get_validity, so the overlapped-cell
prediction cannot trigger false invalidations.  Backspace and
left/right arrows now move by the width of the character they cross.

Wide characters within two columns of the right edge are not
predicted, because terminals disagree on early-wrap behavior there.
Stop the server with SIGSTOP while a mixed ASCII/CJK string is typed
with --predict=always, so anything that reaches the client screen can
only have come from the prediction engine.  The ASCII characters act
as a control for the harness itself; the wide characters are the
subject.  A priming string is typed first, because predictions are
not displayed until one of them has been confirmed, and it must not
end in a carriage return, which starts a new tentative epoch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant