ssh, internal: flush the worker's queued output on every call - #1217
Conversation
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #1217
Scan targets checked: wolfssh-bugs, wolfssh-src
Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
b9d4cc3 to
a650544
Compare
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #1217
Scan targets checked: wolfssh-bugs, wolfssh-src
Findings: 2
2 finding(s) posted as inline comments (see file-level comments below)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
a650544 to
7b1ec65
Compare
7b1ec65 to
d95204d
Compare
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #1217
Scan targets checked: wolfssh-bugs, wolfssh-src
Fenrir result: Approved ✅
No new issues found in the changed files.
Advisory only — this automated result does not count as a GitHub approval.
Fenrir's latest completed scan found no issues; clearing the prior automated change request.
d95204d to
b598ecc
Compare
b598ecc to
e2839fa
Compare
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #1217
Scan targets checked: wolfssh-bugs, wolfssh-src
Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
e2839fa to
e5524ca
Compare
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #1217
Scan targets checked: wolfssh-bugs, wolfssh-src
Findings: 2
2 finding(s) posted as inline comments (see file-level comments below)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
e5524ca to
d15c978
Compare
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #1217
Scan targets checked: wolfssh-src, wolfssh-bugs
Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
97c5d53 to
c2749bf
Compare
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #1217
Scan targets checked: wolfssh-src, wolfssh-bugs
Findings: 3
3 finding(s) posted as inline comments (see file-level comments below)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
- The echoserver and Espressif shell loops and the Windows wolfsshd shell loop treat a WS_WANT_WRITE from wolfSSH_worker() as non-fatal.
4c86666 to
14af093
Compare
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #1217
Scan targets checked: wolfssh-src, wolfssh-bugs
Findings: 4
4 finding(s) posted as inline comments (see file-level comments below)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
14af093 to
92dc1c8
Compare
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #1217
Scan targets checked: wolfssh-src, wolfssh-bugs
Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
92dc1c8 to
9f00f62
Compare
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #1217
Scan targets checked: wolfssh-src, wolfssh-bugs
Findings: 2
2 finding(s) posted as inline comments (see file-level comments below)
This review was generated automatically by Fenrir. Reported findings require changes before merge.
- wolfSSH_worker() calls wolfSSH_SendPacket() whenever ssh->outputBuffer holds bytes and the session is not disconnected. ssh->error keeps the receive's code when the receive failed, and the close's when a WS_CHANNEL_CLOSED pass hard-failed its flush; WS_REKEYING is withheld on a failed flush. Drops the second DoReceive(), the WOLFSSH_TEST_BLOCK fork and the separate WS_CHANNEL_CLOSED flush. - BundlePacket() resets ssh->outputBuffer.length to ssh->packetStartIdx when the framing fails. wolfSSH_shutdown() reports WS_WANT_WRITE when its close-read leaves output queued, and the send's own error in place of it when that send failed. SendPacketFlush() records its code in ssh->error on every transport failure path, and wolfSSH_TriggerKeyExchange() writes it only when SendKexInit() fails. - portfwd, client and scpclient accept WS_WANT_WRITE from wolfSSH_shutdown(); in scpclient the close-message drain runs on it. - wolfssh/ssh.h drops WS_WINDOW_FULL from wolfSSH_worker() and says to read the return and wolfSSH_get_error() as independent channels on every pass. - Twenty unit tests and the extended TestWorkerReportsDisconnect cover what ret and ssh->error hold after a receive, send, buffer, callback or framing failure.
- tcp_select_write() joins tcp_select(), with WS_SELECT_SEND_READY at the end of the enum. tcp_select() passes NULL for writefds and cannot wait on the write side. - The Windows wolfsshd window-change drain and sftp_worker()'s handshake flush retry wait on it. The drain records a give-up in ret, and sftp_worker() breaks only on WS_SELECT_ERROR_READY. - The echoserver, Espressif and both wolfsshd shell loops, and ReceiveScpMessage(), take the event from wolfSSH_worker()'s return when it carries one, in place of reading ssh->error alone. The two echoservers cover WS_CHAN_RXD, WS_REKEYING, WS_CHANNEL_CLOSED and WS_EOF; the wolfsshd loops and ReceiveScpMessage() cover the ones they have arms for. - The POSIX wolfsshd loop takes an owed write into wantWrite before the event overwrites rc, and its WS_WANT_WRITE arm runs the channel drain below in place of skipping it. - FlushQueuedSend() folds the receive's own statuses into WS_SUCCESS inside its loop and keeps flushing while wolfSSH_get_error() reports WS_WANT_WRITE within the deadline, in place of looping on the worker's return. It reports WS_WANT_WRITE when the deadline leaves the packet queued.
ReceiveScpConfirmation() takes a WS_EXTDATA off ScpStreamRead()'s return, the rule ssh.h now states for reading an event. FlushQueuedSend() samples the owed write with the session still locked, since the peer reader thread writes ssh->error too, and reports a send that failed there over the event it rode in on. - A pass that delivers stderr while its flush short-writes returns WS_EXTDATA with WS_WANT_WRITE in ssh->error, so the confirmation read aborted the transfer with the stderr undrained. - wolfSSH_stream_read() on the reader thread clears ssh->error with this thread's packet still queued, which ended the flush early and reported it a success. - An event return with a hard send error latched collapsed to WS_SUCCESS, telling the caller a queued packet reached a socket that was gone.
- ScpStreamRead() calls _DumpExtendedData() and reads again when wolfSSH_stream_read() returns WS_EXTDATA, reporting the drain's own failure when it has one. - ReceiveScpConfirmation() drops its WS_EXTDATA arm and reports a negative read directly. - ReceiveScpMessage() returns _DumpExtendedData()'s failure in place of discarding it.
- wolfSSH_OutputPending() moves from wolfssh/internal.h to wolfssh/ssh.h as a WOLFSSH_API taking a const WOLFSSH*, defined in src/ssh.c beside the other public calls. - wolfSSH_worker() puts the send's code in the return in place of an event when the flush fails outright, and gates its flush on wolfSSH_OutputPending(). wolfssh/ssh.h states both. - Both wolfsshd shell loops, both echoservers and ReceiveScpMessage() dispatch on wolfSSH_worker()'s return, and call wolfSSH_get_error() only to tell a transient failure from a terminal one. The POSIX wolfsshd loop sets wantWrite from wolfSSH_OutputPending(). - Five worker tests expect the send's code where they expected the event, and tests/testsuite.c calls wolfSSH_OutputPending().
9f00f62 to
dd64402
Compare
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #1217
Scan targets checked: wolfssh-src, wolfssh-bugs
Findings: 3
3 finding(s) posted as inline comments (see file-level comments below)
💬 1 finding(s) from an earlier review are still open and were not re-posted:
- ESP-IDF echoserver stalls on queued output —
ide/Espressif/ESP-IDF/examples/wolfssh_echoserver/main/echoserver.c:1172
This review was generated automatically by Fenrir. Reported findings require changes before merge.
Last review turned up already deferred issues.
Problem
A read-only application on a non-blocking socket stalls permanently. A channel read credits the window,
ChannelCreditWindow()queues aCHANNEL_WINDOW_ADJUST, and the socket write blocks; the peer has spent its window and goes silent waiting for it. The application callswolfSSH_worker(), aswolfssh/ssh.hdirects — but the worker gated its flush onDoReceive()'s return, and an idle socket makesDoReceive()returnWS_FATAL_ERROR, which the gate did not list. No write is attempted, on that call or any later one.The fix (
src/ssh.c)wolfSSH_worker()flushes whenever output is queued and the session is live:ssh->errorkeeps the receive's code when the receive failed, and the close's when aWS_CHANNEL_CLOSEDpass hard-failed its flush; every other status keeps the send's, andWS_REKEYINGis withheld on a failed flush. The secondDoReceive(), itsWS_WINDOW_FULLarm, theWOLFSSH_TEST_BLOCKfork and the separateWS_CHANNEL_CLOSEDflush all go — they existed to work around the gate. Ignoringretis deliberate: an idle receive and a hard one both surface asWS_FATAL_ERROR, so a narrower gate would reintroduce the same bug.SendPacketFlush()now records its code on every transport failure, so two rules hold onwolfSSH_SendPacket(): a later write tossh->erroron the same pass must be conditional on the flush succeeding, and success is never written there.What the wider flush exposed
BundlePacket()left an unpadded, un-MAC'd partial packet queued on a framing failure. The old gate never sent it; the new one would. Its failure arm drops it.wolfSSH_shutdown()mapped its close-read'sWS_CHAN_RXDandWS_EOFtoWS_SUCCESS, reporting a clean teardown with bytes owed. It now reportsWS_WANT_WRITE, or the send's error when that send failed.ReceiveScpMessage()found the event throughwolfSSH_get_error(), which the flush now overwrites, so aWS_EXTDATApass fell todefault:and stranded the peer's stderr. It takes the event from the return.ScpStreamRead()handedWS_EXTDATAto all three of its callers as an error: a confirmation read reported success without reading one, and a file-body read aborted mid-transfer. It drains the stderr and reads again. This predates the PR.Consumers
A queued write now reports
WS_WANT_WRITEwhere it reportedWS_WANT_READ, and the return carries the event rather than the flush status.WS_WANT_READas fatal.wolfsshddrain andsftp_worker()'s retry retried with no wait at all; both now use a newtcp_select_write()inwolfssh/test.hand continue only while it reportsWS_SELECT_SEND_READY.ReceiveScpMessage()dispatched offwolfSSH_get_error(), so aWS_CHAN_RXDpass with a blocked flush skipped the drain. They now take the event from the return, the owed write fromwolfSSH_OutputPending(), and readwolfSSH_get_error()only to tell a transient failure from a terminal one.FlushQueuedSend()exited on the first pass once the return carried the event, reporting success with bytes queued. It sampleswolfSSH_get_error()inside the mutex and reports a failed send over the event it rode in on.scpclientgated its close-message drain on the shutdown result, so the newWS_WANT_WRITEclosed the socket with the reply queued. Only that return is fixed: aWS_CHANNEL_CLOSEDteardown still skips the drain and closes with our close reply unsent, which is a follow-up.portfwd,clientandscpclientalso acceptWS_WANT_WRITEfromwolfSSH_shutdown().ssh.hnow tells callers to read the return andwolfSSH_get_error()as independent channels on every pass, rather than enumerating which value lands where;WS_WINDOW_FULLcomes off the return list. Master's new application-driven channel mode hands the session towolfSSH_worker()after user auth, so out-of-tree callers land on this contract directly.One new public API
wolfSSH_OutputPending()becomesWOLFSSH_API. An application had no way to ask whether a write was still owed except by readingssh->error, a field written twice a pass — by the receive, then by the flush. OpenSSH answers this withssh_packet_have_data_to_write()and OpenSSL withSSL_want_write(); wolfSSH had the same predicate and kept it private. The library now gates its own flush on it.A send that fails outright also takes
wolfSSH_worker()'s return in place of the event it arrived with; onlyWS_CHANNEL_CLOSEDkeeps the return there.Tests
Twenty new unit tests cover the flush on an idle receive, the owed flush across calls, and what
retandssh->errorhold after a receive, send, buffer, callback or framing failure — including a cleanWS_SUCCESSreceive whose flush short-writes, driven by a peer window adjust, which was the last uncovered arm.The
WS_CHANNEL_CLOSED-with-a-short-flush arm is covered, bytest_DoChannelCloseWantWrite(), which five reviews have now called untested. Deleting&& sendRet != WS_WANT_WRITEdrops the suite to 164/165 with that test the only failure.Verification
unit.test165 passed / 0 failed;regress.testpassed. Clean under gcc-13-Werroracross 6 configurations, plus lint.-DWOLFSSH_TEST_BLOCK,scripts/sftp.test) passes atWOLFSSH_BLOCK_PROB70/50/30 in 47s/21s/12s, re-run on the current base.FlushQueuedSend():scripts/scp.testskips underWOLFSSH_TEST_BLOCK, andFlushQueuedSend()isstaticin a file no test binary links. Recorded as follow-ups.Known limitations, not addressed here
Six caller sites mishandle an owed flush, all inherited, and deferred because none has a test that can verify a fix. For the three shell loops the consequence does change: where a short flush used to break the loop and drop the session, tolerating
WS_WANT_WRITEleaves it waiting on a read-onlyselect(). On the idle-receive path this PR fixes, master parked there too.examples/echoserver/echoserver.c:1023select()watches read fds with aNULLtimeoutide/Espressif/.../echoserver.c:984apps/wolfsshd/wolfsshd.c:2251select()has an 800us timeout, but the guard below gates the worker onrc != 0, so a timeout iteration never retriesexamples/client/client.c:1213and:1230WS_FATAL_ERROR, so an idle receive reacheserr_sys(). Master reached the same whitelist, so this PR does not change its reachabilityexamples/echoserver/echoserver.c:1566continueskips thetcp_select()below it, and its||re-enters on any pending send, so it spins rather than waitside/Espressif/.../echoserver.c:1464&&, so it skips the wait without the spinssh_worker()has a separate open finding, 10544; both patch the same function, so that lands first.