SFTP: Cyberduck (sshj) connection dropped on initial REALPATH/STAT — server errors `SshEncoding: length invalid`

[Yaminyam]

Summary

Cyberduck (and likely other sshj/Java-based SFTP clients) cannot list any directory against bssh-server: right after the SFTP handshake, on the client's initial REALPATH + STAT of the home directory, the server drops the connection with:

Session error  error=SshEncoding: length invalid

The OpenSSH sftp CLI and FileZilla work fine against the exact same server, config, user, and directory — so this is specific to the request/response pattern that sshj-based clients use.

Client-side symptom (Cyberduck)

디렉터리 sion의 목록을 얻어오는데 실패하였습니다. (127.0.0.1 – SFTP)
Broken transport; encountered EOF. The connection attempt was rejected.
The server may be down, or your network may not be properly configured.

Server-side log (bssh-server run -vv)

INFO  bssh::server::handler: Starting SFTP session user=sion peer=127.0.0.1:54766 home=/Users/sion
DEBUG bssh::server::sftp: Creating SFTP handler user=sion chroot=None cwd=/Users/sion
INFO  bssh::server::sftp: SFTP session initialized user=sion version=3
DEBUG bssh_russh_sftp::protocol: packet type 16      # SSH_FXP_REALPATH
DEBUG bssh_russh_sftp::protocol: packet type 17      # SSH_FXP_STAT
DEBUG russh::server: Connection closed with error
DEBUG bssh_russh_sftp::server: sftp stream ended
ERROR bssh::server: Session error error=SshEncoding: length invalid

The connection dies immediately after the REALPATH (16) / STAT (17) exchange — before any OPENDIR/READDIR. So the server appears to mis-encode the REALPATH/STAT response (an SSH_FXP_NAME / SSH_FXP_ATTRS packet) such that the outgoing SSH channel-data length is invalid, and russh's transport encoder aborts the connection.

Repro

1. bssh-server run -c config.yaml -D (no chroot: sftp.root unset; auth publickey or password; SFTP protocol v3 negotiated).
2. Connect with Cyberduck (SFTP, 127.0.0.1, the configured user) → fails as above.
3. Connect with OpenSSH sftp or FileZilla to the same server → works (list, get, put, rename, etc. all fine).

Impact

Many GUI/mobile SFTP clients are sshj/JSch-based (Cyberduck, and others). If they all hit this, bssh-server cannot be a drop-in SFTP endpoint for those clients.

Environment

• bssh-server 2.2.3 (bssh-server-macos-aarch64)
• SFTP protocol version 3
• Failing client: Cyberduck (sshj); working clients: OpenSSH sftp, FileZilla

[Yaminyam]

Root-cause investigation

This is very likely the same class of bug that historically bit OpenSSH sftp-server + Dropbear with sshj/JSch clients (PyCharm, Cyberduck). Two pieces of prior art:

1. OpenSSH sftp-server hard cap. It rejects any SFTP message whose declared length exceeds a build-time constant:

/* Maximum packet that we are willing to send/accept */
#define SFTP_MAX_MSG_LENGTH (256 * 1024)
...
if (msg_len > SFTP_MAX_MSG_LENGTH) {
    error("bad message from %s local user %s", client_addr, pw->pw_name);
    sftp_server_cleanup_exit(11);
}

When the SSH transport was throughput-tuned (bigger window/payload), sshj/JSch clients ended up putting a packet-header size larger than the server's max and the server killed the connection.

2. Dropbear rollback (backend.ai-agent 7f33f4f, "Rollback SFTP tput optimization due to PyCharm compat issue"): they had bumped DEFAULT_RECV_WINDOW=262144, RECV_MAX_PAYLOAD_LEN=327680, TRANS_MAX_PAYLOAD_LEN=327680 for throughput, which "PyCharm's SSHJ/JSch libraries and throughput-optimized Dropbear breaks with each other, while other SFTP clients work consistently fine". Fix was to revert those to defaults.

Where this maps in bssh

• crates/bssh-russh-sftp/src/server/mod.rs:43 — max_client_packet_len: 262144 (the analog of SFTP_MAX_MSG_LENGTH).
• src/server/sftp.rs:184 — MAX_READ_SIZE = 261120 (255 KiB); bssh is throughput-optimized (255 KiB reads, request pipelining) — same profile that broke sshj on Dropbear.

Failure sequence (server -vv), same channel id

Subsystem request subsystem=sftp channel=ChannelId(2)
Starting SFTP session user=sion home=/Users/sion
SFTP session initialized version=3
bssh_russh_sftp::protocol: packet type 16      # REALPATH
Received data from client channel=ChannelId(2) bytes=14
No shell_data_tx found for channel, dropping data channel=ChannelId(2)   # <-- see below
bssh_russh_sftp::protocol: packet type 17      # STAT
Received data from client channel=ChannelId(2) bytes=24
No shell_data_tx found for channel, dropping data channel=ChannelId(2)
russh::server: Connection closed with error
bssh_russh_sftp::server: sftp stream ended
ERROR bssh::server: Session error error=SshEncoding: length invalid

Two suspects

1. Channel-data routing / windowing. subsystem_request runs SFTP via russh_sftp::server::run(channel.into_stream(), …) (src/server/handler.rs:1354), but the overridden Handler::data() (src/server/handler.rs:1379) also fires for the same SFTP channel and, finding no shell_data_tx, logs "No shell_data_tx found for channel, dropping data". So the SFTP channel's bytes are handled by into_stream() and by data() (which drops them). Whether or not this double-path corrupts the stream, the interaction between the custom data() override and into_stream()'s window accounting is the most likely source of the transport-level SshEncoding: length invalid. sshj's tighter request cadence (REALPATH immediately followed by STAT) appears to trigger it where OpenSSH sftp/FileZilla do not.

2. Packet-size negotiation, per the Dropbear precedent — if bssh advertises a large SFTP limit / SSH window that leads sshj to frame a packet the transport later can't encode/decode.

Repro

• Server: bssh-server run (no chroot), SFTP v3.
• Cyberduck (sshj) → connection dropped on first listing, client shows Broken transport; encountered EOF.
• OpenSSH sftp / FileZilla against the same server → fine.

Happy to help bisect with an sshj-based minimal client if useful.

[Yaminyam]

Narrowing: not the SFTP request logic — sshj/transport-specific

Tried to reproduce with paramiko (Python) against the same server, including the exact REALPATH → STAT(home) cadence the Cyberduck/sshj session used:

realpath then stat(home): OK
realpath then stat('.'):   OK
stat first:                OK
lstat:                     OK
stat then listdir:         OK
normalize + listdir:       OK

All pass. So the SSH_FXP_REALPATH/SSH_FXP_STAT handling itself is fine — paramiko cannot reproduce it. The failure is specific to sshj/JSch and manifests as a russh transport-level error (SshEncoding: length invalid), not an SFTP-layer decode error. KEX (curve25519 / Ed25519) completes cleanly; the break is in the post-auth channel-data phase.

This points at an sshj-vs-russh interop issue at the SSH binary-packet layer (cipher/MAC/packet-length framing, or the custom Handler::data() override interacting with channel.into_stream() windowing), rather than anything in the SFTP request handlers.

Note: the SSH window/max-packet in the failing session are russh defaults (window=2097152, max_packet=32768), so this is not an over-tuned-window case like the Dropbear precedent — the throughput tuning in bssh is at the SFTP layer (255 KiB reads), not the SSH window.

A faithful minimal repro needs an actual sshj client (Java). Cyberduck is a reliable manual repro in the meantime.

[Yaminyam]

Root cause + fix

The SFTP subsystem never sent the server-side channel EOF/CLOSE when the session ended: it ran russh_sftp::server::run(), which spawned the request loop on a detached task and returned immediately, so the handler had no completion hook (unlike the shell/exec/SCP paths, which send exit_status + eof + close). OpenSSH sftp/FileZilla tolerate the missing server CLOSE; sshj/JSch strictly wait for the server's SSH_MSG_CHANNEL_CLOSE in the close handshake, so it hangs.

Reproduced (scriptable)

Minimal sshj 0.38 client — canonicalize(".") → stat(home) → ls(home) → close():

• before: SFTPClient.close() times out after ~30s
• after fix: session completes and closes in ~1s

(paramiko couldn't reproduce it — it doesn't strictly block on the channel-close handshake — which is why this only surfaced with sshj-based clients like Cyberduck.)

Fix

Proposed in Yaminyam#1 (stacked on #212):

• bssh-russh-sftp: run_with_config runs the loop inline instead of tokio::spawn-ing internally, so the caller can observe completion.
• handler: run the SFTP session on our own task and, when it ends, send handle.eof() + handle.close().

Regression-checked: OpenSSH sftp (ls/get/put), paramiko, and cargo test -p bssh-russh-sftp all fine.

The run()-should-be-awaitable part is worth mirroring upstream in russh-sftp (the fork's parent) — the fire-and-forget spawn leaves callers with no way to clean up the SSH channel after the SFTP session.

Cyberduck re-test against the patched binary pending to confirm it also clears the original Broken transport; encountered EOF symptom.

[Yaminyam]

Resolution: fixed by the russh 0.62.1 upgrade (#212) — it's a russh (SSH channel) issue, not an SFTP one

After isolating with a scriptable sshj repro across russh versions:

build	sshj (canonicalize→stat→ls→close)
russh 0.61.1 (vendored fork and vanilla)	❌ ~30s (SFTPClient.close timeout)
russh 0.62.1 (#212), no extra patch	✅ ~1s

The bug is in the SSH channel-close layer, not SFTP. The SFTP protocol itself worked fine on all builds (REALPATH/STAT/READDIR succeed, sshj gets the full listing). The hang is the SSH channel-close handshake: the client sends SSH_MSG_CHANNEL_CLOSE and waits for the server's reply, which russh 0.61.1 never sends (RFC 4254 violation). sshj/JSch strictly wait for it; OpenSSH/FileZilla don't.

Fixed upstream in russh by 6fc20b28 "Reply with CHANNEL_CLOSE in server handler per RFC 4254" (Eugeny/russh#675) (+ #674 "Forward ChannelMsg::Close to channel before dropping sender"), first in russh 0.62.1. Verified: clean #212 (russh 0.62.1, no other change) resolves it.

So this is resolved by merging #212 (the russh 0.61.1 → 0.62.1 upgrade). No SFTP-layer change is needed:

• The bssh-russh-sftp / handler change I first proposed (fix(sftp): send channel EOF+CLOSE when the SFTP session ends (fixes #215, sshj/Cyberduck hang) #216) is redundant — russh 0.62.1 already replies with CHANNEL_CLOSE.
• The russh-sftp JoinHandle proposal (feat(server): return JoinHandle from run/run_with_config AspectUnk/russh-sftp#93) was aimed at the wrong layer and isn't needed either.

Closing #216 and AspectUnk/russh-sftp#93 accordingly.

[Yaminyam]

Correction + actual root cause (compression, not channel-close) — fixed in #217

My earlier comment claiming this was resolved by the russh 0.62.1 upgrade (#212) was wrong. Retesting a real Cyberduck client against a clean 0.62.1 build reproduced the exact same failure. The 30s channel-close hang I fixed there was a separate sshj-0.38 issue; it never reproduced Cyberduck's symptom, so I mis-attributed the fix. Apologies for the noise (#216 and the russh-sftp PR remain correctly closed as redundant).

Real root cause: russh delayed-zlib (zlib@openssh.com) stream desync

Full russh=trace on the failing Cyberduck session showed the negotiated algorithms:

negotiated: cipher: chacha20-poly1305@openssh.com,
            client_compression: ZlibOpenSSH, server_compression: ZlibOpenSSH

The handshake and first SFTP requests (INIT / REALPATH / STAT) all succeed, then the connection dies with SshEncoding: length invalid on the next inbound packet — because compression activates after auth and russh's flate2 stream desyncs a few packets later, so the following packet's length prefix is decoded from corrupted plaintext.

It's compression, not Cyberduck-specific and not the SFTP/channel-close layer:

client	compression	result
OpenSSH sftp (default)	none	✅
FileZilla / paramiko	none	✅
OpenSSH sftp -C	zlib@openssh.com	❌ SshEncoding: length invalid
Cyberduck 9.5	zlib@openssh.com	❌ same

One-command repro against an unpatched server:

sftp -C -P 2223 user@host      # -> "SshEncoding: length invalid", closed by remote

Reproduces on both russh 0.61.1 and 0.62.1.

Fix (#217)

Server advertises only none compression (russh::Preferred { compression: [NONE], ..DEFAULT }), so clients fall back to the uncompressed transport — matching the Dropbear / OpenSSH sftp-server defaults used in the container SFTP path anyway. Verified: sftp -C now negotiates compression: none and a live Cyberduck 9.5 session connects and browses fine.

The underlying russh delayed-zlib desync should still be reported/fixed upstream; #217 is the safe server-side workaround.