Repository navigation
cat: report EBADF for a write-only stdin - #15953
eylulseyit wants to merge 1 commit into
Conversation
|
please convert to draft until it is ready |
Merging this PR will not alter performance1 warning, 1 note
Comparing Footnotes
|
| // std's Stdin reports EBADF as end of file, so a write-only stdin | ||
| // would be read as empty. Read the descriptor directly instead. |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
| // std's Stdin reports EBADF as end of file, so a write-only stdin | ||
| // would be read as empty. Read the descriptor directly instead. | ||
| #[cfg(any(unix, target_os = "wasi"))] | ||
| let stdin = uucore::io::RawReader(stdin); |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
There was a problem hiding this comment.
If I understood correctly, I think buffering doesn't change. std's Stdin buffer is 8 KiB and it is skipped for reads at least that big, and cat already reads 64 KiB (31 KiB with -n). I also counted read syscalls with /proc//io for cat -n, it was 43 reads for 1.3 MB both before and after.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
There was a problem hiding this comment.
yes, I checked them all now. I piped 1.3 MB into cat and counted read syscalls in /proc//io, old build (main) vs this branch:
- plain cat and -u: 0 reads, it uses splice
- -A -b -e -E -n -s -t -T -v: 43 reads (31 KiB each)
- plain cat and -u with
>> fileso splice is not used: 22 reads (64 KiB each)
the numbers are the same before and after in all cases.
what I did not check: only Linux, I did not test macOS/BSD (no splice there, but they use the same read loop as the >> file case), a terminal as stdin, or timing. I only counted reads.
There was a problem hiding this comment.
thanks for the review, your comments helped a lot
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
There was a problem hiding this comment.
tried it like this, for each option:
(for i in 1 2 3; do sleep 1; printf 'hello\nworld'; done) | cat -n | python3 -c 'import os,time; t=time.monotonic()
while b := os.read(0, 65536): print(f"+{time.monotonic()-t:.1f}s", b)'
output for -n, same on main and this branch:
+1.0s b' 1\thello\n 2\tworld'
+2.0s b'hello\n 3\tworld'
+3.0s b'hello\n 4\tworld'
all options (none, -u -A -b -e -E -n -s -t -T -v) print each chunk right away at +1s +2s +3s, same on main and this branch, and same as GNU cat for plain and -n. Is it ok?
| } | ||
| // std's Stdin reports EBADF as end of file, so a write-only stdin | ||
| // would be read as empty. Read the descriptor directly instead. | ||
| #[cfg(any(unix, target_os = "wasi"))] |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
| // EOF (GNU opens /dev/null write-only). Since nohup execs the command, the | ||
| // exit status is the command's own. | ||
| // exit status is the command's own. Run our own cat rather than the one in | ||
| // PATH, which may be an older uutils cat that reads such a stdin as empty. |
There was a problem hiding this comment.
I don't think this change is needed since portable test is always better.
There was a problem hiding this comment.
I thought the system cat on ubuntu-26.04 runners is uutils, and an older one with this bug. So with the portable version, the nohup test would keep failing there until Ubuntu updates its cat, even after this is merged. Is that ok, or should the test stay failing on those runners for now? I can drop this commit if you prefer the portable test.
There was a problem hiding this comment.
I meant your test is better (portable), but comment can be reverted.
There was a problem hiding this comment.
Oh sorry I understand now
There was a problem hiding this comment.
I thought the system cat on ubuntu-26.04 runners is uutils
@oech3 We should really switch to GNU coreutils for all runners.
There was a problem hiding this comment.
I don't think that it is necessary. If our pure-Rust tests are not portable, it is our fault and they should be improved instead of working around at system side.
There was a problem hiding this comment.
Also it means we abandon actual people on Ubuntu running our tests.
There was a problem hiding this comment.
Wait maintainer's response.
I'm looking at alternative for cat which is 100% installed on unix or Rust toolchain which fails with nohup cmd. But compiling something for this is overkill.
| // PATH, which may be an older uutils cat that reads such a stdin as empty. | ||
| #[test] | ||
| #[cfg(unix)] | ||
| #[cfg(all(unix, feature = "cat"))] |
There was a problem hiding this comment.
this is the opposite of portable, we shouldn't be depending on our own utils for unrelated tests
There was a problem hiding this comment.
Sorry, I don't understand. Previous test depends on system binary which is not portable.
System cat can be busybox, BSD, vlang-coreutils, etc...
This PR made the test reproducible and our cat is verified at different test.
There was a problem hiding this comment.
My point is that using uutils cat from main is neither portable nor reproducible. Its behavior can change independently of the test, meaning the test results may depend on the current version of another utility instead of just the code being tested.
As for system cat being BusyBox, BSD, vlang-coreutils, etc., that's exactly what portability means: the test should work across different implementations, not replace that variation with a dependency on our own utility.
There was a problem hiding this comment.
Why do you 100% believe that system coreutils is not buggy? It is not portable.
This PR adds test case for cat which means we pass the CI if and only if we are using reliable cat.
We can catch false-negative too if we rely on system cat.
There was a problem hiding this comment.
Sorry, I’m not sure I understand your point. Are you saying that GNU coreutils are not portable? Could you clarify what you mean by "portable" in this context?
There was a problem hiding this comment.
Don't asume that system coreutils is GNU coreutils. Also don't assume that GNU coreutils does not have bug too.
4bf739f to
4fb7db4
Compare
4fb7db4 to
71da60d
Compare
|
rebased and dropped the nohup test commit, since #15944 handles that test now. |
| } | ||
| #[cfg(any(unix, target_os = "wasi"))] | ||
| impl<T: AsFd> AsFd for RawReader<T> { | ||
| fn as_fd(&self) -> BorrowedFd<'_> { |
There was a problem hiding this comment.
I don't think RawReader should implement AsFd. Since it's an abstraction over a file descriptor, exposing the underlying file descriptor defeats that abstraction.
There was a problem hiding this comment.
This is actually used at Linux and Android.
Aim of RawReader is removing buffering only. It is not intended to refuse using as fd.
There was a problem hiding this comment.
I added it because cat's InputHandle needs Read + AsFd (for splice and the input == output check), so RawReader didn't fit there without it. As far as I understand, the inner fd is already reachable through the pub field, so I don't think it exposes anything new, but I may be missing something. If you'd prefer not to have it in uucore, I can wrap it in a small type inside cat instead.
There was a problem hiding this comment.
I'm fine to add it since it is same with std::io::stdin().as_fd(). We could use stdin.0, but it is difficult to understand.
reading a write-only stdin fails with EBADF, but std's Stdin reads it as end of file. So cat printed nothing and exited with 0. Read stdin with uucore's RawReader instead and added AsFd to RawReader so cat can use it. Closes uutils#15937
71da60d to
78dbff5
Compare
When stdin is open but write-only, reading it fails with EBADF. GNU cat reports that and exits 1; uutils cat printed nothing and exited 0.
The Rust standard library's
Stdintreats EBADF as end of file, so the error never reached cat. cat now reads stdin throughuucore::io::RawReaderon unix and wasi, which needed anAsFdimpl forRawReader. Windows is unchanged.test_nohup_replaced_stdin_is_not_readableran the systemcat, which on ubuntu-26.04 runners is uutils and has this bug, so it failed there. The test now runs the uutilscatthat was just built, so it checks this fix and no longer depends on the runner.Closes #15937