Skip to content

docs: clarify writableNeedDrain behavior and usage in streams#56419

Open
oussamaBGZ wants to merge 1 commit into
nodejs:mainfrom
oussamaBGZ:main
Open

docs: clarify writableNeedDrain behavior and usage in streams#56419
oussamaBGZ wants to merge 1 commit into
nodejs:mainfrom
oussamaBGZ:main

Conversation

@oussamaBGZ

@oussamaBGZ oussamaBGZ commented Dec 31, 2024

Copy link
Copy Markdown

This PR updates the documentation for the writableNeedDrain property in the streams module to clarify its behavior and guide developers on how to handle backpressure effectively.

Problem
Inconsistent handling of backpressure can lead to issues like incomplete data transfer.

For example:

  1. Correct Backpressure Handling: The following code correctly handles backpressure by using the return value of writable.write(chunk) and listens for the 'drain' event to resume the readable stream:
image
  1. Incorrect Backpressure Handling: The following code checks writableNeedDrain directly, which leads to incomplete copying (only 80% of the file is copied) because writableNeedDrain isn't updated immediately after each write():
image

@nodejs-github-bot nodejs-github-bot added doc Issues and PRs related to the documentations. stream Issues and PRs related to the stream subsystem. labels Dec 31, 2024
@lpinca

lpinca commented Dec 31, 2024

Copy link
Copy Markdown
Member

The problem in your example seems to be that the chunk of data is not written when writable.writableNeedDrain is true, so it is simply discarded.

@lpinca lpinca dismissed a stale review December 31, 2024 19:35

Approved too quickly without enough thought

@oussamaBGZ

Copy link
Copy Markdown
Author

The problem in your example seems to be that the chunk of data is not written when writable.writableNeedDrain is true, so it is simply discarded.

when its discarded the read stream is paused so technically we should not lost any data.
the example doesn't skip chunks intentionally, the problem lies in the timing of when writable.writableNeedDrain is updated.

@lpinca

lpinca commented Dec 31, 2024

Copy link
Copy Markdown
Member

when its discarded the read stream is paused so technically we should not lost any data

No, you get the chunk, the chunk it not written (else branch in your example), and then the stream is paused. The chunk is lost forever. You should at least write it before pausing the stream.

@oussamaBGZ

Copy link
Copy Markdown
Author

e chunk it not wri

Ahh ok thanks a lot, i added writable.write(chunk) before the pause func, the outpout file size was exactly the same. It was a bit tricky to notice hahaha maybe we could add a section in the docs to talk about situations like these

@Ethan-Arrowood

Copy link
Copy Markdown
Contributor

Hello, I'm doing some backlog hygiene on all PRs related to networking and streaming modules. There hasn't been any activity here for quite a while so I'm going to mark it stalled. It will auto-close in ~30 days. This is not a rejection; if you've like to pick it back up, rebase onto latest main and give me a ping.

Thank you!

@Ethan-Arrowood Ethan-Arrowood added the stalled Issues and PRs that are stalled. label Jul 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This issue/PR was marked as stalled, it will be automatically closed in 30 days. If it should remain open, please leave a comment explaining why it should remain open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc Issues and PRs related to the documentations. stalled Issues and PRs that are stalled. stream Issues and PRs related to the stream subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants