Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stream: add writable.writeAhead() #49055

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

KhooHaoYit
Copy link
Contributor

This PR add a new function called writeAhead to writable

The main reason I feel like it should belong to nodejs is because its hard to do first in last out using stream which my code needs to do

Example use case:

queue = new stream.Writable({
  objectMode: true,
  writev(chunks, cb){
    // Assuming that it could only process 2 chunk at a time
    console.log([chunks.shift(), chunks.shift()]);
    // Any write before returning would be cleared soo .nextTick is needed
    process.nextTick(() => {
      this.cork();
      for(let index = chunks.length; index--; ){
        const { chunk, encoding, callback } = chunks[index];
        // Callback needs to be cleared due to that `cb()` would call them automatically
        chunks[index].callback = () => {};
        this.writeAhead(chunk, encoding, callback); // Quite hard to achieve
      }
      process.nextTick(() => this.uncork());
    });
    cb();
  },
});
queue.cork();
queue.write({ a: 1 });
queue.write({ b: 2 });
queue.write({ c: 3 });
queue.write({ d: 4 });
queue.write({ e: 5 });
queue.uncork(); // Assuming the stream never .end()

Any suggestion/improvement is welcomed

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/streams

@nodejs-github-bot nodejs-github-bot added the needs-ci PRs that need a full CI run. label Aug 7, 2023
Copy link
Member

@ronag ronag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe this belongs in streams. What's wrong with unshift?

@KhooHaoYit
Copy link
Contributor Author

Assuming you are talking about readable.unshift, no, there's nothing wrong with unshift

But for writable, you can only add data behind (.write), while readable can add data behind and in front (.push and .unshift)

This PR gives the ability to do something similar to .unshift for readable in writable

@benjamingr
Copy link
Member

unshift (which I don't like tbh) is used for "read then regret if the chunk doesn't match" cases, typically streams go in one direction - I'm also -1 about having this in core although a userland subclass is free to add this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-ci PRs that need a full CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants