Skip to content

Re-worked Node.js readReadable API - #2663

Merged
mpilquist merged 4 commits into
typelevel:mainfrom
armanbilge:bug/read-readable
Oct 6, 2021
Merged

Re-worked Node.js readReadable API#2663
mpilquist merged 4 commits into
typelevel:mainfrom
armanbilge:bug/read-readable

Conversation

@armanbilge

@armanbilge armanbilge commented Oct 4, 2021

Copy link
Copy Markdown
Member

So far this fixes the issue reported in #2661, but I think the API for converting a Node.js Readable stream into a Stream of bytes is still unsafe. I have another idea I want to try.

@armanbilge armanbilge changed the title Re-working readReadable API Re-working Node.js readReadable API Oct 4, 2021
Comment on lines +67 to +72
// Implementation Note: why suspend in `SyncIO` and then `unsafeRunSync()` inside `F.delay`?
// In many cases creating a `Readable` starts async side-effects (e.g. negotiating TLS handshake or opening a file handle).
// Furthermore, these side-effects will invoke the listeners we register to the `Readable`.
// Therefore, it is critical that the listeners are registered to the `Readable` _before_ these async side-effects occur:
// in other words, before we next yield (cede) to the event loop. Because an arbitrary effect `F` (particularly `IO`) may cede at any time,
// our only recourse is to suspend the entire creation/listener registration process within a single atomic `delay`.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

☝️

@armanbilge
armanbilge marked this pull request as ready for review October 4, 2021 03:35
@armanbilge

Copy link
Copy Markdown
Member Author

I'd really appreciate some critical eyes on this. These methods for translating between Node.js and fs2 Streams are at the heart of all the implementations (Sockets, TLS, files). So it's really important to get this right.

@armanbilge armanbilge changed the title Re-working Node.js readReadable API Re-worked Node.js readReadable API Oct 4, 2021
@armanbilge

Copy link
Copy Markdown
Member Author

I publishedLocal and tested against ember-{client/server}. No regressions. I'd like to test skunk too, but it's easier to use the CI there so I'll wait for a hash snapshot when this merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants