Linux sockets perf: Reduce syscalls by being smart about handling partial reads/writes

[geoffkizer]

Currently, for every send/receive operation, we always issue a syscall to see if we can perform the operation synchronously. If this fails (i.e. E_AGAIN), we enqueue the operation and wait for an epoll notification before trying again.

We can avoid this initial sync attempt for streaming protocols (i.e. TCP) in many cases by remembering whether the previous send/receive returned a partial response. That is, for receive, it didn't fill the entire buffer; for send, it didn't consume the entire buffer. In these cases we can know that the socket is now not ready and we need to wait for another epoll readiness notification. So on the next send/receive, we can detect that the socket is still not ready and skip the attempt to perform the operation synchronously.

Here is the relevant passage from Linux epoll man:

       9.  Do I need to continuously read/write a file descriptor until
       EAGAIN when using the EPOLLET flag (edge-triggered behavior)?

       Receiving an event from epoll_wait(2) should suggest to you that
       such file descriptor is ready for the requested I/O operation.
       You must consider it ready until the next (nonblocking)
       read/write yields EAGAIN.  When and how you will use the file
       descriptor is entirely up to you.

       For packet/token-oriented files (e.g., datagram socket, terminal
       in canonical mode), the only way to detect the end of the
       read/write I/O space is to continue to read/write until EAGAIN.

       For stream-oriented files (e.g., pipe, FIFO, stream socket), the
       condition that the read/write I/O space is exhausted can also be
       detected by checking the amount of data read from / written to
       the target file descriptor.  For example, if you call read(2) by
       asking to read a certain amount of data and read(2) returns a
       lower number of bytes, you can be sure of having exhausted the
       read I/O space for the file descriptor.  The same is true when
       writing using write(2).  (Avoid this latter technique if you can‐
       not guarantee that the monitored file descriptor always refers to
       a stream-oriented file.)

@adamsitnik prototyped this in #38747 and it showed a nice win -- 5% on TechEmpower JSON. So this definitely seems worth doing.

In #38747 we ran into some issues with handling EOF properly -- i.e. EPOLLHUP and/or EPOLLRDHUP. I think these are solvable -- see discussion on that issue. We didn't have time to chase this down for 5.0.

@adamsitnik Any other thoughts/comments here?

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[tmds]

Though the man page says otherwise, I think receive may return with less bytes than the buffer size when there is a signal pending.

[geoffkizer]

Though the man page says otherwise, I think receive may return with less bytes than the buffer size when there is a signal pending.

Is there any way we can confirm or deny that?

I've seen references in other projects about leveraging this optimization. (Of course, I can't find them at the moment...)

[wfurt]

I thin receive can receive less than asked for when more data is awailable - but I would think that is rare case. If this is optimization, that can still work in our favor IMHO.

[dotnet-policy-service]

Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process.

This process is part of our issue cleanup automation.