SocketsHttpHandler allocating more per HTTP/1.1 request due to additional async state machines

[stephentoub]

In .NET 5 for an HTTP/1.1 request, SocketsHttpHandler would typically end up allocating two async state machines for a total of 480 bytes:

In .NET 6 for the same request, it now typically ends up allocating five async state machines for a total of 952 bytes:

(These traces are for ~10K requests).

The extra state machines come from the DetermineVersionAndSendAsync, SendUsingHttp11Async, and SendAndProcessAltSvcAsync async methods.

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

In .NET 5 for an HTTP/1.1 request, SocketsHttpHandler would typically end up allocating two async state machines for a total of 480 bytes:

In .NET 6 for the same request, it now typically ends up allocating five async state machines for a total of 952 bytes:

(These traces are for ~10K requests).

The extra state machines come from the DetermineVersionAndSendAsync, SendUsingHttp11Async, and SendAndProcessAltSvcAsync async methods.

Author:	stephentoub
Assignees:	-
Labels:	area-System.Net.Http, tenet-performance, untriaged, regression-from-last-release
Milestone:	6.0.0

[geoffkizer]

Makes sense, as those routines all got refactored a bunch.

I really wish we could combine these state machines without having to reorganize the code.

[geoffkizer]

The other piece of data that would be interesting here is, how much does this contribute to allocation overall?

There's some allocation overhead per request that is unavoidable... stuff like request/response objects, response streams, header collections, etc. I don't have a great sense for how much these state machines impact overall memory usage. Obviously the ideal would be to eliminate as much as possible, but if that becomes painful it would be good to understand the overall impact here.

[MihaZupan]

I posted an allocation breakdown for YARP in OneNote - this was for preview 3 of .NET 6.
The new state machines mean an ~8% increase in overall allocated bytes when proxying a request.

Comparing to just HttpClient using non-validated headers, it is a ~1/3 increase in allocated bytes.

[stephentoub]

(Miha and I were writing concurrently; I'll post this anyway, but it's consistent with his numbers...)

For this benchmark:

Code

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Columns;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Reports;
using BenchmarkDotNet.Running;
using Perfolizer.Horology;
using System;
using System.Globalization;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Sockets;
using System.Text;
using System.Threading.Tasks;

[MemoryDiagnoser]
public class Program
{
    public static void Main(string[] args) =>
        BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args, DefaultConfig.Instance
            .WithSummaryStyle(new SummaryStyle(CultureInfo.InvariantCulture, printUnitsInHeader: false, SizeUnit.B, TimeUnit.Microsecond))
            );

    [Benchmark(Baseline = true, OperationsPerInvoke = 1000)]
    public async Task Request()
    {
        for (int i = 0; i < 1000; i++)
        {
            var request = new HttpRequestMessage(HttpMethod.Get, s_uri);
            using var resp = await s_client.SendAsync(request, default);
            await resp.Content.CopyToAsync(Stream.Null);
        }
    }

    private static readonly Socket s_listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
    private static readonly HttpMessageInvoker s_client = new HttpMessageInvoker(new HttpClientHandler { UseProxy = false, UseCookies = false, AllowAutoRedirect = false });
    private static Uri s_uri;

    [GlobalSetup]
    public void CreateSocketServer()
    {
        s_listener.Bind(new IPEndPoint(IPAddress.Loopback, 0));
        s_listener.Listen(int.MaxValue);
        var ep = (IPEndPoint)s_listener.LocalEndPoint;
        s_uri = new Uri($"http://{ep.Address}:{ep.Port}/");
        byte[] response = Encoding.UTF8.GetBytes("HTTP/1.1 200 OK\r\nDate: Tue, 01 Jul 2021 12:00:00 GMT \r\nServer: Example\r\nContent-Length: 5\r\n\r\nHello");
        byte[] endSequence = new byte[] { (byte)'\r', (byte)'\n', (byte)'\r', (byte)'\n' };

        Task.Run(async () =>
        {
            while (true)
            {
                Socket s = await s_listener.AcceptAsync();
                _ = Task.Run(() =>
                {
                    using (var ns = new NetworkStream(s, true))
                    {
                        byte[] buffer = new byte[1024];
                        int totalRead = 0;
                        while (true)
                        {
                            int read = ns.Read(buffer, totalRead, buffer.Length - totalRead);
                            if (read == 0) return;
                            totalRead += read;
                            if (buffer.AsSpan(0, totalRead).IndexOf(endSequence) == -1)
                            {
                                if (totalRead == buffer.Length) Array.Resize(ref buffer, buffer.Length * 2);
                                continue;
                            }

                            ns.Write(response, 0, response.Length);

                            totalRead = 0;
                        }
                    }
                });
            }
        });
    }
}

I get:

Method	Runtime	Allocated
Request	.NET 5.0	1,416 B
Request	.NET 6.0	1,888 B

which means the extra 472 bytes represents a 33% increase over .NET 5.

[geoffkizer]

BTW this seems like a benchmark we should preserve somewhere and measure on an ongoing basis.