Reduce time for serverless cold-start of .Net/C# code

[James2397]

Reduce time for serverless cold-start of .Net/C# code

At the moment .Net Core is one of the slowest loading languages you can use for serverless development in Azure Functions of Aws Lambda. Each time you start a new function written in .Net Core there is significant "cold-start" time which means users experience an extended time waiting compared to using other languages. With serverless becoming a core architectural approach for many companies it would make sense to look into this problem now. If the "cold start" of .net core programs could be reduced to low milliseconds it would significantly increase the appeal of the language for serverless development.

The second part of this problem is the artifact that is generated from a dotnet core publish is large compared to other languages which also attributes to slow "cold-start" times. It would be good if the IL Linker could be brought into the normal publishing process to reduce dll sizes (https://github.com/dotnet/core/blob/master/samples/linker-instructions.md). If namespaces or classes are not being used they should be removed from the published dlls. Or combine dlls into a single binary that only contains used classes/methods/properties. Regardless, just make it super small and fast to work the best out of all languages with serverless.

[Petermarcu]

@russellhadley @valenis

[Petermarcu]

Can you share any number you have for cold start times? There are things that can be done to make cold start better and some of it depends on how AWS has built and configured the runtime to work.

[James2397]

Here is a comparison by a tech blogger and it summarizes what we have been experiencing: http://theburningmonk.com/2017/06/aws-lambda-compare-coldstart-time-with-different-languages-memory-and-code-sizes/

[James2397]

The below blog article also shows how .Net Core serverless code is significantly slower than all other languages, and the main language it is most similar too - Java - is actually the quickest for cold starts:

https://read.acloud.guru/comparing-aws-lambda-performance-when-using-node-js-java-c-or-python-281bef2c740f
Summary: .Net Core was over 5 times slower than Java in those tests for cold start serverless invocation.

Hopefully, this can be looked at soon because it is bad for our customers if we use .Net Core and this happens regularly. Java being the fastest shows that it should be possible for a language like C# to achieve similar positive results.

[Petermarcu]

I noticed that article is using .NET Core 1.0. I know there have been significant investments in both startup and and throughput since. Particularly on Linux. I think it would be useful to do the exercise again with the latest bits to see where things stand. Overall I think this is a good scenario to try to tackle and improve.

On the comment for the size of a published application. How large is the published size of your serverless application?

[James2397]

A lot of customers are using .Net Core 1.0 (.Net Stardard 1.6) because that is what AWS supports for most features (like Lambda serverless) where cold-start time is critical. No serverless provider currently has support for .Net 2.0 yet. In Azure you can manually hack the configs to get .Net Core 2.0 working but I don't think it would be operating in an optimized way when it's not fully supported in the UI yet. Nobody can do the .Net 2.0 numbers yet because AWS and Azure does not fully support .Net Core 2.0 yet. What I do know from working on a few non-serverless .Net Core 2.0 projects recently though, is the startup time is better than .Net 1.0 but not significantly better - the cold start time is still an issue.

[James2397]

In regards to the size, in Node we can have some scripts that are a few KB to the do the same job in .Net is 10-20mb because of the tree of dependencies that are pulled in Microsoft .Net Core framework and externally. For example, we may only use 1 tiny enum from a dependency dll and it bings in the whole 5mb dll instead of only the code for the 1 enum that is < 1KB. Our function sizes in .Net Core are on average 100's of times the size of the equivalent in Node because of the 'publish' command bringing in whole dlls, and things we don't need or will ever use - and the code never references. Traditional systems where space was not a concern this is fine, however for serverless where cold-start is most important and every large chunk of unrequired code slows down the loading of the function, it would be good to get dll's cut-down and optimized to be smaller and faster, or removed all together if the code never uses classes/methods/properties from that dll (I am particularly talking about dependencies of dependencies).

[Petermarcu]

On the size thing, we have our linker on the way which will allow the code in your "app" to be trimmed down to only what you need. https://github.com/dotnet/core/blob/master/samples/linker-instructions-advanced.md

We're going to take a deeper look at the serverless startup problem. We know how fast runtime startup is locally. We'll need to determine what other aspects of the serverless scenario are contributing to the coldstart problem.

We definitely want this to be great. Any additional insight is appreciated.

[Petermarcu]

We are having someone reproduce the results so we can dig into more of the details.

[kannank]

Is there any update in the issue. Has there been any noticeable improvements in 2.0

[dotnetchris]

FWIW AWS supports Core 2.0 now in lambdas, added support early 2018.

[jansabbe]

I ran the benchmarks from theburningmonk again using .net core 2.0 (and adding golang as well). See my results here: https://plot.ly/%7Ejansabbe/6/csharp-java-python-nodejs6-golang/ .Net core 2.0 is faster than java, but still way slower than nodejs/python or Go.

Interestingly, filesize doesn't matter that much. The .net core 2.0 deployment package was around 200K, while the golang deployment package was close to 3MB. Go was still an order of a magnitude faster for simply responding "hello". Not sure if this is a .net issue, seems more like something is weird at AWS.

[Petermarcu]

Just saw this article from yesterday showing .NET Core 2.0 as the top dog https://read.acloud.guru/comparing-aws-lambda-performance-of-node-js-python-java-c-and-go-29c1163c2581 .

[asabla]

Sadly @Petermarcu they're excluding cold startup time

Similar to the original performance tests, we’ll ignore the initial cold start time — and focus only on the duration metric to compare runtime performance between the different languages