improve cold start times

[kamyker]

You’d need to ask ibm :)

As an Apache project we’d love the contribution, which you could then build and push to dockerhub and use independent of the ibm lifecycle.
Originally posted by @rabbah in #22 (comment)

Didn't know it was possible but 3.1 image from docker works fine with ibm functions.

Instead of:

ibmcloud fn action update {methodName} out.zip --kind dotnet:2.2 --main {namespaceName}::{namespaceName}.{className}::{methodName} --web true

Simply run:

ibmcloud fn action update {methodName} out.zip --docker openwhisk/action-dotnet-v3.1 --main {namespaceName}::{namespaceName}.{className}::{methodName} --web true

Just letting you know :)

[rabbah]

That's right. You can even push your own images with your own dependencies and use them as "docker actions". The main difference is performance. The platform built-in images will load faster whereas the docker actions will have occasional slow starts because the image is pulled.

[kamyker]

Interesting!

Finally managed to make openwhisk work on my windows with the kubernates guide. Also made (my first :) ) docker image of 3.1.

Created simple bat to make iteration little bit faster

docker build -t kamyker/openwhisk-action-dotnet-v3.1:latest .
docker push kamyker/openwhisk-action-dotnet-v3.1:latest
pause

It takes only 20-30 sec but I guess there's a better way.

Is there any way to use local image? Something like this https://github.com/apache/openwhisk-deploy-kube#deploying-a-locally-built-docker-image but for runtime --kind 3.1

[kamyker]

Changing issue name and will write sometime what I've found.

This should speed up cold times:
kamyker@f9a7397#diff-6bc49d26f451cad6d8312deca5a0eadeL89-L96

Instead of writing zip and deleting it, we simply extract it from memory. Probably there's a way to do it directly from stream.

[kamyker]

After some changes my cold start compared to openwhisk/action-dotnet-v3.1 and --kind 2.2 on ibm almost halved!
From ~1.45 sec
To ~750ms

That's right. You can even push your own images with your own dependencies and use them as "docker actions". The main difference is performance. The platform built-in images will load faster whereas the docker actions will have occasional slow starts because the image is pulled.

Opposite :)

What's also interesting is that real response time dropped by more than 2 sec:

[kamyker]

Removed Newtonsoft json and used System.Text.Json that overall has less dependencies. Used few more features to strip down C# build.

Cold start of empty function ~350ms

The one in the post above (it's 6mb zip) ~570ms

That's almost 1 sec lower than --kind dotnet:3.1 and 2.2 :)

Great results, much better than Azure Functions or even AWS Lambda.

[rabbah]

@kamyker are you interested in opening a PR for your improvements to the .NET 3.1 runtime? Sorry we missed this, I think the speeds ups are great.

[kamyker]

I've switched to aws lambda due to unexpected crashes mentioned in #35

I'm not using ow anymore, can't provide more info, sorry :/

[mattwelke]

That's right. You can even push your own images with your own dependencies and use them as "docker actions". The main difference is performance. The platform built-in images will load faster whereas the docker actions will have occasional slow starts because the image is pulled.

Opposite :)

Sorry to resurrect an old issue, but I'm curious about this. Why is it the opposite? It sounds intuitive that using platform images would result in quicker cold starts because the node tasked with running the container would be more likely to have the image already.

[mattwelke]

This should speed up cold times:
kamyker/openwhisk-dotnet-csharp@f9a7397#diff-6bc49d26f451cad6d8312deca5a0eadeL89-L96

Instead of writing zip and deleting it, we simply extract it from memory. Probably there's a way to do it directly from stream.

@kamyker

That's a clever approach to speed it up. Do you think that contributed most to the cold start times?

Btw I'll pick this up if you aren't working on it anymore. I'm using the Node.js runtime on ICF right now and I want to switch to a compiled language runtime for better DX. I like C#, but the cold start times are all 2+ seconds for me on ICF with the 2.2 runtime right now. Plus, 2.2 is EOL. D: This would be a fun way to improve my performance in prod and learn more about OpenWhisk at the same time. Mind if I ask you questions about what you've done so far if I have any?

[shawnallen85]

@mattwelke Can you try the 3.1 runtime and see if you are seeing any improvements there? I've made some changes to the runtime but I don't think I've kicked off an official release of it recently -- I'll try to get on that soon.

[mattwelke]

@shawnallen85 Sure I'll add the 3.1 runtime to my A/B test. I've been using nodejs:12 for a while and I added dotnet:2.2 last night. I have a setup where the URLs are chosen randomly by a client that's making about one request per hour on average. It works great for A/B testing cold starts. For example, so far, I've seen about 500ms for nodejs:12 cold starts but 2s for dotnet:2.2 cold starts.

I just deployed a third action using dotnet:3.1. I'm using IBM Cloud Functions so I had to make this a Docker action since dotnet:3.1 isn't a registered kind there:

This means my very first request to it was really slow. I think this is because of it having to pull the runtime image:

~ > time curl -X POST https://us-south.functions.appdomain.cloud/api/v1/web/REDACTED/default/google-analytics-privacy-proxy-dotnet-3.1
{
  "code": "REDACTED",
  "error": "There was an error processing your request."
}
real	0m22.122s
user	0m0.017s
sys	0m0.000s

(don't mind the error, this is a bad curl on purpose, I just wanted to test the time)

If I let this run for another day, the dotnet:3.1 kind will probably get quite a few requests, and I might be able to see some in the activation logs where the time is less than 22 seconds, inferring that a cold start happened to be assigned to a node that had the image already. I believe that's the info you're looking for.

[mattwelke]

Also, once we get an idea of the improvements to make to 3.1, we can move on to 5.0. I was able to get a 5.0 runtime working pretty easily today by basing it off the 3.1 runtime. :)

https://hub.docker.com/repository/docker/mwelke/openwhisk-action-dotnet-v5.0