Profiler causes the application to not exit [randomly]

[iamKunal]

Running on JDK 14:

When running an application with the profiler enabled, once the application shuts down, I get a log with On VM death after the last log of the application and the application does not exit. Tried exiting explicitly with System.exit() as well.

This has happened on 2 separate occasions on 2 separate applications.

Args for running: -agentpath:${GCP_PROFILER_JAVA_AGENT}=-cprof_service=${APP_NAME},-cprof_project_id=${PROJECT_ID},-logtostderr,-cprof_enable_heap_sampling=true

I am running it inside GKE with WorkloadIdentity enabled if that is of any concern (it automatically picks up the zone name, so metadata server might not be a problem).

Also:
I do not get the subsequent logs for Exiting the profiling loop and Heap sampling GC waiting thread finished which I get when the application exits normally.

[nolanmar511]

I've not seen any of our integration tests fail because the application isn't able to exit naturally. We do have at least one integration test which would fail if the application cannot exit. So, it seems like this must not happen always (or at least not under all conditions), so any additional details to help narrow down how to reproduce this would be appreciated.

As a few starting questions:

• Do you know what fraction of the time this happens?
• Could you identify the version of the agent that's used? (If the debug log level is at info, it will be logged with the message "Google Cloud Profiler Java agent version:"; this documentation describes how to get the agent version). Do you use a version of the agent downloaded from https://storage.googleapis.com/cloud-profiler/java/latest, included as part of a default image, or one built from this repo?

Any other details about the runtime environment (for example, if an Alpine image is used; the precise java version used) would also be helpful.

[iamKunal]

Do you know what fraction of the time this happens?

I have enabled a bigquery sink to count On VM death vs Exiting the profiling loop to get these numbers. Will get them to you in a day or two

Could you identify the version of the agent that's used? (If the debug log level is at info, it will be logged with the message "Google Cloud Profiler Java agent version:"; this documentation describes how to get the agent version).

Google Cloud Profiler Java agent version: cloud-profiler-java-agent_20210208_RC00"

Do you use a version of the agent downloaded from https://storage.googleapis.com/cloud-profiler/java/latest, included as part of a default image, or one built from this repo?

I use the latest one from https://storage.googleapis.com/cloud-profiler/java/latest

Any other details about the runtime environment (for example, if an Alpine image is used; the precise java version used) would also be helpful.

openjdk version "14.0.2" 2020-07-14
OpenJDK Runtime Environment (build 14.0.2+12-46)
OpenJDK 64-Bit Server VM (build 14.0.2+12-46, mixed mode, sharing)

We use a openjdk:14-slim image

[iamKunal]

Also, some log data of about 1.5 hours:

[leelamv]

My guess is this issue is one possible manifestation of the issue mentioned here

[iamKunal]

Any updates @nolanmar511 ?

[aalexand]

Hopefully fixed by #37.