Avoid pyroscope.ebpf to run without targets

[luweglarz]

Background

I am deploying Alloy as a DaemonSet with the pyroscope.ebpf component with discovery.kubernetes targets matching a specific label, like

discovery.kubernetes "k8s_ebpf_pods" {
      role = "pod"
      selectors {
        role  = "pod"
        label = "pyroscope.io/ebpf=true"
      }
}

When this label is not set on any pods of the node where Alloy is deployed, the targets are empty but we still have the pyroscope.ebpf component running and using resources for nothing.

Proposal

A way to enable pyroscope.ebpf component only if it has available targets, I would be happy to look into it if you think this would be an interesting features to be handled on Alloy side

[luweglarz]

Hi @korniltsev , I would like to tackle this anytime soon but before I would like to ask if this fits with the pyroscope.ebpf component scope and intent?

My idea would be to implement a dynamic startup/shutdown of the otel controller based on the len of c.args.Targets, what do you think? Thank you !

[korniltsev-grafanista]

I am fine with this. I saw some comments in the otel repo that the profiler was not designed with restarting in mind, but if it works fine, let's do it. Maybe worth adding some throttling / delay for a case when there is a single target that is redeployed so that the profiler is not loaded/ unloaded frequently.

Hopefully there are no memory/FD leaks

Maybe make this new lazy mode optional/configurable.