preserve_discovered_labels does not preserve __meta_kubernetes_* labels for downstream components

[suever]

Description

The preserve_discovered_labels option in loki.source.podlogs does not work as documented. When enabled, the __meta_kubernetes_* labels are still stripped before being passed to downstream components like loki.relabel.

Version

• Alloy version: v1.12.0

Steps to Reproduce

1. Create a PodLogs CRD:

apiVersion: monitoring.grafana.com/v1alpha2
kind: PodLogs
metadata:
  name: test-pods
  namespace: test
spec:
  selector:
    matchLabels: {}
  namespaceSelector:
    matchLabels:
      kubernetes.io/metadata.name: test

2. Configure Alloy with preserve_discovered_labels = true and a downstream loki.relabel to extract metadata:

loki.source.podlogs "pod_logs" {
  forward_to = [loki.relabel.pod_logs.receiver]
  preserve_discovered_labels = true
}

loki.relabel "pod_logs" {
  forward_to = [loki.process.pod_logs.receiver]

  rule {
    source_labels = ["__meta_kubernetes_namespace"]
    target_label  = "namespace"
  }

  rule {
    source_labels = ["__meta_kubernetes_pod_name"]
    target_label  = "pod"
  }
}

loki.process "pod_logs" {
  forward_to = [loki.write.loki.receiver]
}

3. Enable live debugging and check the input labels to loki.relabel.pod_logs:

curl http://localhost:12345/api/v0/component/loki.relabel.pod_logs/debug

Expected Behavior

The __meta_kubernetes_* labels should be available as input to loki.relabel:

Input labels:
  __meta_kubernetes_namespace = "test"
  __meta_kubernetes_pod_name = "my-pod-abc123"
  __meta_kubernetes_pod_container_name = "app"
  ...

Actual Behavior

Only basic labels are passed through, the __meta_kubernetes_* labels are stripped:

Input labels:
  instance = ""
  job = "test/test-pods"

Documentation Example Issue

The example in the documentation also has an issue - it shows:

loki.source.podlogs "with_label_preservation" {
  forward_to                  = [loki.write.local.receiver]  # <-- forwards to loki.write, not loki.relabel
  preserve_discovered_labels  = true
}

loki.relabel "pod_relabeling" {
  forward_to = [loki.write.local.receiver]
  # ...rules...
}

The loki.source.podlogs forwards directly to loki.write, not to loki.relabel, so the relabel rules would never be applied. This should probably be:

loki.source.podlogs "with_label_preservation" {
  forward_to                  = [loki.relabel.pod_relabeling.receiver]
  preserve_discovered_labels  = true
}

• PR that added the feature: feat(loki.source.podlogs): add preserve_discovered_labels option #4606
• Previous issue discussing label preservation: loki.source.podlogs: Add option to keep discovered labels #2526

[kalleep]

Yes you are right. It seems like there is another place that is stripping out meta labels that we need to update.