Skip to content

fix targets conversion to maps for some components#2834

Merged
thampiotr merged 4 commits intomainfrom
thampiotr/fix-targets-reading-from-files
Feb 26, 2025
Merged

fix targets conversion to maps for some components#2834
thampiotr merged 4 commits intomainfrom
thampiotr/fix-targets-reading-from-files

Conversation

@thampiotr
Copy link
Contributor

@thampiotr thampiotr commented Feb 25, 2025

PR Description

Fixes an issue where a component defines a map[string]string argument and is passed a []discovery.Target. While conversion between map and target works in context of Alloy config literals, it requires special handling for the above case. See comments for more details.

Which issue(s) this PR fixes

Fixes #2831

Notes to the Reviewer

Tested locally to repro issue raised in the ticket above:

thampiotr@MacWork ~/w/g/a/repro-targets-conversion (main)> # before fix
thampiotr@MacWork ~/w/g/a/repro-targets-conversion (main)> /opt/homebrew/opt/alloy/bin/alloy --version
alloy, version v1.7.0 (branch: HEAD, revision: unknown)
  build user:       grafana
  build date:       2025-02-24T12:41:29Z
  go version:       go1.23.6
  platform:         darwin/arm64
  tags:             builtinassets,noebpf
thampiotr@MacWork ~/w/g/a/repro-targets-conversion (main)> /opt/homebrew/opt/alloy/bin/alloy run ./config.alloy
ts=2025-02-26T11:18:03.451381Z level=info "boringcrypto enabled"=false
ts=2025-02-26T11:18:03.451424Z level=info msg="no peer discovery configured: both join and discover peers are empty" service=cluster
ts=2025-02-26T11:18:03.451428Z level=info msg="running usage stats reporter"
ts=2025-02-26T11:18:03.451431Z level=info msg="starting complete graph evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9
ts=2025-02-26T11:18:03.451438Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 node_id=discovery.file.snmp duration=1.251834ms
ts=2025-02-26T11:18:03.451443Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 node_id=prometheus.exporter.snmp.snmp duration=50.052875ms
ts=2025-02-26T11:18:03.451447Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 node_id=prometheus.scrape.snmp duration=3.634916ms
ts=2025-02-26T11:18:03.451464Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 node_id=livedebugging duration=5.458µs
ts=2025-02-26T11:18:03.451467Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 node_id=tracing duration=4.875µs
ts=2025-02-26T11:18:03.45147Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 node_id=logging duration=101.917µs
ts=2025-02-26T11:18:03.451491Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 node_id=remotecfg duration=17.667µs
ts=2025-02-26T11:18:03.451499Z level=info msg="applying non-TLS config to HTTP server" service=http
ts=2025-02-26T11:18:03.451502Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 node_id=http duration=5.792µs
ts=2025-02-26T11:18:03.451544Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 node_id=ui duration=1.791µs
ts=2025-02-26T11:18:03.451554Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 node_id=cluster duration=3.917µs
ts=2025-02-26T11:18:03.45156Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 node_id=otel duration=1.666µs
ts=2025-02-26T11:18:03.451566Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 node_id=labelstore duration=3.083µs
ts=2025-02-26T11:18:03.45157Z level=info msg="finished complete graph evaluation" controller_path=/ controller_id="" trace_id=8fd8cf81cfbfc42b670a25e0b3930eb9 duration=55.4575ms
ts=2025-02-26T11:18:03.451589Z level=info msg="scheduling loaded components and services"
ts=2025-02-26T11:18:03.452256Z level=info msg="starting cluster node" service=cluster peers_count=0 peers="" advertise_addr=127.0.0.1:12345
ts=2025-02-26T11:18:03.452547Z level=info msg="peers changed" service=cluster peers_count=1 peers=MacWork.local
ts=2025-02-26T11:18:03.453182Z level=info msg="now listening for http traffic" service=http addr=127.0.0.1:12345
ts=2025-02-26T11:18:08.455587Z level=error msg="failed to evaluate config" controller_path=/ controller_id="" node=prometheus.exporter.snmp.snmp err="decoding configuration: ./config.alloy:6:16: discovery.file.snmp.targets[0] target::ConvertInto: conversion to '*map[string]string' is not supported"
^Cinterrupt received
ts=2025-02-26T11:18:14.490611Z level=error msg="failed to start reporter" err="context canceled"
ts=2025-02-26T11:18:14.49077Z level=info msg="node exited without error" node=labelstore
ts=2025-02-26T11:18:14.490878Z level=info msg="node exited without error" node=prometheus.exporter.snmp.snmp
ts=2025-02-26T11:18:14.49096Z level=info msg="node exited without error" node=remotecfg
ts=2025-02-26T11:18:14.491169Z level=info msg="node exited without error" node=ui
ts=2025-02-26T11:18:14.491328Z level=info msg="http server closed" service=http addr=127.0.0.1:12345 err="http: Server closed"
ts=2025-02-26T11:18:14.491388Z level=info msg="http server closed" service=http addr=memory err="http: Server closed"
ts=2025-02-26T11:18:14.491432Z level=info msg="node exited without error" node=http
ts=2025-02-26T11:18:14.491436Z level=info msg="node exited without error" node=livedebugging
ts=2025-02-26T11:18:14.491452Z level=info msg="node exited without error" node=otel
ts=2025-02-26T11:18:14.491518Z level=info msg="node exited without error" node=cluster
ts=2025-02-26T11:18:14.491813Z level=info msg="node exited without error" node=discovery.file.snmp
ts=2025-02-26T11:18:14.492729Z level=info msg="node exited without error" node=prometheus.scrape.snmp

After my fix, the isssue no longer happens.

thampiotr@MacWork ~/w/g/a/repro-targets-conversion (main)> ~/workspace/alloy/build/alloy --version
alloy, version v1.8.0-devel+f15234689 (branch: thampiotr/fix-targets-reading-from-files, revision: f15234689)
  build user:       thampiotr@MacWork.local
  build date:       2025-02-26T11:10:41Z
  go version:       go1.24.0
  platform:         darwin/arm64
  tags:             unknown
thampiotr@MacWork ~/w/g/a/repro-targets-conversion (main)> ~/workspace/alloy/build/alloy run ./config.alloy 
ts=2025-02-26T11:18:46.749457Z level=info "boringcrypto enabled"=false
ts=2025-02-26T11:18:46.749492Z level=info msg="no peer discovery configured: both join and discover peers are empty" service=cluster
ts=2025-02-26T11:18:46.749495Z level=info msg="running usage stats reporter"
ts=2025-02-26T11:18:46.749497Z level=info msg="starting complete graph evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65
ts=2025-02-26T11:18:46.749502Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 node_id=logging duration=46.375µs
ts=2025-02-26T11:18:46.749582Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 node_id=remotecfg duration=75.292µs
ts=2025-02-26T11:18:46.749594Z level=info msg="applying non-TLS config to HTTP server" service=http
ts=2025-02-26T11:18:46.749597Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 node_id=http duration=6.084µs
ts=2025-02-26T11:18:46.749602Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 node_id=cluster duration=708ns
ts=2025-02-26T11:18:46.749605Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 node_id=otel duration=208ns
ts=2025-02-26T11:18:46.74961Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 node_id=livedebugging duration=1.75µs
ts=2025-02-26T11:18:46.749613Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 node_id=ui duration=250ns
ts=2025-02-26T11:18:46.749647Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 node_id=labelstore duration=1.625µs
ts=2025-02-26T11:18:46.749717Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 node_id=discovery.file.snmp duration=64.208µs
ts=2025-02-26T11:18:46.795724Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 node_id=prometheus.exporter.snmp.snmp duration=45.999625ms
ts=2025-02-26T11:18:46.802855Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 node_id=prometheus.scrape.snmp duration=7.095458ms
ts=2025-02-26T11:18:46.802884Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 node_id=tracing duration=6.583µs
ts=2025-02-26T11:18:46.802891Z level=info msg="finished complete graph evaluation" controller_path=/ controller_id="" trace_id=e33d3c59542feab74ad20175bfaddc65 duration=53.509875ms
ts=2025-02-26T11:18:46.802916Z level=info msg="scheduling loaded components and services"
ts=2025-02-26T11:18:46.803312Z level=info msg="starting cluster node" service=cluster peers_count=0 peers="" advertise_addr=127.0.0.1:12345
ts=2025-02-26T11:18:46.803585Z level=info msg="peers changed" service=cluster peers_count=1 peers=MacWork.local
ts=2025-02-26T11:18:46.803676Z level=info msg="now listening for http traffic" service=http addr=127.0.0.1:12345
^Cinterrupt received
ts=2025-02-26T11:19:17.571343Z level=error msg="failed to start reporter" err="context canceled"
ts=2025-02-26T11:19:17.57147Z level=info msg="node exited without error" node=remotecfg
ts=2025-02-26T11:19:17.5715Z level=info msg="node exited without error" node=prometheus.exporter.snmp.snmp
ts=2025-02-26T11:19:17.571522Z level=info msg="node exited without error" node=livedebugging
ts=2025-02-26T11:19:17.571592Z level=info msg="node exited without error" node=labelstore
ts=2025-02-26T11:19:17.571575Z level=info msg="node exited without error" node=ui
ts=2025-02-26T11:19:17.57172Z level=info msg="node exited without error" node=discovery.file.snmp
ts=2025-02-26T11:19:17.571561Z level=info msg="node exited without error" node=otel
ts=2025-02-26T11:19:17.571758Z level=info msg="http server closed" service=http addr=127.0.0.1:12345 err="http: Server closed"
ts=2025-02-26T11:19:17.571734Z level=warn msg="failed to wait for rate limiter on peers update" service=cluster err="context canceled"
ts=2025-02-26T11:19:17.571706Z level=info msg="http server closed" service=http addr=memory err="http: Server closed"
ts=2025-02-26T11:19:17.571898Z level=info msg="node exited without error" node=cluster
ts=2025-02-26T11:19:17.571906Z level=info msg="node exited without error" node=http
ts=2025-02-26T11:19:17.571998Z level=info msg="node exited without error" node=prometheus.scrape.snmp

PR Checklist

  • CHANGELOG.md updated
  • Documentation added
  • Tests updated
  • Config converters updated

@github-actions
Copy link
Contributor

github-actions bot commented Feb 25, 2025

@thampiotr thampiotr force-pushed the thampiotr/fix-targets-reading-from-files branch from ff47ada to 6f464ef Compare February 26, 2025 10:58
select {
case <-ctx.Done():
}
<-ctx.Done()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes to make linter happier.

Comment on lines +117 to +118
// If we were adding one, may need to clean it up too.
delete(t.toAdd, label)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also changes to make linter happier.

})
*dst = result
return nil
case *map[string]string:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the code path required to fix the issue with

ts=2025-02-25T12:04:02.660000145Z level=error msg="failed to evaluate config" controller_path=/ controller_id="" node=prometheus.exporter.snmp.snmp err="decoding configuration: /etc/alloy/config.alloy:204:16: discovery.file.snmp.targets[0] target::ConvertInto: conversion to '*map[string]string' is not supported"

see test for more details

}
*t = NewTargetFromLabelSet(labelSet)
return nil
case map[string]string:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the other way around just in case we ever need it.

Comment on lines +164 to +183
t.Run("decode from target into map via scope", func(t *testing.T) {
// If not supported, this would lead to error: target::ConvertInto: conversion to '*map[string]string' is not supported
scope := vm.NewScope(map[string]interface{}{"export": NewTargetFromMap(tc.input)})
expr, err := parser.ParseExpression("export")
require.NoError(t, err)
eval := vm.New(expr)
actualMap := map[string]string{}
require.NoError(t, eval.Evaluate(scope, &actualMap))
require.Equal(t, tc.input, actualMap)
})

t.Run("decode from map into target via scope", func(t *testing.T) {
scope := vm.NewScope(map[string]interface{}{"map": tc.input})
expr, err := parser.ParseExpression("map")
require.NoError(t, err)
eval := vm.New(expr)
actual := Target{}
require.NoError(t, eval.Evaluate(scope, &actual))
require.Equal(t, NewTargetFromMap(tc.input), actual)
})
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These were the missing tests. The issue is not obvious because we technically already have a way to convert from map to Target, because we can do map -> alloy syntax -> Target, but that is not how Alloy works, it does not figure out intermediate formats and that's probably good for performance reasons. So an explicit test of map -> Target was needed, as well as Target -> map is added for any such cases in the future.

The reason this was an issue in SNMP exporter is that in its arguments instead of Target it uses a de-facto map[string]string here: https://github.com/grafana/alloy/blob/main/internal/component/prometheus/exporter/snmp/snmp.go#L157-L160

There are other possible such cases, and they could be added in the future, so a solution of allowing this automatic conversion is preferred.

@thampiotr thampiotr requested a review from wildum February 26, 2025 11:22
@thampiotr thampiotr marked this pull request as ready for review February 26, 2025 11:22
@thampiotr thampiotr requested a review from a team as a code owner February 26, 2025 11:22
@thampiotr thampiotr changed the title fix targets reading from files fix targets conversion to maps for some components Feb 26, 2025
@thampiotr thampiotr merged commit 1278fcb into main Feb 26, 2025
30 checks passed
@thampiotr thampiotr deleted the thampiotr/fix-targets-reading-from-files branch February 26, 2025 11:43
thampiotr added a commit that referenced this pull request Feb 26, 2025
* fix targets conversion to maps for some components (#2834)

* update to latest walqueue (#2845)

* update to latest walqueue

* fix typo

* fix typo

* Update ckit dependency (#2847)

* changelog fix

---------

Co-authored-by: mattdurham <mattdurham@ppog.org>
Co-authored-by: Sam DeHaan <sam.dehaan@grafana.com>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 29, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Alloy v1.7.0 - prometheus.exporter.snmp - target::ConvertInto: conversion to '*map[string]string' is not supported

2 participants