feat(prometheus.exporter.databricks): Add Databricks exporter component by aalhour · Pull Request #5054 · grafana/alloy

aalhour · 2025-12-10T11:14:04Z

PR Description

Add prometheus.exporter.databricks component for collecting metrics from Databricks System Tables including billing, jobs, pipelines, and SQL warehouse performance data.

Features:

OAuth2 M2M authentication via Service Principal
Configurable lookback windows for each metric domain
Optional high-cardinality task retry metrics
19 metrics covering 3 personas: Finance, SRE/Platform, Analytics/BI

Arguments:

server_hostname, warehouse_http_path (required)
client_id, client_secret (required, secret)
query_timeout (default: 5m)
billing_lookback (default: 24h)
jobs_lookback, pipelines_lookback (default: 2h)
queries_lookback (default: 1h)
sla_threshold_seconds (default: 3600)
collect_task_retries (default: false)

Depends on: grafana/databricks-prometheus-exporter#14

Which issue(s) this PR fixes

Fixes: https://github.com/grafana/cloud-onboarding/issues/10029
Part of project: https://github.com/grafana/cloud-onboarding/issues/9716

Notes to the Reviewer

PR Checklist

CHANGELOG.md updated
Documentation added
Tests updated
Config converters updated

CLAassistant · 2025-12-10T11:14:13Z

All committers have signed the CLA.

github-actions · 2025-12-10T11:15:43Z

💻 Deploy preview available (WIP - feat(prometheus.exporter.databricks): add Databricks exporter component):

https://deploy-preview-alloy-5054-zb444pucvq-vp.a.run.app/docs/alloy/latest/

Add prometheus.exporter.databricks component for collecting metrics from Databricks System Tables including billing, jobs, pipelines, and SQL warehouse performance data. Features: - OAuth2 M2M authentication via Service Principal - Configurable lookback windows for each metric domain - Optional high-cardinality task retry metrics - 19 metrics covering 3 personas: Finance, SRE/Platform, Analytics/BI Arguments: - server_hostname, warehouse_http_path (required) - client_id, client_secret (required, secret) - query_timeout (default: 5m) - billing_lookback (default: 24h) - jobs_lookback, pipelines_lookback (default: 2h) - queries_lookback (default: 1h) - sla_threshold_seconds (default: 3600) - collect_task_retries (default: false) Depends on: github.com/grafana/databricks-prometheus-exporter

Update default lookback windows to match the upstream databricks-exporter: - JobsLookback: 2h → 3h - PipelinesLookback: 2h → 3h - QueriesLookback: 1h → 2h These changes ensure consistency with the exporter's lookback windows which were increased to prevent data loss with scrape intervals up to 30 minutes.

Add github.com/grafana/databricks-prometheus-exporter v0.0.0-20251219150331-5730cb38c831 to go.mod, matching the Snowflake exporter pattern.

github-actions · 2026-01-05T15:32:28Z

🔍 Dependency Review

Below are the dependency upgrades found in the go.mod files of this PR, with an assessment of potential code changes needed. Each section includes a brief summary of relevant upstream changes between the “as-is” and “to-be” versions and any action items for maintainers.

Scope: Only dependencies changed in go.mod files are assessed (per rules). Net-new dependencies are noted at the end.

golang.org/x/sys v0.38.0 -> v0.39.0 — ✅ Safe

Summary
- Minor release with additional syscall constants and platform updates. No API removals reported. Backward compatible with existing usages.
Code changes required
- None expected.
Notes/Evidence
- x/sys maintains backward compatibility; this bump typically includes new symbols and platform fixes without breaking existing imports.

github.com/klauspost/compress v1.18.1 -> v1.18.2 — ✅ Safe

Summary
- Patch-level update with bug fixes and potential minor performance improvements across gzip/zstd/flate paths. No breaking API changes reported.
Code changes required
- None expected.
Notes/Evidence
- Maintainer changelogs for 1.18.x lines indicate fixes rather than API changes. Import paths and primary types are stable.

golang.org/x/crypto v0.45.0 -> v0.46.0 — ✅ Safe

Summary
- Regular security/bugfix train; updates across subpackages (e.g., ssh, nacl, scrypt). No known breaking API removals between these versions.
Code changes required
- None expected.
Notes/Evidence
- x/crypto follows compatible evolution; version notes between v0.45.0 and v0.46.0 highlight fixes rather than signature changes.

golang.org/x/net v0.47.0 -> v0.48.0 — ✅ Safe

Summary
- Minor update including http2/headers/proxy and networking internals. No breaking API changes documented between these tags.
Code changes required
- None expected.
Notes/Evidence
- x/net HTTP/2 and QUIC iteration typically remains source-compatible across these minor bumps.

golang.org/x/oauth2 v0.32.0 -> v0.34.0 — ✅ Safe

Summary
- Bug fixes and small feature additions; no breaking API changes noted in releases 0.33.0–0.34.0 (token sources, context handling remain compatible).
Code changes required
- None expected.
Notes/Evidence
- Changelog across these minors shows non-breaking improvements (e.g., transport/context hygiene).

golang.org/x/sync v0.18.0 -> v0.19.0 — ✅ Safe

Summary
- Minor release; x/sync packages (errgroup, singleflight, semaphore) unchanged in breaking ways. Behavioral parity expected.
Code changes required
- None expected.
Notes/Evidence
- No public API removals/deprecations in this range.

golang.org/x/text v0.31.0 -> v0.32.0 — ✅ Safe

Summary
- Minor release with Unicode tables refresh and locale data updates; no breaking API changes reported.
Code changes required
- None expected.
Notes/Evidence
- Typical evolution of x/text includes table updates and bug fixes without signature changes.

golang.org/x/tools v0.38.0 -> v0.40.0 — ⚠️ Needs Review

Summary
- x/tools frequently updates analysis, packages, and related tooling APIs. While minor versions are generally source-compatible, these releases sometimes adjust internal behaviors or add stricter validations in subpackages (e.g., go/packages, go/analysis, guru, etc.).
Code changes recommended to review
- If you import specific x/tools packages (for example):
  - golang.org/x/tools/go/packages
  - golang.org/x/tools/go/analysis/...
  - golang.org/x/tools/imports
- Re-run the build/tests locally. In rare cases, call sites need minor adjustments due to stricter options or updated diagnostics ordering.
Notes/Evidence
- x/tools changelogs between these versions are sizable; nothing broadly breaking is advertised, but toolchain-adjacent code sometimes needs small adjustments.

github.com/go-jose/go-jose/v4 v4.1.2 -> v4.1.3 — ✅ Safe

Summary
- Patch release with bug fixes/security hardening. No public API changes reported. Same module path v4.
Code changes required
- None expected.
Notes/Evidence
- Project release notes for 4.1.x emphasize fixes to JOSE/JWT processing while retaining the API.

github.com/google/flatbuffers v25.2.10 -> v25.9.23 — ✅ Safe

Summary
- Date-based versions; Go API for table builders and readers remains stable across these updates. Primarily generator/runtime improvements and fixes.
Code changes required
- None expected.
Notes/Evidence
- The Go runtime API for FlatBuffers rarely changes in a breaking way between same-year minor updates; module remains “+incompatible” but stable.

github.com/hashicorp/go-retryablehttp v0.7.7 -> v0.7.8 — ✅ Safe

Summary
- Patch with minor fixes (e.g., retry logic, context handling, logging). No breaking API changes advertised.
Code changes required
- None expected.
Notes/Evidence
- Changelog from 0.7.8 indicates bugfixes and internal enhancements only.

github.com/klauspost/cpuid/v2 v2.2.11 -> v2.3.0 — ✅ Safe

Summary
- Minor update detecting newer CPU features; API remains stable for feature probing.
Code changes required
- None expected.

golang.org/x/mod v0.29.0 -> v0.31.0 — ✅ Safe

Summary
- Used mostly in tooling/build contexts; incremental changes to module-loading internals with compatible API.
Code changes required
- None expected.

golang.org/x/term v0.37.0 -> v0.38.0 — ✅ Safe

Summary
- Minor platform updates; stable public API.
Code changes required
- None expected.

golang.org/x/telemetry 2025-10-08 -> 2025-12-15 pseudo-version — ✅ Safe

Summary
- Internal Go telemetry client library; not generally used at runtime unless explicitly imported. Updates focus on collection reliability and internal wiring.
Code changes required
- None expected (unless you directly use x/telemetry APIs, which is uncommon).

Notes

Net-new dependencies (not assessed per rules):
- github.com/grafana/databricks-prometheus-exporter (direct)
- github.com/databricks/databricks-sql-go (indirect)
- github.com/apache/arrow/go/v12 (indirect)
- github.com/coreos/go-oidc/v3 (indirect)
- github.com/rs/zerolog (indirect)
- github.com/dnephin/pflag (indirect)
- github.com/google/shlex (indirect)
- gotest.tools/gotestsum (indirect)
A new component prometheus.exporter.databricks was added; this review only evaluates dependency changes.

- Add github.com/grafana/databricks-prometheus-exporter to go.mod - Fix logger type conversion (go-kit → slog) in NewIntegration - Update GitHub issue templates with new component labels

github-actions · 2026-01-05T15:42:58Z

💻 Deploy preview available (feat(prometheus.exporter.databricks): add Databricks exporter component):

https://deploy-preview-alloy-5054-zb444pucvq-vp.a.run.app/docs/alloy/latest/

Run 'go generate ./internal/tools/docs_generator/' to include prometheus.exporter.databricks in the compatible components list.

Add prometheus.exporter.databricks to TestInstanceKey to ensure all exporters are covered by the instance key test suite.

github-actions · 2026-01-07T09:53:41Z

💻 Deploy preview deleted (feat(prometheus.exporter.databricks): Add Databricks exporter component).

docs/sources/reference/components/prometheus/prometheus.exporter.databricks.md

…er.databricks.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

clayton-cornell · 2026-01-07T13:23:03Z

Docs look OK. Over to @grafana/grafana-alloy-maintainers for code review.

aalhour · 2026-01-07T14:36:44Z

Thanks @clayton-cornell, any specific person I can ask to review? I just added the team slug to the list of reviewers.

clayton-cornell · 2026-01-07T17:44:38Z

any specific person I can ask to review?

Nope. The team reviewer thing is enough. The whole Alloy team gets notified. They get to is soon as they can.

docs/sources/reference/components/prometheus/prometheus.exporter.databricks.md

internal/static/integrations/databricks_exporter/databricks_exporter.go

- Add Lookback windows subsection explaining sliding SQL queries - Document source tables for each lookback parameter - Update default values: jobs/pipelines 3h, queries 2h - Update example scrape config to 10m/9m - Improve tuning recommendations with lookback vs interval guidance

Remove the duplicated config/static pattern by consolidating everything into the component package. Since Databricks is a new exporter that was never part of the legacy agent, there's no need to support the legacy integration system.

docs/sources/reference/components/prometheus/prometheus.exporter.databricks.md

…er.databricks.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

dehaansa

LGTM.

Ran 'go mod tidy' in collector/ to update dependencies after 'go generate' added new OpenTelemetry Collector components. This fixes the CI error: 'go: updates to go.mod needed'

Ran 'make generate-module-dependencies' to propagate dependency updates from collector/go.mod to extension/alloyengine/go.mod. This fixes the CI error: 'go.mod files are out of sync with generate-module-dependencies'

aalhour self-assigned this Dec 10, 2025

aalhour force-pushed the bundle-databricks-exporter branch from b17d1d1 to ae221d2 Compare December 10, 2025 14:15

aalhour force-pushed the bundle-databricks-exporter branch from ae221d2 to 136572e Compare December 15, 2025 14:17

aalhour added 3 commits January 5, 2026 12:19

Merge branch 'main' into bundle-databricks-exporter

70478a6

chore(databricks): add databricks-prometheus-exporter dependency

07af161

Add github.com/grafana/databricks-prometheus-exporter v0.0.0-20251219150331-5730cb38c831 to go.mod, matching the Snowflake exporter pattern.

aalhour changed the title ~~WIP - feat(prometheus.exporter.databricks): add Databricks exporter component~~ feat(prometheus.exporter.databricks): add Databricks exporter component Jan 5, 2026

aalhour marked this pull request as ready for review January 5, 2026 15:30

aalhour requested review from a team and clayton-cornell as code owners January 5, 2026 15:30

chore(databricks): add databricks-prometheus-exporter dependency

89bf968

- Add github.com/grafana/databricks-prometheus-exporter to go.mod - Fix logger type conversion (go-kit → slog) in NewIntegration - Update GitHub issue templates with new component labels

aalhour added 2 commits January 5, 2026 17:06

docs: regenerate compatibility docs for databricks exporter

99455ee

Run 'go generate ./internal/tools/docs_generator/' to include prometheus.exporter.databricks in the compatible components list.

test: add databricks exporter to instance key test coverage

c5ed250

Add prometheus.exporter.databricks to TestInstanceKey to ensure all exporters are covered by the instance key test suite.

aalhour changed the title ~~feat(prometheus.exporter.databricks): add Databricks exporter component~~ feat(prometheus.exporter.databricks): Add Databricks exporter component Jan 7, 2026

Merge branch 'main' into bundle-databricks-exporter

31b9c98

clayton-cornell reviewed Jan 7, 2026

View reviewed changes

aalhour and others added 5 commits January 7, 2026 13:00

Update docs/sources/reference/components/prometheus/prometheus.export…

1ecac04

…er.databricks.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Update docs/sources/reference/components/prometheus/prometheus.export…

fe2177c

…er.databricks.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Update docs/sources/reference/components/prometheus/prometheus.export…

0da0499

…er.databricks.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Update docs/sources/reference/components/prometheus/prometheus.export…

59eee6a

…er.databricks.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

fix(docs): improve docs for default interval and timeout values

d89956f

aalhour enabled auto-merge (squash) January 7, 2026 12:12

Merge branch 'main' into bundle-databricks-exporter

458ac60

clayton-cornell added the type/docs Docs Squad label across all Grafana Labs repos label Jan 7, 2026

aalhour requested a review from a team January 7, 2026 14:36

Merge branch 'main' into bundle-databricks-exporter

0754321

dehaansa reviewed Jan 7, 2026

View reviewed changes

docs/sources/reference/components/prometheus/prometheus.exporter.databricks.md Show resolved Hide resolved

internal/static/integrations/databricks_exporter/databricks_exporter.go Outdated Show resolved Hide resolved

aalhour and others added 3 commits January 8, 2026 15:29

Merge branch 'main' into bundle-databricks-exporter

07a34a4

clayton-cornell reviewed Jan 9, 2026

View reviewed changes

docs/sources/reference/components/prometheus/prometheus.exporter.databricks.md Outdated Show resolved Hide resolved

aalhour and others added 2 commits January 9, 2026 10:44

Update docs/sources/reference/components/prometheus/prometheus.export…

35c03cc

…er.databricks.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Merge branch 'main' into bundle-databricks-exporter

ec3fd5d

dehaansa approved these changes Jan 9, 2026

View reviewed changes

aalhour and others added 4 commits January 12, 2026 09:47

Merge branch 'main' into bundle-databricks-exporter

2cf28c9

chore(merge): Resolve conflicts

11a03f2

chore(deps): Update go.mod and go.sum after go generate

26a3212

Ran 'go mod tidy' in collector/ to update dependencies after 'go generate' added new OpenTelemetry Collector components. This fixes the CI error: 'go: updates to go.mod needed'

chore(deps): Synchronize module dependencies

f609df3

Ran 'make generate-module-dependencies' to propagate dependency updates from collector/go.mod to extension/alloyengine/go.mod. This fixes the CI error: 'go.mod files are out of sync with generate-module-dependencies'

aalhour merged commit 4442836 into main Jan 12, 2026
52 of 53 checks passed

aalhour deleted the bundle-databricks-exporter branch January 12, 2026 10:15

grafana-alloybot bot mentioned this pull request Jan 10, 2026

Unreleased Changes #5170

Draft

grafana-alloybot bot mentioned this pull request Jan 26, 2026

chore(release/v1.13): Release 1.13.0 #5351

Merged

github-actions bot added the frozen-due-to-age label Jan 27, 2026

github-actions bot locked as resolved and limited conversation to collaborators Jan 27, 2026

Conversation

aalhour commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Description

Which issue(s) this PR fixes

Notes to the Reviewer

PR Checklist

Uh oh!

CLAassistant commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Dependency Review

Notes

Uh oh!

github-actions bot commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

clayton-cornell commented Jan 7, 2026

Uh oh!

aalhour commented Jan 7, 2026

Uh oh!

clayton-cornell commented Jan 7, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dehaansa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

aalhour commented Dec 10, 2025 •

edited

Loading

CLAassistant commented Dec 10, 2025 •

edited

Loading

github-actions bot commented Dec 10, 2025 •

edited

Loading

github-actions bot commented Jan 5, 2026 •

edited

Loading

github-actions bot commented Jan 5, 2026 •

edited

Loading

github-actions bot commented Jan 7, 2026 •

edited

Loading