Prometheus S3 Exporter

• Purpose: Export S3-compatible storage metrics for Prometheus.
• Compatibility: Works with AWS S3 and third‑party S3 servers (e.g., MinIO, Ceph RADOSGW, Wasabi) via the official MinIO Go client.

Features

• Bucket metrics per target: size bytes, object count, oldest/newest object timestamps.
• Storage class size breakdown per bucket.
• Supports multiple endpoints/targets concurrently.
• Background refresh at configurable interval; Prometheus scrapes cached metrics quickly.
• Simple TOML configuration. Optional bucket discovery and prefix filters.

Install

• Build: go build -o prometheus-s3-exporter .
• Run: ./prometheus-s3-exporter -config config.toml

Config

• See config.example.toml for a complete example.
• Minimal example for MinIO:

[server]
listen_address = ":9877"
metrics_path = "/metrics"

[[target]]
name = "minio"
endpoint = "localhost:9000"
access_key = "minioadmin"
secret_key = "minioadmin"
secure = false
path_style = true
refresh_interval = "10m"
timeout = "60s"

Environment overrides (optional):

• S3_EXPORTER_ACCESS_KEY, S3_EXPORTER_SECRET_KEY — used if a target omits credentials.

Prometheus Scrape Example

scrape_configs:
  - job_name: s3
    static_configs:
      - targets: ["localhost:9877"]

Exported Metrics

• s3_bucket_size_bytes{target,endpoint,region,bucket}: Sum of object sizes.
• s3_bucket_objects_total{target,endpoint,region,bucket}: Object count.
• s3_bucket_oldest_object_timestamp{...}: Oldest object mtime (unix seconds).
• s3_bucket_newest_object_timestamp{...}: Newest object mtime (unix seconds).
• s3_bucket_storage_class_size_bytes{...,storage_class}: Bytes per storage class.
• s3_scrape_success{target,endpoint}: Last refresh success (1/0).
• s3_scrape_duration_seconds{target,endpoint}: Duration of each refresh.

Health endpoints:

• /-/ready readiness; /-/healthy liveness.

Notes & Caveats

• Bucket size requires walking objects. For very large buckets, choose a longer refresh_interval and tune timeout. Consider prefix filters to limit scope.
• When buckets is empty, all buckets are discovered for the target. Set explicitly to restrict.
• Set path_style = true for path-style services (e.g., MinIO dev). Use TLS via secure = true and insecure_skip_verify = true only for testing.