-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Description
What happened:
When using Docker versions with complex suffixes like v20.10.12-v1.0.2, cAdvisor fails to parse container metadata labels (e.g., container, pod, namespace, image), resulting in empty label values in metrics.
Example metric:
container_cpu_usage_seconds_total{
container="",
pod="",
namespace="",
image=""
} 0.184540494
What expected to happen :
Container labels should be correctly populated from Docker metadata, allowing proper association of metrics with Kubernetes pods and containers.
How to reproduce it :
- Install Docker with version containing multiple suffixes (e.g., v 20.10.12-v 1.0.2)
- Start kubelet and scrape /metrics/cadvisor
- Observe that container-level metrics have empty values for Kubernetes-related labels
Root cause:
cAdvisor uses a strict regex (\d+)\.(\d+)\.(\d+) to parse Docker version strings. This regex fails to match versions that contain multiple hyphens (e.g., v 20.10.12-v 1.0.2), causing the Docker-specific handler to fail registration and fall back to the raw handler, which does not enrich metrics with Kubernetes metadata.
cadvisor/container/docker/docker.go at master · google/cadvisor
func ParseVersion(versionString string, regex *regexp.Regexp, length int) ([]int, error) {
matches := regex.FindAllStringSubmatch(versionString, -1)
if len(matches) != 1 {
return nil, fmt.Errorf("version string \"%v\" doesn't match expected regular expression: \"%v\"", versionString, regex.String())
}
versionStringArray := matches[0][1:]
versionArray := make([]int, length)
for index, versionStr := range versionStringArray {
version, err := strconv.Atoi(versionStr)
if err != nil {
return nil, fmt.Errorf("error while parsing \"%v\" in \"%v\"", versionStr, versionString)
}
versionArray[index] = version
}
return versionArray, nil
}Proposed fix :
To improve compatibility with custom or extended Docker version strings, I propose the following fix:
Even if the Docker version string contains non-standard suffixes, cAdvisor should still attempt to extract and use the first valid semantic version part (e.g., X.Y.Z) to register the Docker handler. This ensures that container metadata enrichment (e.g., pod, namespace, container, image) continues to function normally, without falling back to the raw handler.
- Improve version parsing logic
- Use the regular expression
(\d+\.\d+\.\d+)to extract the first matching semantic version part (e.g., 20.10.12). - If no valid version is found, return an error.
- If a match is found, continue to register and use the Docker handler normally.
- Add warning logging
- When a version string contains non-standard suffixes is detected, log a warning message in English for better compatibility with internationalized logging systems.
- Example log:
[WARNING] Detected non-standard Docker version "v20.10.12-v1.0.2", using base version "20.10.12" for handler registration