This benchmark uses Keycloak, an Open Source Identity and Access Management solution for modern applications and services. Keycloak is a large-scale Java/Maven project with complex dependency management, making it an excellent benchmark target.
Version: 26.4.7
Tools from the sbomify resource list, specifically:
- Trivy
- Syft
- sbomify github-action
The benchmark workflow:
- Clones the Keycloak repository at the specified tag
- Runs each SBOM generator against the project's
pom.xmland source tree - Scores each generated SBOM using sbomqs
- Produces a comparison table in the workflow summary
The full process is automated and you can see the exact commands in java.yml.
If you look at the Java CI/CD run, you can see the quality score of the SBOMs (from sbomqs) as well as download the actual SBOMs as artifacts.
Keycloak was chosen as a benchmark target because:
- Complex dependency tree: Hundreds of Maven dependencies across multiple modules
- Real-world project: Actively maintained, widely deployed enterprise software
- Mixed ecosystem: Combines Java libraries, JavaScript frontend components, and more
- Multi-module Maven: Tests SBOM generators' ability to handle complex build structures