Skip to content

Spec design for release strategy and versioned tags #102

@aryeko

Description

@aryeko

Motivation

The benchmarks repository currently has no releases or tags. Tagged releases serve multiple purposes:

  • Pin methodology versions: The METHODOLOGY.md already has a changelog policy, but there is no corresponding versioned snapshot to reference.
  • Stable references: Dependents and consumers of benchmark results need a way to reference a known-good state.
  • Project maturity signal: Releases indicate an actively maintained project and provide clear milestones.

Goal

Define a semver-based release strategy starting from v0.1.0 on the current stable state.

Requirements

  • Initial tag: Create v0.1.0 on current main once this spec is approved.
  • Version bump semantics: Define what constitutes each bump level:
    • Major: Breaking methodology changes, incompatible fixture contract changes.
    • Minor: New benchmark targets, new report dimensions, additive fixture changes.
    • Patch: Bug fixes, script improvements, documentation corrections.
  • CHANGELOG policy: Decide between CHANGELOG.md (in-repo) vs GitHub Release notes (auto-generated or hand-written).
  • CI automation: Evaluate goreleaser or a manual GitHub Actions workflow for tag-triggered releases.
  • Alignment with METHODOLOGY.md: Ensure methodology changelog entries reference the corresponding release version.

Open questions

  • Should releases include generated artifacts (report.md, summary.json) as release assets?
  • Should methodology version and release version be kept in sync or tracked independently?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions