Sync rekor-tiles protos into protobuf-specs by loosebazooka · Pull Request #661 · sigstore/protobuf-specs

loosebazooka · 2025-06-05T14:44:37Z

sync is by manually running service-protos/sync-rekor-tiles.sh
- this current commit does HEAD
pull in latest tag (it doesn't do releases) from rekor-tiles
rewrite the go_package to match protobuf specs
do NOT put service protos in ./protos because we export those in the service-builder container image
rework language builder to include ./service-protos
include ./service-protos in exported jar
remove jsonschema
rust updated (thanks @jku)

Summary

Release Note

Documentation

loosebazooka · 2025-06-05T14:56:19Z

Looks like typescript would need an import of "bufbuild/wire", this is only a problem because we're also sync'ing in the service definition. I can "exclude" that from the sync. service-protos kind of takes on a different meaning then

jku · 2025-06-05T15:33:51Z

handle rust code gen in the rust codebase?

if the issue is that you didn't want to poke the custom rust codegen code, I can give it a try (no promises on results though)

loosebazooka · 2025-06-05T15:35:26Z

if the issue is that you didn't want to poke the custom rust codegen code, I can give it a try (no promises on results though)

I took a quick look, and decided someone else maybe would want to. I don't think it's very complicated, was just worried I was gonna burn a bunch of hours

jku · 2025-06-05T16:38:03Z

Two commits for rust in https://github.com/jku/protobuf-specs/commits/sync-rekor-tiles/ if you want them (first is the codegen change, second is "make rust" result)

(I can make a separate PR as well, let me know)

- sync is by manually running service-protos/sync-rekor-tiles.sh - pull in latest tag (it doesn't do releases) from rekor-tiles - rewrite the go_package to match protobuf specs - ignore protos for grpc service defintions - do NOT put service protos in ./protos because we export those in the service-builder container image - rework language builder to include ./service-protos - include ./service-protos in exported jar - remove jsonschema TODO: - handle rust code gen in the rust codebase? Signed-off-by: Appu Goundan <appu@google.com>

loosebazooka · 2025-06-05T17:43:20Z

oh woops, I updated, amended and force pushed. I'll cherry pick in your diffs

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Hayden-IO

Great work on this!

Do you want to add a nightly GHA that tracks when there are new tags and changes to the protos in all services?

Hayden-IO · 2025-06-05T18:38:20Z

Makefile

+	${DOCKER_RUN} -v ${PWD}:/defs ${PROTOC_RUBY_IMAGE} ${PROTO_INCLUDES}\
+	  --ruby_out=/defs/gen/pb-ruby/lib ${PROTOS}

+# TODO: serivce-proto code gen for rust


Delete TODO?

loosebazooka · 2025-06-05T18:39:36Z

Do you want to add a nightly GHA that tracks when there are new tags and changes to the protos in all services?

I can add it in a followup, this is already kind of a hefty PR.

Signed-off-by: Appu Goundan <appu@google.com>

Hayden-IO · 2025-06-05T18:41:25Z

gen/pb-ruby/lib/rekor_service_pb.rb

Is this supposed to be here, or is this leftover from a generation with the service proto?

Maybe try a make clean; make all to confirm?

I thought I did, weird.

gotta update the makefile

I think there are some other ruby things in there that might need updating: https://github.com/sigstore/protobuf-specs/blob/main/gen/pb-ruby/lib/sigstore_protobuf_specs.rb

@segiddins ?

we're not using the generated protos from this repo since they depend on the protobuf gem

Signed-off-by: Appu Goundan <appu@google.com>

loosebazooka · 2025-06-05T19:04:27Z

it looks like typescript needs an update too, to: https://github.com/sigstore/protobuf-specs/blob/main/gen/pb-typescript/src/index.ts @bdehamer ?

loosebazooka · 2025-06-05T20:04:47Z

Should I add TODOs/issues for typescript and ruby?

Hayden-IO · 2025-06-05T20:43:32Z

What's the current issue(s)? Given we aren't immediately targeting JS/Ruby Rekor v2 support, filing issues for now would be fine. Should we omit js/ruby entirely if there are issues?

jku

leaving further ts and ruby changes to issues seems fine, let's just ping the relevant folks

jku · 2025-06-06T08:52:27Z

service-protos/sync-rekor-tiles.sh

+git clone --filter=blob:none --no-checkout --depth=1 https://github.com/sigstore/rekor-tiles.git ./rekor-tiles && \
+    cd ./rekor-tiles && \
+    git sparse-checkout set --no-cone '/api/proto/rekor/v2/*.proto' '!**/rekor_service.proto' && \
+    git fetch origin tag "$latest_tag" --no-tags && \
+    git checkout "$latest_tag" && \
+    cd ../ && \
+    cp -R ./rekor-tiles/api/proto/* . && \
+    rm -rf ./rekor-tiles


was there a reason to use "&&" here? You use errexit already so it shouldn't be needed

No. Started with a script from somewhere else and continued with that style. It's unnecessary, I wasnt thinking.

jku · 2025-06-06T09:19:23Z

This is just a note, not suggesting any changes:

do NOT put service protos in ./protos

I thought this would have no affect on the sigstore-python PR but it turns out we did actually use CreateEntryRequest from the service protos to build the request message. Doing that manually is not an issue.

loosebazooka · 2025-06-06T11:58:09Z

I thought this would have no affect on the sigstore-python PR but it turns out we did actually use CreateEntryRequest from the service protos to build the request message. Doing that manually is not an issue.

Oh I didn't think about that. Hrmmm.. seems silly to not include it.

Signed-off-by: Appu Goundan <appu@google.com>

loosebazooka · 2025-06-06T14:18:00Z

okay cleaned up the script a bit

loosebazooka · 2025-06-06T14:23:53Z

oh shoot, jku contributed to this PR so need another review.

Hayden-IO

@loosebazooka does this need to be rerun with the new tag or is this generated from a fix locally?

loosebazooka · 2025-06-06T15:50:08Z

Will merge, and rerun with updates on a new tag

bdehamer · 2025-06-09T22:06:18Z

Open #669 to expose these new types as part of the generated npm package.

loosebazooka force-pushed the sync-rekor-tiles branch from bc4bcd8 to 1c55db8 Compare June 5, 2025 17:41

rust codegen: Include rekor v2 protos in codegen

fdf108c

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

loosebazooka marked this pull request as ready for review June 5, 2025 18:04

loosebazooka requested review from Hayden-IO, jku and kommendorkapten June 5, 2025 18:17

Hayden-IO previously approved these changes Jun 5, 2025

View reviewed changes

Hayden-IO reviewed Jun 5, 2025

View reviewed changes

rust: Update generated code

6a8402f

Signed-off-by: Appu Goundan <appu@google.com>

loosebazooka dismissed Hayden-IO’s stale review via 6a8402f June 5, 2025 18:40

loosebazooka force-pushed the sync-rekor-tiles branch from ed0752a to 6a8402f Compare June 5, 2025 18:40

loosebazooka requested a review from Hayden-IO June 5, 2025 18:41

Hayden-IO reviewed Jun 5, 2025

View reviewed changes

fix ruby clean

bb9f477

Signed-off-by: Appu Goundan <appu@google.com>

Hayden-IO previously approved these changes Jun 5, 2025

View reviewed changes

loosebazooka mentioned this pull request Jun 5, 2025

rekor v2 support super issue sigstore/sigstore-java#982

Closed

jku mentioned this pull request Jun 6, 2025

Add Rekor v2 client sigstore/sigstore-python#1422

Merged

jku approved these changes Jun 6, 2025

View reviewed changes

jku reviewed Jun 6, 2025

View reviewed changes

loosebazooka mentioned this pull request Jun 6, 2025

Move createEntryRequest into entry.proto sigstore/rekor-tiles#340

Merged

clean up sync script

4eac02b

Signed-off-by: Appu Goundan <appu@google.com>

loosebazooka dismissed Hayden-IO’s stale review via 4eac02b June 6, 2025 14:17

loosebazooka mentioned this pull request Jun 6, 2025

Handle new code gen for rekor v2 protos #664

Open

6 tasks

loosebazooka requested review from Hayden-IO and jku June 6, 2025 14:23

Hayden-IO approved these changes Jun 6, 2025

View reviewed changes

loosebazooka merged commit b7e49be into main Jun 6, 2025
27 checks passed

loosebazooka deleted the sync-rekor-tiles branch June 6, 2025 15:50

bdehamer mentioned this pull request Jun 9, 2025

export rekor types from pb-typescript #669

Merged

Conversation

loosebazooka commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Release Note

Documentation

Uh oh!

loosebazooka commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jku commented Jun 5, 2025

Uh oh!

loosebazooka commented Jun 5, 2025

Uh oh!

jku commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

loosebazooka commented Jun 5, 2025

Uh oh!

Hayden-IO left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

loosebazooka commented Jun 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

loosebazooka commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

loosebazooka commented Jun 5, 2025

Uh oh!

Hayden-IO commented Jun 5, 2025

Uh oh!

jku left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jku commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

loosebazooka commented Jun 6, 2025

Uh oh!

loosebazooka commented Jun 6, 2025

Uh oh!

loosebazooka commented Jun 6, 2025

Uh oh!

Hayden-IO left a comment

Choose a reason for hiding this comment

Uh oh!

loosebazooka commented Jun 6, 2025

Uh oh!

Uh oh!

bdehamer commented Jun 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

loosebazooka commented Jun 5, 2025 •

edited

Loading

loosebazooka commented Jun 5, 2025 •

edited

Loading

jku commented Jun 5, 2025 •

edited

Loading

loosebazooka commented Jun 5, 2025 •

edited

Loading

jku commented Jun 6, 2025 •

edited

Loading