Skip to content

docs(storage): add open_object_walkthrough.md#5783

Open
joshuatants wants to merge 2 commits into
googleapis:mainfrom
joshuatants:open_object_walkthrough
Open

docs(storage): add open_object_walkthrough.md#5783
joshuatants wants to merge 2 commits into
googleapis:mainfrom
joshuatants:open_object_walkthrough

Conversation

@joshuatants
Copy link
Copy Markdown
Contributor

The fruit of my efforts to understand open_object. Adding it to the repo as it might be:

  1. useful for future contributors to the codebase
  2. useful for bootstrapping the AI for the Elephant-Goldfish Model.

@product-auto-label product-auto-label Bot added the api: storage Issues related to the Cloud Storage API. label May 29, 2026
@joshuatants joshuatants requested review from vsharonlynn and xlai20 May 29, 2026 05:42
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds a comprehensive documentation walkthrough (open_object_walkthrough.md) for the Storage::open_object bidirectional streaming RPC in the google-cloud-storage crate. The document details the architectural mental model, call graphs, linear execution traces, channel multiplexing, and the client-side resilience machinery such as redirects and reconnects. As there are no review comments associated with this pull request, I have no further feedback to provide.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.88%. Comparing base (81e51dd) to head (2f94ac4).
⚠️ Report is 12 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5783   +/-   ##
=======================================
  Coverage   97.88%   97.88%           
=======================================
  Files         226      226           
  Lines       56465    56465           
=======================================
  Hits        55272    55272           
  Misses       1193     1193           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joshuatants joshuatants force-pushed the open_object_walkthrough branch from 6067bb5 to 2c5ce96 Compare May 29, 2026 05:56
@joshuatants joshuatants marked this pull request as ready for review May 29, 2026 05:56
@joshuatants joshuatants requested review from a team as code owners May 29, 2026 05:56
Copy link
Copy Markdown
Contributor

@coryan coryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be durable to save the prompts used to produce this, rather than the output. The document may be helpful, but it is more helpful to produce updated versions of the doc.

Comment on lines +628 to +629
never leave the application). The latter two represent the actual **gRPC stream
halves** sent over the network.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gRPC streams halves are not "sent over the network".... If anything, they send and receive messages to and from the network.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cleaned up the section.

Comment on lines +1840 to +1851
The server is entirely stateless; it retains no per-read progress state across
streams. The server acts as a dumb, incredibly fast pipe. **The entire
conversation state lives on the client.**

Because every dial is completely self-contained, any backend can serve any
portion of any read at any given time. Backends can vanish without warning, and
the client gracefully self-heals.

> **Takeaway:** The server is the dumb-but-fast pipe; the client is the
> smart-but-stateful brain. The SDK's worker, equipped with its
> `HashMap<i64, ActiveRead>`, is literally that brain in code form—a live,
> dynamic model tracking exactly "where we are in every conversation."
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just nonsense.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

Comment on lines +1620 to +1622
gracefully shut down (`worker.rs:88–93`). By keeping a clone here, we creatively
repurpose standard channel liveness semantics into a foolproof reference
counting mechanism.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I cannot stand the sycophantic verbiage produced by LLMs 🤷 The comment ("... holding a copy prevents ...") gives you all the information you need.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a particulary egregious example. Updated the wording to

read-request channel as its signal to shut down (`worker.rs:88–93`), holding a
clone here keeps the worker alive.```

Comment on lines +1454 to +1456
r.as_proto(id) ──▶ ProtoRange { read_id: 7, ... }
Server processes read
◀── ObjectRangeData {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of inconsistent arrow types is annoying.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cleaned up.

@@ -0,0 +1,1885 @@
# `open_object` — A Walkthrough

*Last updated: 2026-05-27*
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is full of line numbers, you may want to capture the exact commit SHA

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent idea. Done.

@joshuatants
Copy link
Copy Markdown
Contributor Author

joshuatants commented Jun 1, 2026

I think it would be durable to save the prompts used to produce this, rather than the output. The document may be helpful, but it is more helpful to produce updated versions of the doc.

Good point. I added a How this document was created at the top of the doc.

Also, to your other comments and as I note in the new section, I naively assumed that correcting the AI in conversation and making the AI go over the doc would result in a quality artifact. I guess not.

To remedy that, I'll be going over the doc manually and checking it in piecemeal.

@joshuatants joshuatants force-pushed the open_object_walkthrough branch from 26fdc25 to 2f94ac4 Compare June 1, 2026 07:30
@coryan
Copy link
Copy Markdown
Contributor

coryan commented Jun 1, 2026

Thanks! I plan to use these prompts in other places too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: storage Issues related to the Cloud Storage API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants