[codex-analytics] emit terminal review events by rhan-oai · Pull Request #18748 · openai/codex

rhan-oai · 2026-04-20T21:26:36Z

Why

Review telemetry should describe reviews as first-class events, not only as counters denormalized onto terminal tool-item events. That lets us analyze guardian and user reviews consistently across command execution, file changes, permissions, and network access, while still preserving the terminal item summaries that existing tool analytics need.

To make those review events accurate, analytics also needs the observed completion time for each review and enough command metadata to distinguish shell from unified_exec reviews.

What changed

emit generic codex_review_event rows for completed user and guardian reviews, with review subjects, reviewer, trigger, terminal status, resolution, and observed duration
reduce approval request / response / abort facts into review events for command execution, file change, and permissions flows
keep denormalized review counts, final approval outcome, and permission-request flags on terminal tool-item events for item-associated reviews
plumb review completion timing so user-review responses and aborts use app-server-observed completion times, while guardian analytics reuse the same terminal timestamps emitted on guardian assessment events
carry command approval source through the protocol and app-server layers so review analytics can distinguish shell from unified_exec
add analytics coverage for user-review emission, guardian-review emission, permission reviews that should not denormalize onto tool items, item-summary isolation across threads, and the serialized review-event shape

Verification

cargo test -p codex-analytics

Stack created with Sapling. Best reviewed with ReviewStack.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0c8f41eab9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-20T21:44:07Z

+                let status = match response.scope {
+                    PermissionGrantScope::Turn => ToolReviewStatus::Approved,
+                    PermissionGrantScope::Session => ToolReviewStatus::ApprovedForSession,
+                };


Classify empty permission grants as denied reviews

PermissionsRequestApproval maps status from scope only, so Turn is always emitted as approved. But UI denial paths return an empty permissions profile with scope=Turn (see approval_overlay::handle_permissions_decision). That causes denied/timed-out permission reviews to be reported as approvals and denormalized into incorrect final_approval_outcome values on tool item events.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-04-20T21:44:07Z

+        let started = self.guardian_review_starts.remove(&notification.review_id);
+        let created_at = started
+            .as_ref()
+            .map(|start| start.created_at)
+            .unwrap_or(completed_at);
+        let Some(status) = guardian_review_status(notification.review.status) else {


Deduplicate guardian review completion notifications

This handler emits a review even when guardian_review_starts.remove(review_id) returns None. Because notifications are tracked per connection, multi-client threads ingest the same completion once per connection, creating duplicate codex_tool_call_review_event records and inflating per-item review counters.

Useful? React with 👍 / 👎.

rhan-oai · 2026-05-08T05:07:09Z

@codex

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e4cc9b8873

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-05-08T05:22:12Z

+                        turn_id: params.turn_id,
+                        item_id: Some(params.item_id),
+                        review_id: user_review_id(&request_id),
+                        subject_kind: ReviewSubjectKind::CommandExecution,


Classify network approval reviews as network access

When params.network_approval_context is present, this request is the app-server's Network approval presentation, not a command execution review. Recording it unconditionally as CommandExecution with subject_name of shell/unified_exec makes user network-access review telemetry inconsistent with guardian network reviews and the commit's stated first-class network access subjects.

Useful? React with 👍 / 👎.

owenlin0 · 2026-05-08T21:50:25Z

-                ..
-            } => {}
+                completed_at_ms,
+                response,


This path consumes ServerResponse facts that app-server records from the raw JSON-RPC result before the approval-specific handlers apply their effective fallback/validation. If the client returns a malformed command/file approval response, the handler later falls back to Decline, but analytics emits no review event because response_from_result never produced a typed response. For permissions, analytics can mark a raw non-empty grant as approved before the handler intersects/validates it and potentially grants nothing. Consider tracking the effective approval decision after the approval handler computes it, so analytics matches the behavior actually applied to the turn.

^ codex-generated - if it complicates the implementation too much I actually don't think it's worth doing, fwiw

feels like we should be tracking malformed client responses separately anyways

fixed up permissions issue, agree on pushing off malformed response tracking

owenlin0 · 2026-05-08T21:54:24Z

+    ///
+    /// Uses `#[serde(default)]` for backwards compatibility with older senders.
+    #[serde(default = "default_guardian_command_source")]
+    pub source: GuardianCommandSource,


let's remove this, I don't want to add this implementation detail to the public app-server API.

can we populate codex_review_event.subject_name as just command_execution instead?

rhan-oai · 2026-05-11T16:48:50Z

@codex

chatgpt-codex-connector · 2026-05-11T17:01:58Z

Codex Review: Didn't find any major issues. More of your lovely PRs please.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

owenlin0 · 2026-05-11T18:55:36Z

+    ///
+    /// Uses `#[serde(default)]` for backwards compatibility with older senders.
+    #[serde(default = "default_guardian_command_source")]
+    pub source: GuardianCommandSource,


Can we remove this too? We also polluted the protocol a bit

whoops yes my bad

owenlin0 · 2026-05-11T19:00:18Z

+    let (assessment, count_denial_for_circuit_breaker, completed_at_ms) = match outcome {
        GuardianReviewOutcome::Completed(assessment) => {
            let approved = matches!(assessment.outcome, GuardianAssessmentOutcome::Allow);
+            let completed_at_ms = now_unix_timestamp_ms();


nit: we do this in each of the match outcome branch arms below. can we define it once right above the match outcome?

owenlin0

just some small comments at this point, but preapproving

chatgpt-codex-connector Bot reviewed Apr 20, 2026

View reviewed changes

rhan-oai marked this pull request as draft April 20, 2026 21:47

rhan-oai force-pushed the pr18748 branch 4 times, most recently from f61bae4 to dfed0b4 Compare April 22, 2026 07:02

rhan-oai force-pushed the pr18748 branch 10 times, most recently from 6158941 to edf6a86 Compare April 29, 2026 01:53

github-actions Bot mentioned this pull request Apr 29, 2026

📊 AI CLI 工具社区动态日报 2026-04-29 borq168/radar-forge#67

Open

rhan-oai force-pushed the pr18748 branch 4 times, most recently from 507ec65 to 7a1a231 Compare April 29, 2026 17:51

rhan-oai changed the base branch from main to pr18747 April 29, 2026 17:52

This was referenced Apr 29, 2026

[codex-analytics] replay protocol-native item timing #20239

Closed

[codex-analytics] centralize thread analytics state #20241

Merged

rhan-oai force-pushed the pr18747 branch from 9d74ec3 to 189f24a Compare April 29, 2026 18:15

rhan-oai force-pushed the pr18748 branch from 7a1a231 to 82d5268 Compare April 29, 2026 18:15

rhan-oai force-pushed the pr18747 branch from 4e75e65 to 2db917f Compare May 2, 2026 06:03

rhan-oai force-pushed the pr18748 branch 2 times, most recently from a51cdee to e70e29a Compare May 3, 2026 03:38

rhan-oai force-pushed the pr18747 branch 2 times, most recently from 0835613 to 9795736 Compare May 3, 2026 06:30

rhan-oai force-pushed the pr18748 branch from e70e29a to aed879c Compare May 3, 2026 06:30

rhan-oai force-pushed the pr18747 branch from 9795736 to 0a0cb1a Compare May 3, 2026 23:31

rhan-oai force-pushed the pr18748 branch 2 times, most recently from ed9f62b to fd08089 Compare May 3, 2026 23:43

rhan-oai force-pushed the pr18747 branch 2 times, most recently from e246f33 to 637e3bb Compare May 4, 2026 22:13

rhan-oai force-pushed the pr18748 branch from fd08089 to dd1483a Compare May 4, 2026 22:13

rhan-oai force-pushed the pr18747 branch from 637e3bb to 4b59b83 Compare May 4, 2026 22:30

rhan-oai force-pushed the pr18748 branch from dd1483a to c8b8cd8 Compare May 4, 2026 22:31

github-actions Bot mentioned this pull request May 6, 2026

📊 AI CLI 工具社区动态日报 2026-05-06 gsscsd/big_model_radar#300

Open

rhan-oai mentioned this pull request May 6, 2026

[codex-analytics] plumb protocol-native review timing #21434

Merged

github-actions Bot mentioned this pull request May 8, 2026

📊 AI CLI 工具社区动态日报 2026-05-08 zx0828/big_model_radar#43

Open

chatgpt-codex-connector Bot reviewed May 8, 2026

View reviewed changes

owenlin0 reviewed May 8, 2026

View reviewed changes

This was referenced May 11, 2026

📊 AI CLI 工具社区动态日报 2026-05-11 zx0828/big_model_radar#49

Open

📊 AI CLI 工具社区动态日报 2026-05-11 gsscsd/big_model_radar#325

Open

📊 AI CLI 工具社区动态日报 2026-05-11 ivanweng2077/big_model_radar#25

Open

owenlin0 reviewed May 11, 2026

View reviewed changes

owenlin0 approved these changes May 11, 2026

View reviewed changes

[codex-analytics] emit terminal tool review events

dfc4d34

Uh oh!

Conversation

rhan-oai commented Apr 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changed

Verification

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

rhan-oai commented May 8, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot May 8, 2026

Choose a reason for hiding this comment

Uh oh!

owenlin0 May 8, 2026

Choose a reason for hiding this comment

Uh oh!

owenlin0 May 8, 2026

Choose a reason for hiding this comment

Uh oh!

owenlin0 May 8, 2026

Choose a reason for hiding this comment

Uh oh!

rhan-oai May 11, 2026

Choose a reason for hiding this comment

Uh oh!

owenlin0 May 8, 2026

Choose a reason for hiding this comment

Uh oh!

rhan-oai commented May 11, 2026

Uh oh!

chatgpt-codex-connector Bot commented May 11, 2026

Uh oh!

owenlin0 May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rhan-oai May 11, 2026

Choose a reason for hiding this comment

Uh oh!

owenlin0 May 11, 2026

Choose a reason for hiding this comment

Uh oh!

owenlin0 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rhan-oai commented Apr 20, 2026 •

edited

Loading

owenlin0 May 11, 2026 •

edited

Loading