Skip to content

fix(ci): rotate iPad showcase captures without Simulator UI scripting - #5094

Merged
juliusmarminge merged 3 commits into
mainfrom
fix/ipad-landscape-no-accessibility
Jul 31, 2026
Merged

fix(ci): rotate iPad showcase captures without Simulator UI scripting#5094
juliusmarminge merged 3 commits into
mainfrom
fix/ipad-landscape-no-accessibility

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Jul 31, 2026

Copy link
Copy Markdown
Member

Summary

The Mobile Showcase Screenshots workflow's iOS job fails on main (run): #5065 rotates the iPad simulator by AppleScript-clicking Simulator's menus, and the CI runner denies osascript the required macOS Accessibility permission:

execution error: System Events got an error: osascript is not allowed assistive access. (-1719)

simctl has no orientation command (checked through Xcode 26.5), so this replaces UI scripting with in-app rotation. Getting there surfaced a chain of iPadOS constraints; the fix has four pieces:

  1. In-app rotation. t3-native-controls gains getShowcaseOrientation (reads a --showcaseOrientation launch arg the harness now passes), applyShowcaseOrientation (UIWindowScene.requestGeometryUpdate), and getInterfaceOrientation so ShowcaseCaptureCoordinator can verify the rotation took effect and retry. Scene readiness is gated on the orientation having settled, and the orientation getter reports from screen bounds (which mirror the captured framebuffer) — a failed rotation now times out loudly instead of shipping wrong-size screenshots.
  2. iPadOS ignores programmatic orientation for multitasking-capable apps, so the capture build sets requireFullScreen in app.config.ts, gated on T3_SHOWCASE_CAPTURE_BUILD=1 which only the showcase harness sets — production builds are unchanged. (The Expo config key really is requireFullScreen in config-plugins 56; the requiresFullScreen spelling is silently ignored.)
  3. iPadOS 26 windowing runs a requires-full-screen app in a fixed portrait compatibility window, so before landscape captures the harness switches the simulator to Full Screen Apps mode via simctl spawn defaults — writing all three SpringBoard keys the Settings toggle writes (SBChamoisWindowingEnabled, SBMedusaMultitaskingEnabled, SBFlexibleWindowingPreviouslyEnabledAutomaticStageCreation; the first alone is ignored on a freshly created device), then reboots the device and verifies the mode stuck.
  4. Headless displays don't rotate. With Simulator.app attached the display rotates with the app and screenshots come out natively 2752×2064; a headless CI simulator keeps its display portrait with the app rendered sideways inside it. The harness detects which case it got from the PNG dimensions and post-rotates 270° only when needed.

The osascript helper and the unconditional sips --rotate 90 are gone.

Verification

  • iOS CI run on this branch: ✅ green — all 6 captures × 5 scenes; iPad artifacts confirmed upright landscape 2752×2064 by inspection.
  • Full both-platform run: final verification.
  • Locally verified end-to-end against a freshly created simulator (the CI-equivalent path), plus @t3tools/scripts/@t3tools/mobile typechecks and the 197 scripts tests.

🤖 Generated with Claude Code

The landscape iPad captures from #5065 rotate the simulator by
AppleScript-clicking Simulator's menus, which CI cannot do: the runner
denies osascript Accessibility access (error -1719), and simctl has no
orientation command. Rotate from inside the app instead:

- t3-native-controls gains getShowcaseOrientation (launch-arg reader),
  applyShowcaseOrientation (UIWindowScene.requestGeometryUpdate), and
  getInterfaceOrientation so the coordinator can verify the rotation
  actually took effect and retry attempts made before the scene was
  foreground-active.
- iPadOS ignores programmatic orientation for multitasking-capable apps,
  so the capture build sets requireFullScreen (env-gated via
  T3_SHOWCASE_CAPTURE_BUILD; production builds are unchanged).
- iPadOS 26 windowing would put a requires-full-screen app in a fixed
  portrait compatibility window, so the harness switches the simulator
  to Full Screen Apps mode (SBChamoisWindowingEnabled=false + SpringBoard
  restart) before landscape captures.
- The framebuffer is now natively landscape, so the sips post-rotation
  is gone along with the osascript helper.

Verified locally end to end from a clean simulator: the iPad capture
renders upright landscape at 2752x2064 and passes validation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c3756bbc-ef82-4a88-8ff1-4cdc0303610c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Jul 31, 2026
Comment thread apps/mobile/src/features/showcase/ShowcaseCaptureCoordinator.tsx
Comment thread apps/mobile/src/features/showcase/ShowcaseCaptureCoordinator.tsx
// portrait compatibility window, which defeats the in-app landscape rotation
// the capture build relies on. Switch the device to Full Screen Apps mode
// (Settings > Multitasking & Gestures) and restart SpringBoard to apply it.
async function ensureIosFullScreenAppsMode(udid: string): Promise<void> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium scripts/mobile-showcase.ts:802

ensureIosFullScreenAppsMode permanently writes SBChamoisWindowingEnabled=false to the simulator and never restores the previous value. Running a landscape capture against a pre-existing developer simulator leaves that simulator stuck in Full Screen Apps mode after the harness exits, silently changing its multitasking configuration. The function reads the current value (and early-returns when already 0) but neither records it for cleanup nor restores it in the finally block. Consider capturing the prior value and writing it back during iOS cleanup, or only toggling the setting on simulators the runner created.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @scripts/mobile-showcase.ts around line 802:

`ensureIosFullScreenAppsMode` permanently writes `SBChamoisWindowingEnabled=false` to the simulator and never restores the previous value. Running a landscape capture against a pre-existing developer simulator leaves that simulator stuck in Full Screen Apps mode after the harness exits, silently changing its multitasking configuration. The function reads the current value (and early-returns when already `0`) but neither records it for cleanup nor restores it in the `finally` block. Consider capturing the prior value and writing it back during iOS cleanup, or only toggling the setting on simulators the runner created.

…ulators

The first CI run captured a floating portrait-screen window: on a brand
new device SBChamoisWindowingEnabled alone is not honored. The Settings
toggle writes three SpringBoard keys (Chamois windowing, Medusa
multitasking, flexible-windowing stage creation), so write all three,
reboot the simulator instead of kickstarting SpringBoard, and verify the
mode applied. Also gate showcase scene readiness on the orientation
having settled, and report orientation from screen bounds so a floating
landscape window on a portrait screen counts as portrait — an early or
failed rotation now times out loudly instead of shipping wrong-size
screenshots.

Verified locally against a freshly created simulator (the CI-equivalent
path): upright landscape 2752x2064, validation passes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is ON, but a cloud agent failed to start.

Reviewed by Cursor Bugbot for commit 87ff819. Configure here.

"-bool",
"false",
]);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Early exit skips multitasking keys

Medium Severity

The ensureIosFullScreenAppsMode function exits early if SBChamoisWindowingEnabled is 0. This is problematic because fully disabling iPadOS windowing now requires setting three SpringBoard keys. If the other two keys remain enabled, simulators can retain windowing behavior, causing landscape rotation and orientation checks to fail, which leads to showcase capture timeouts or validation failures.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 87ff819. Configure here.

@macroscopeapp

macroscopeapp Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

1 blocking correctness issue found. CI/screenshot capture tooling changes with two unresolved review comments identifying potential bugs: simulator settings are not restored after capture, and the early-exit logic may leave multitasking keys in an inconsistent state. These issues warrant human review before merging.

You can customize Macroscope's approvability policy. Learn more.

…ortrait

A headless CI simulator keeps its display portrait while the rotated app
renders sideways inside it (with Simulator.app attached, the display
itself rotates and no correction is needed). Detect which case we are in
from the captured PNG's dimensions and rotate 270 degrees only when the
buffer is portrait — verified against the actual CI capture.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@juliusmarminge
juliusmarminge merged commit ab8182c into main Jul 31, 2026
21 checks passed
@juliusmarminge
juliusmarminge deleted the fix/ipad-landscape-no-accessibility branch July 31, 2026 11:11
github-actions Bot added a commit to omarcresp/t3code-flake that referenced this pull request Jul 31, 2026
## What's Changed
* feat(cli): `npx t3 pair` - generate QR code from a running server by @t3dotgg in pingdotgg/t3code#4955
* fix(server): self-update no longer rolls itself back on restart by @t3dotgg in pingdotgg/t3code#5095
* fix(ci): rotate iPad showcase captures without Simulator UI scripting by @juliusmarminge in pingdotgg/t3code#5094
* feat(web): render terminals with libghostty-vt by @StiensWout in pingdotgg/t3code#4860
* refactor: move the canonical libghostty-vt vendor to the repository root by @StiensWout in pingdotgg/t3code#5102


**Full Changelog**: pingdotgg/t3code@v0.0.32-nightly.20260731.964...v0.0.32-nightly.20260731.965

Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.32-nightly.20260731.965
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant