Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

Large diffs are not rendered by default.

89 changes: 89 additions & 0 deletions .cursor/plans/homepage_scenario_workflow_dialogs_20260216.plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Homepage Scenario Workflow Dialogs

## Summary
Implement a shared workflow-drawer system on the homepage immersive workstreams so each scenario has exactly 3 right-click actions, and each action opens a relevant multi-step workflow dialog.

Locked choices:
- GitHub submission is real only for software "Report this bug" GitHub path.
- All other workflow submissions are demo-only.
- Keep software code editor visual, but keep menu actions fixed.
- Use heuristic AI-style title generation (no model call).

## Assumptions and Defaults
1. Use right-side panel UI for all workflows.
2. Every workflow shows Anyclick context: target selector, container selector, container screenshot, page URL/title.
3. Screenshot failures are non-blocking and show fallback.
4. GitHub failures/non-configured state offer "Complete as demo" fallback.
5. Changes remain in `apps/web` only (no `packages/*` API changes).

## Public API / Interface / Type Changes
1. Add `POST /api/feedback/github` at `apps/web/src/app/api/feedback/github/route.ts`.
2. Endpoint body is `AnyclickPayload`; supports:
- `metadata.github.title` as preferred GitHub issue title.
- `metadata.routing.adapter = "github"`.
3. Internal immersive changes:
- Remove menu definitions from theme ownership.
- Add workflow types under `apps/web/src/components/immersive-workstream/workflows/types.ts`.

## Workflow Catalog
- Software Development
- Report this bug: Ticket system -> Issue details -> Review & submit (GitHub real, Jira demo)
- Send to Cursor: Fix scope -> Prompt & constraints -> Review (demo)
- Copy selector: Selector format -> Copy bundle (demo)
- E-commerce & Logistics
- Item missing: Incident details -> Resolution path -> Confirm (demo)
- Shipping issue: Shipment details -> Customer impact -> Confirm (demo)
- Escalate order: Escalation target -> Priority/SLA -> Confirm (demo)
- Healthcare
- Check-in issue: Issue type -> Patient context -> Confirm (demo)
- Vital alert: Vital entry -> Clinical action -> Confirm (demo)
- Flag urgent: Urgency classification -> Notification route -> Confirm (demo)
- Social Media
- Save asset: Destination -> Metadata/tags -> Confirm (demo)
- Flag content: Policy category -> Enforcement action -> Confirm (demo)
- Quick reply: Draft reply -> Tone/approval -> Confirm (demo)

## Architecture Decisions
1. Use `ContextMenuItem.onClick` for all 12 actions; bypass default submit flow.
2. Build shared workflow launcher + drawer; action definitions live in registry.
3. Keep software editor visible and add fixed-workflow indicator.
4. Capture context with Anyclick core utilities:
- `buildElementContext`
- `captureScreenshot(..., "container")`
5. Isolate GitHub submit via dedicated `/api/feedback/github` endpoint.

## Implementation Steps
1. Add workflow foundation files:
- `workflows/types.ts`
- `workflows/registry.ts`
- `workflows/titleHeuristics.ts`
- `workflows/contextCapture.ts`
2. Add shared launcher + UI:
- `workflows/WorkflowDrawer.tsx`
- `workflows/useWorkflowLauncher.tsx`
- `workflows/WorkflowFlowRenderer.tsx`
3. Integrate sections:
- `sections/SoftwareDevelopmentSection.tsx`
- `sections/DefaultWorkstreamSection.tsx`
- `sections/HealthcareSection.tsx`
4. Lock software editor behavior and copy:
- `cards/SoftwareEditorCard.tsx`
- `softwareCode.ts`
5. Move menu ownership out of theme model:
- `types.ts`
- `themes.tsx`
6. Add GitHub-only endpoint:
- `app/api/feedback/github/route.ts`
7. Wire software GitHub submit path in workflow renderer.

## Validation
1. Each scenario shows exactly 3 right-click menu items.
2. All 12 actions open relevant dialogs with multi-step forms.
3. Context panel shows selector/container selector/screenshot or fallback.
4. Software bug flow first step has Jira + GitHub choices.
5. GitHub title prefilled heuristically; editable/regeneratable.
6. GitHub submit succeeds when configured.
7. GitHub failure offers demo completion.
8. All non-software flows complete without backend dependency.
9. Escape/close behavior works and body scroll restores.
10. Run `yarn --cwd /Users/ericjohnson/Desktop/projects/anyclick/apps/web lint`.
158 changes: 158 additions & 0 deletions .cursor/plans/swift_native_anyclick_implementation_69a4d5c4.plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
---
name: Swift Native AnyClick Implementation
overview: Create a native macOS application using Swift that intercepts right-clicks, resolves context (including deep browser context via an extension bridge), and displays a dynamic, graph-based native menu.
todos:
- id: scaffold-swift-project
content: Scaffold Swift Project in `apps/anyclick-native`
status: pending
- id: implement-input-monitor
content: Implement InputMonitor and MenuRenderer (Proof of Concept)
status: pending
dependencies:
- scaffold-swift-project
- id: implement-graph-engine
content: Design and Implement Graph Data Structure in Swift
status: pending
dependencies:
- scaffold-swift-project
- id: update-extension-manifest
content: Update Extension Manifest for Native Messaging
status: pending
- id: implement-browser-bridge
content: Implement Browser Bridge in Swift and Extension
status: pending
dependencies:
- update-extension-manifest
- scaffold-swift-project
isProject: false
---

# AnyClick Native macOS Implementation Plan

This plan outlines the creation of a high-performance native macOS application that replaces/augments the web-based AnyClick flows. It uses system-level hooks for event tracking and a bridge to the browser for DOM interactions.

## Architecture

```mermaid
graph TD
User((User)) -->|Right Click| InputMonitor[Input Monitor (CGEventTap)]
InputMonitor -- Suppress OS Menu --> ContextEngine[Context Engine]

subgraph Native App [Swift macOS App]
InputMonitor
ContextEngine
GraphEngine[Graph Engine]
MenuRenderer[Menu Renderer]
ActionExecutor[Action Executor]
end

subgraph Browser [Chrome/Arc/Safari]
Extension[AnyClick Extension]
DOM[DOM Context]
end

ContextEngine -->|Get Active App| OS[macOS Accessibility API]
ContextEngine -.->|Native Messaging| Extension

Extension -->|DOM Data| ContextEngine

ContextEngine -->|Context + AppID| GraphEngine
GraphEngine -->|Menu Items| MenuRenderer
MenuRenderer -->|Show| User

User -->|Select Action| ActionExecutor
ActionExecutor -->|Perform| OS
ActionExecutor -->|Execute Script| Extension
```



## Core Components

### 1. Native Application Structure (`apps/anyclick-native`)

- **Technology**: Swift 6, AppKit (for low-level events/menus), SwiftUI (for Settings UI).
- **Permissions**:
- `Accessibility`: To read selected text and UI elements.
- `Input Monitoring`: To intercept global mouse clicks.
- `Native Messaging`: To communicate with the browser extension.

### 2. The Graph Engine

A JSON-based configuration engine that resolves menu items based on `(Application, Context)`.

**Schema Definition:**

```json
{
"rules": [
{
"appId": "com.microsoft.VSCode",
"context": { "hasSelection": true },
"actions": ["copy", "save-to-context", "ask-ai"]
},
{
"appId": "com.google.Chrome",
"context": { "domType": "image" },
"actions": ["save-image", "upload-thing"]
}
]
}
```

### 3. Context & Event Loop

1. **Hook**: `CGEvent.tapCreate` listens for right-mouse-down.
2. **Filter**: If a modifier key (e.g., `Cmd`) is held (optional config), passthrough to OS. Otherwise, suppress.
3. **Resolve**:
- Get `NSWorkspace.shared.frontmostApplication`.
- Use `AXUIElement` to get selected text.
- If Browser, send "Ping" to Extension via Native Messaging to get DOM element under cursor.

### 4. Extension Bridge

- **Update** `packages/anyclick-extension` to support Native Messaging.
- **Protocol**:
- `Request: { type: "GET_CONTEXT", x: 100, y: 200 }`
- `Response: { element: "img", src: "...", id: "..." }`
- `Command: { type: "EXECUTE", action: "remove-element", targetId: "..." }`

## Implementation Steps

### Phase 1: Native Foundation

- Initialize Xcode project in `apps/anyclick-native`.
- Implement `InputMonitor` class using `CoreGraphics`.
- Create `PermissionManager` to request/check Accessibility/Input permissions.
- Implement basic `MenuRenderer` using `NSMenu`.

### Phase 2: Graph & Context

- Define `GraphConfig` structs and load default rules.
- Implement `ContextResolver` (getting frontmost app bundle ID and selected text).
- Connect `InputMonitor` -> `ContextResolver` -> `Graph` -> `Menu`.

### Phase 3: Browser Integration

- Add `nativeMessaging` permission to `packages/anyclick-extension/manifest.json`.
- Implement Native Messaging Host registration script.
- Create `BrowserBridge` in Swift to handle stdio communication with Chrome.
- Update Extension content script to answer "Context Requests".

## File Structure

- `apps/anyclick-native/`
- `Sources/`
- `Core/` (Graph, EventLoop)
- `Mac/` (Accessibility, InputTap)
- `Bridge/` (NativeMessaging)
- `UI/` (Settings, Menu)
- `packages/anyclick-extension/src/native/` (New bridge logic)

## Key Challenges & Mitigations

- **Speed**: Waiting for Chrome extension response might delay menu.
- *Mitigation*: Show menu immediately with "Loading..." or generic actions, update when response arrives (or strict 50ms timeout).
- **Sandboxing**: App Store distribution is hard with Accessibility/NativeMessaging.
- *Mitigation*: Distribute as signed Developer ID app (outside App Store).

83 changes: 83 additions & 0 deletions .cursor/plans/swiftui-anyclick-macos_ff4c2c5e.plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
name: swiftui-anyclick-macos
overview: Assess feasibility and outline a staged implementation of a SwiftUI macOS Anyclick menu that is driven by a local graph, integrated app‑specifically with Chrome and Cursor, and reuses existing Anyclick types and context capture patterns.
todos:
- id: define-graph-model
content: Define menu graph schema + local persistence + JSON import/export
status: pending
- id: native-menu
content: Build SwiftUI/NSMenu renderer + action dispatch
status: pending
- id: chrome-bridge
content: Send Chrome right-click context to Mac app
status: pending
- id: cursor-bridge
content: Send Cursor right-click context to Mac app
status: pending
- id: action-adapters
content: Implement Anyclick/Cursor/3rd-party action adapters
status: pending
isProject: false
---

# SwiftUI Anyclick Menu Plan

## Feasibility Notes

- This is feasible with app‑specific integrations. The repo already has menu modeling (`ContextMenuItem`) and extension‑side right‑click interception that can be reused as the semantic model and context capture pattern for a native SwiftUI menu.
- A global system‑wide menu would require Accessibility/event taps; with your app‑specific choice we can avoid those and integrate per app (Chrome extension + Cursor plugin) while still presenting a native macOS menu.

## Key Existing References (reuse/adapt)

- Menu item modeling: `ContextMenuItem` in `packages/anyclick-react/src/types.ts` and `ExtensionMenuItem` in `packages/anyclick-extension/src/contextMenu.ts`.
- Right‑click interception in Chrome extension: `packages/anyclick-extension/src/content.ts`.
- Intent/protocol semantics to map actions: `packages/anyclick-protocol/src/spec.ts`.

## Plan

1. **Define the local menu graph & action schema**
- Create a Swift model for a graph that maps `(AppContext, SelectionContext, PageContext)` → ordered menu items + actions.
- Use a local‑only store (SwiftData/CoreData or SQLite) plus JSON import/export for easy editing and future sync.
- Add action “targets” for: Anyclick web chat/context, Cursor agent, and third‑party APIs.
2. **Build the SwiftUI macOS app shell + native menu renderer**
- Implement a background/status‑bar app that can:
- Receive a context event, resolve the menu graph, and render a native `NSMenu` at the cursor location.
- Execute action handlers (copy to clipboard, send to Anyclick web endpoints, invoke Cursor local adapter, call 3rd‑party API).
- Add a simple UI to edit graph rules and reorder menu items.
3. **Chrome integration (app‑specific)**
- Extend `packages/anyclick-extension` to send a right‑click context payload (selection text, URL, element metadata) to the Mac app.
- Use either Native Messaging or a local HTTP bridge to deliver events to the Swift app.
- The extension defers menu rendering to the Swift app (native menu), but still uses existing capture logic for context consistency.
4. **Cursor integration (app‑specific)**
- Create a small Cursor extension/plugin that captures selection + editor context on right‑click and forwards it to the Swift app.
- Reuse `@ewjdev/anyclick-cursor` / `@ewjdev/anyclick-cursor-local` conventions for action execution where appropriate.
5. **Action execution + telemetry (local‑only MVP)**
- Implement action adapters in the macOS app:
- **Anyclick web chat/context**: call configured endpoints with selection + page context.
- **Cursor agent**: use local adapter protocol (or CLI) for “add to chat” actions.
- **Third‑party**: generic HTTP webhook action.
- Capture local logs for debugging; no cloud sync in MVP.

```mermaid
flowchart LR
ChromeExt --> LocalBridge
CursorExt --> LocalBridge
LocalBridge --> MacApp
MacApp --> MenuGraph
MenuGraph --> NativeMenu
NativeMenu --> ActionAdapters
ActionAdapters --> AnyclickWeb
ActionAdapters --> CursorAgent
ActionAdapters --> ThirdPartyAPI
```



## Initial Implementation Todos

- `define-graph-model`: Swift models + local persistence + JSON import/export.
- `native-menu`: SwiftUI/NSMenu rendering and action dispatch.
- `chrome-bridge`: Extension event payload + local bridge to Mac app.
- `cursor-bridge`: Cursor extension + local bridge to Mac app.
- `action-adapters`: Anyclick web, Cursor agent, third‑party webhooks.

3 changes: 3 additions & 0 deletions .cursor/worktrees.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"setup-worktree": ["yarn install"]
}
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,5 @@ yarn-error.log*
# turbo
.turbo
.vercel

.myh/
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Right-click any element in your app to capture feedback with full DOM context, s
### Installation

```bash
yarn install @ewjdev/anyclick-react @ewjdev/anyclick-github
yarn install @ewjdev/anyclick-core @ewjdev/anyclick-react @ewjdev/anyclick-github
```

### Usage
Expand Down Expand Up @@ -54,7 +54,7 @@ export async function POST(request: Request) {
"use client";

import { FeedbackProvider } from "@ewjdev/anyclick-react";
import { createHttpAdapter } from "@ewjdev/anyclick-github";
import { createHttpAdapter } from "@ewjdev/anyclick-core";

const adapter = createHttpAdapter({ endpoint: "/api/feedback" });

Expand Down
1 change: 1 addition & 0 deletions apps/web/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
"@ewjdev/anyclick-github": "*",
"@ewjdev/anyclick-jira": "*",
"@ewjdev/anyclick-react": "*",
"@ewjdev/anyclick-react-tailwind": "*",
"@ewjdev/anyclick-uploadthing": "*",
"@headlessui/react": "^2.2.8",
"@heroicons/react": "^2.1.3",
Expand Down
1 change: 1 addition & 0 deletions apps/web/src/app/api/feedback/github/route.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export declare function POST(req: Request): Promise<Response>;
Loading