Implement intent-driven object detection and iterative zoom#596
Implement intent-driven object detection and iterative zoom#596ngoiyaeric wants to merge 3 commits into
Conversation
- Fix user query passthrough in resolution search - Add server-side image cropping utility using sharp - Implement multi-pass iterative analysis loop in resolutionSearch agent - Enhance GeoJSON schema with category and confidence fields - Implement coordinate mapping from zoomed crops to absolute Lat/Lng - Provide incremental UI feedback for summary and Mapbox layers during passes Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
|
|
|
Warning Rate limit exceeded
To continue reviewing without waiting, purchase usage credits in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (4)
WalkthroughThis PR introduces iterative zoom-based image analysis for geographic feature detection. Map bounds are now collected from the client, passed through form submission to the server, and forwarded to the resolution-search agent, which executes up to three analysis passes while remapping crop-relative coordinates back to absolute geographic space and optionally requesting new image crops. ChangesZoom-driven analysis pipeline
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Review Summary by QodoImplement intent-driven iterative zoom detection with coordinate mapping
WalkthroughsDescription• Implement iterative zoom-based object detection with multi-pass analysis • Add server-side image cropping utility using sharp library • Implement coordinate mapping from zoomed crops to absolute Lat/Lng • Pass user query and map bounds through resolution search pipeline • Provide incremental UI feedback during analysis passes Diagramflowchart LR
A["User Input & Map Bounds"] --> B["Resolution Search Agent"]
B --> C["Initial Analysis Pass"]
C --> D{"Zoom Needed?"}
D -->|Yes| E["Extract Region with Sharp"]
E --> F["Zoom Pass Analysis"]
F --> G["Map Coordinates Back"]
D -->|No| H["Finalize Results"]
G --> H
H --> I["GeoJSON Features"]
I --> J["UI Update with Features"]
File Changes1. lib/utils/image-server-utils.ts
|
Code Review by Qodo
1.
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
| if (partialObject.geoJson) { | ||
| uiStream.update( | ||
| <GeoJsonLayer | ||
| id={groupeId} | ||
| data={partialObject.geoJson as FeatureCollection} | ||
| /> | ||
| ); |
There was a problem hiding this comment.
2. Geojson update blanks response 🐞 Bug ≡ Correctness
During streaming, processResolutionSearch() calls uiStream.update(<GeoJsonLayer .../>), which replaces the already-rendered response section with a component that returns null, making the chat response disappear.
Agent Prompt
### Issue description
`uiStream.update(<GeoJsonLayer .../>)` replaces the previously rendered `<Section>` (carousel + BotMessage) with a component that renders `null`, so users lose the visible response while streaming.
### Issue Context
`GeoJsonLayer` is meant to be a side-effect-only component (map overlay), not the message body.
### Fix Focus Areas
- app/actions.tsx[109-137]
- app/actions.tsx[223-233]
### Suggested fix
- Render `GeoJsonLayer` *inside* the same `<Section>` and call `uiStream.update()` with the entire `<Section>` whenever geoJson changes, e.g. a helper `renderResolutionUI({geoJson})`.
- Alternatively, keep a single UI tree and drive GeoJSON updates via a `StreamableValue<FeatureCollection>` prop (if you have a pattern for that), rather than replacing the entire stream with `GeoJsonLayer`.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| const finalObjectPromise = (async () => { | ||
| const generator = runIteration(); | ||
| let lastValue: any = {}; | ||
| for await (const value of generator) { | ||
| lastValue = value; | ||
| } | ||
| return lastValue; | ||
| })(); | ||
|
|
||
| return { | ||
| partialObjectStream: runIteration(), | ||
| object: finalObjectPromise | ||
| } as any; |
There was a problem hiding this comment.
3. Concurrent iteration double-runs 🐞 Bug ☼ Reliability
resolutionSearch() starts two runIteration() generators (one for partialObjectStream, one inside finalObjectPromise) that share mutable state (currentPass, isCompleteFlag, crop bounds), causing race conditions and duplicate/incorrect analysis passes.
Agent Prompt
### Issue description
The function returns two consumers of the same logical iterative process by calling `runIteration()` twice, while sharing mutable closure state. This can cause duplicated passes, skipped passes, inconsistent crop bounds, and mismatched `partialObjectStream` vs `object`.
### Issue Context
Callers (e.g. `app/actions.tsx`) expect `partialObjectStream` and `object` to refer to the *same* run.
### Fix Focus Areas
- lib/agents/resolution-search.tsx[160-173]
- lib/agents/resolution-search.tsx[174-343]
### Suggested fix
Implement a *single underlying* iteration and derive `object` from that same iteration without starting a second generator. One approach:
- Create an internal generator `iterate()`.
- Return `partialObjectStream` as a wrapper generator that:
- updates `lastValue` each yield
- yields through to the caller
- resolves a `finalObjectPromise` when iteration completes
- Return `{ partialObjectStream: wrapper(), object: finalObjectPromise }`.
This ensures only one set of model calls and no shared-state races.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| } else if (!isZoomPass && processedFeature.geometry.type === 'Point' && location?.bounds) { | ||
| const [relX, relY] = processedFeature.geometry.coordinates; | ||
| const latRange = location.bounds.ne.lat - location.bounds.sw.lat; | ||
| const lngRange = location.bounds.ne.lng - location.bounds.sw.lng; | ||
| const lat = location.bounds.ne.lat - (relY * latRange); | ||
| const lng = location.bounds.sw.lng + (relX * lngRange); | ||
| processedFeature.geometry.coordinates = [lng, lat]; | ||
| } |
There was a problem hiding this comment.
4. First-pass coordinates mis-mapped 🐞 Bug ≡ Correctness
When location.bounds exists, first-pass Point coordinates are treated as normalized [x,y] and mapped into lat/lng, but the system prompt only mandates normalized coordinates for zoom passes—so the model may output real lon/lat and get incorrectly remapped.
Agent Prompt
### Issue description
Non-zoom pass point mapping assumes model output is normalized (0–1), but the prompt doesn’t state that requirement for the first pass. If the model outputs GeoJSON with standard lon/lat, your mapping logic will corrupt coordinates.
### Issue Context
You already have a strong instruction for zoom passes, and you already have view bounds available; the contract just needs to be consistent.
### Fix Focus Areas
- lib/agents/resolution-search.tsx[178-226]
- lib/agents/resolution-search.tsx[263-287]
### Suggested fix
Pick one of:
1) **Normalized everywhere**: Update the system prompt to require all Point coordinates to be normalized `[x,y]` (0–1) relative to the current view for *all* passes, then always map using bounds when available.
2) **Detect and branch**: Only apply the bounds-mapping if coordinates look normalized (e.g., both in `[0,1]`). If they look like lon/lat (e.g., abs(x) > 1 or abs(y) > 1), keep as-is.
Also consider clamping normalized coords to `[0,1]` before mapping to avoid out-of-bounds values.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/actions.tsx`:
- Around line 125-127: The branch checking fullSummary for the misspelled
literal 'ZOOM PASSContext' will never match (missing space) and relying on
AI-generated text is brittle; instead add and use an explicit zoom-pass boolean
flag (e.g., isZoomPass or zoomInProgress) that is set when extractRegion is
invoked by the agent and cleared when the zoom pass completes, then change the
code path that currently inspects fullSummary (the block calling
summaryStream.update and referencing fullSummary) to consult that flag and call
summaryStream.update(`${fullSummary}\n\n[Zoom Pass in progress...]`) only when
the flag is true.
In `@components/followup-panel.tsx`:
- Around line 33-36: The code is trying to read a non-existent
cameraState.bounds and append it to formData, which is dead; remove the whole
bounds branch (the mapData.cameraState?.bounds check and the
formData.append('bounds', ...) call) from components/followup-panel.tsx, or if
bounds is intentionally needed instead update the CameraState type to include
bounds and ensure mapData.cameraState actually supplies it before calling
formData.append — reference mapData, cameraState, bounds, and the
formData.append('bounds', ...) call when making the change.
In `@components/header-search-button.tsx`:
- Around line 158-165: The code only captures bounds when mapProvider ===
'mapbox'; update the branch that builds formData so it also captures bounds for
mapProvider === 'google' by calling map.getBounds() (or equivalent) and
appending the same JSON structure via formData.append('bounds',
JSON.stringify({...})). Ensure you use the Google Maps LatLngBounds methods
(getSouthWest/getNorthEast or toJSON) to extract lat/lng into the sw/ne shape,
and keep the existing variable names (mapProvider, map, bounds, formData.append)
so resolution-search.tsx receives location.bounds for the zoom-pass coordinate
remapping.
In `@lib/agents/resolution-search.tsx`:
- Around line 330-343: The problem is that runIteration() is called twice
creating two independent generator runs; fix by instantiating a single generator
instance (const generator = runIteration()) and use that same instance for both
partialObjectStream and the final object promise: set partialObjectStream to an
async generator that yields from that single generator instance and make
finalObjectPromise iterate the same generator to capture the last yielded value
(ensuring the stream is exhausted before resolving). Reference runIteration,
partialObjectStream, and finalObjectPromise/object in your changes.
- Around line 280-287: The coordinate remapping inside the branch that checks
!isZoomPass is wrong because it assumes AI returned normalized 0–1 image coords;
update the logic so pass-1 trusts real-world coordinates instead of remapping:
in the block referencing isZoomPass, processedFeature.geometry and
location.bounds (the branch starting with "else if (!isZoomPass &&
processedFeature.geometry.type === 'Point' && location?.bounds)"), remove the
transformation that treats coordinates as relative (the relX/relY ->
latRange/lngRange -> computed lat,lng mapping) and instead assign
processedFeature.geometry.coordinates directly from the AI-provided [lng, lat];
alternatively, if you prefer normalized inputs, add the explicit normalized
(0-1) requirement to the pass-1 system prompt so the existing remap code is
valid—choose one consistent approach and keep the isZoomPass handling consistent
with that choice.
- Around line 151-158: The code calls .find() on (lastUserMessage?.content as
any[]) which can be undefined at runtime, causing a TypeError before your
explicit image check; fix by guarding access to content before calling .find():
ensure lastUserMessage?.content is an array (e.g., test
Array.isArray(lastUserMessage?.content) or use optional chaining like
lastUserMessage?.content?.find) and only call .find() when content is present,
updating the variables originalImagePart, originalImageData and
originalImageMimeType accordingly so the existing throw new Error("Resolution
search requires an image.") runs as intended.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 7b29b030-ae9e-42be-850f-075feeb77a3d
⛔ Files ignored due to path filters (1)
bun.lockis excluded by!**/*.lock
📒 Files selected for processing (7)
app/actions.tsxcomponents/chat.tsxcomponents/followup-panel.tsxcomponents/header-search-button.tsxlib/agents/resolution-search.tsxlib/utils/image-server-utils.tspackage.json
📜 Review details
🔇 Additional comments (3)
lib/utils/image-server-utils.ts (1)
9-51: LGTM —extractRegionis correctly implemented.The clamping logic is sound, the sharp pipeline is correctly sequenced (metadata read before extract/resize), and the base64 prefix strip handles all standard image MIME types.
package.json (1)
93-93: LGTM —sharp@^0.34.5is the current stable release with no known vulnerabilities.components/chat.tsx (1)
141-141: LGTM —HeaderSearchButtoncorrectly receives the controlledinput/setInputprops in both the mobile and desktop branches.
| if (fullSummary.includes('ZOOM PASSContext') || fullSummary.includes('zoomed-in CROP')) { | ||
| summaryStream.update(`${fullSummary}\n\n[Zoom Pass in progress...]`); | ||
| } |
There was a problem hiding this comment.
'ZOOM PASSContext' is missing a space — this branch can never be entered.
The string 'ZOOM PASSContext' has no space between PASS and Context. The system prompt uses **ZOOM PASS Context:** (with a space), but even so, the AI's summary output is unlikely to reproduce that system-prompt phrase verbatim. The [Zoom Pass in progress...] indicator therefore never appears in the streamed summary.
🐛 Proposed fix
- if (fullSummary.includes('ZOOM PASSContext') || fullSummary.includes('zoomed-in CROP')) {
+ if (fullSummary.includes('ZOOM PASS') || fullSummary.includes('zoomed-in crop')) {
summaryStream.update(`${fullSummary}\n\n[Zoom Pass in progress...]`);
}A more reliable alternative is to track zoom-pass state in a flag set when extractRegion is called by the agent, rather than checking for keywords in the AI-generated text.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (fullSummary.includes('ZOOM PASSContext') || fullSummary.includes('zoomed-in CROP')) { | |
| summaryStream.update(`${fullSummary}\n\n[Zoom Pass in progress...]`); | |
| } | |
| if (fullSummary.includes('ZOOM PASS') || fullSummary.includes('zoomed-in crop')) { | |
| summaryStream.update(`${fullSummary}\n\n[Zoom Pass in progress...]`); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@app/actions.tsx` around lines 125 - 127, The branch checking fullSummary for
the misspelled literal 'ZOOM PASSContext' will never match (missing space) and
relying on AI-generated text is brittle; instead add and use an explicit
zoom-pass boolean flag (e.g., isZoomPass or zoomInProgress) that is set when
extractRegion is invoked by the agent and cleared when the zoom pass completes,
then change the code path that currently inspects fullSummary (the block calling
summaryStream.update and referencing fullSummary) to consult that flag and call
summaryStream.update(`${fullSummary}\n\n[Zoom Pass in progress...]`) only when
the flag is true.
| const bounds = mapProvider === 'mapbox' && map ? map.getBounds() : null; | ||
| if (bounds) { | ||
| formData.append('bounds', JSON.stringify({ | ||
| sw: { lat: bounds.getSouthWest().lat, lng: bounds.getSouthWest().lng }, | ||
| ne: { lat: bounds.getNorthEast().lat, lng: bounds.getNorthEast().lng } | ||
| })) | ||
| } | ||
| } |
There was a problem hiding this comment.
Bounds are not captured for the Google Maps provider.
For mapProvider === 'google', no bounds field is appended to formData. As a result, location.bounds is always undefined in resolution-search.tsx for Google Maps users. The zoom-pass coordinate remapping (lines 271–286 of resolution-search.tsx) requires location.bounds to map detected features back to lat/lng; without it, coordinates fall back to raw normalized (0–1) image-space values, which are not valid GeoJSON coordinates.
🛠️ Suggested fix — capture bounds for Google Maps provider
const center = mapProvider === 'mapbox' && map ? map.getCenter() : mapData.cameraState?.center;
if (center) {
formData.append('latitude', center.lat.toString())
formData.append('longitude', center.lng.toString())
const bounds = mapProvider === 'mapbox' && map ? map.getBounds() : null;
if (bounds) {
formData.append('bounds', JSON.stringify({
sw: { lat: bounds.getSouthWest().lat, lng: bounds.getSouthWest().lng },
ne: { lat: bounds.getNorthEast().lat, lng: bounds.getNorthEast().lng }
}))
+ } else if (mapData.cameraState?.bounds) {
+ formData.append('bounds', JSON.stringify(mapData.cameraState.bounds))
}
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@components/header-search-button.tsx` around lines 158 - 165, The code only
captures bounds when mapProvider === 'mapbox'; update the branch that builds
formData so it also captures bounds for mapProvider === 'google' by calling
map.getBounds() (or equivalent) and appending the same JSON structure via
formData.append('bounds', JSON.stringify({...})). Ensure you use the Google Maps
LatLngBounds methods (getSouthWest/getNorthEast or toJSON) to extract lat/lng
into the sw/ne shape, and keep the existing variable names (mapProvider, map,
bounds, formData.append) so resolution-search.tsx receives location.bounds for
the zoom-pass coordinate remapping.
| const lastUserMessage = [...messages].reverse().find(m => m.role === 'user'); | ||
| const originalImagePart = (lastUserMessage?.content as any[]).find(p => p.type === 'image'); | ||
| const originalImageData = originalImagePart?.image; | ||
| const originalImageMimeType = originalImagePart?.mimeType; | ||
|
|
||
| if (!originalImageData) { | ||
| throw new Error("Resolution search requires an image."); | ||
| } |
There was a problem hiding this comment.
TypeError crash if no user message is found — optional chain doesn't guard against calling .find() on undefined.
On line 152, lastUserMessage?.content evaluates to undefined when no user message exists. The subsequent (undefined as any[]).find(...) is not guarded by optional chaining — as any[] is a TypeScript-only cast with no runtime effect — so calling .find() throws TypeError before the explicit guard at line 156–158 is ever reached. The error message "Resolution search requires an image." would therefore never be seen.
🛡️ Proposed fix
- const originalImagePart = (lastUserMessage?.content as any[]).find(p => p.type === 'image');
+ const originalImagePart = Array.isArray(lastUserMessage?.content)
+ ? (lastUserMessage!.content as any[]).find(p => p.type === 'image')
+ : undefined;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@lib/agents/resolution-search.tsx` around lines 151 - 158, The code calls
.find() on (lastUserMessage?.content as any[]) which can be undefined at
runtime, causing a TypeError before your explicit image check; fix by guarding
access to content before calling .find(): ensure lastUserMessage?.content is an
array (e.g., test Array.isArray(lastUserMessage?.content) or use optional
chaining like lastUserMessage?.content?.find) and only call .find() when content
is present, updating the variables originalImagePart, originalImageData and
originalImageMimeType accordingly so the existing throw new Error("Resolution
search requires an image.") runs as intended.
| } else if (!isZoomPass && processedFeature.geometry.type === 'Point' && location?.bounds) { | ||
| const [relX, relY] = processedFeature.geometry.coordinates; | ||
| const latRange = location.bounds.ne.lat - location.bounds.sw.lat; | ||
| const lngRange = location.bounds.ne.lng - location.bounds.sw.lng; | ||
| const lat = location.bounds.ne.lat - (relY * latRange); | ||
| const lng = location.bounds.sw.lng + (relX * lngRange); | ||
| processedFeature.geometry.coordinates = [lng, lat]; | ||
| } |
There was a problem hiding this comment.
Pass-1 coordinate remapping assumes the AI returns normalized (0–1) image coordinates, but the prompt never asks for this.
For zoom passes (line 266), the system prompt explicitly instructs: "All coordinates you provide MUST be relative to THIS CROP (0-1)." For pass 1 (line 280), there is no equivalent instruction. An LLM given real-world lat/lng context will almost certainly return actual geographic coordinates (e.g., [-122.5, 37.8]). Treating those values as normalized fractions and multiplying by the lat/lng range produces completely wrong GeoJSON coordinates:
lat = ne.lat - (37.8 * latRange)wherelatRange ≈ 0.1→ wildly out of boundslng = sw.lng + (-122.5 * lngRange)→ same problem
Either add the same normalized-coordinate instruction to the pass-1 system prompt, or remove the remapping for pass 1 and trust the AI to return real lat/lng (which can be used directly in GeoJSON without any mapping).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@lib/agents/resolution-search.tsx` around lines 280 - 287, The coordinate
remapping inside the branch that checks !isZoomPass is wrong because it assumes
AI returned normalized 0–1 image coords; update the logic so pass-1 trusts
real-world coordinates instead of remapping: in the block referencing
isZoomPass, processedFeature.geometry and location.bounds (the branch starting
with "else if (!isZoomPass && processedFeature.geometry.type === 'Point' &&
location?.bounds)"), remove the transformation that treats coordinates as
relative (the relX/relY -> latRange/lngRange -> computed lat,lng mapping) and
instead assign processedFeature.geometry.coordinates directly from the
AI-provided [lng, lat]; alternatively, if you prefer normalized inputs, add the
explicit normalized (0-1) requirement to the pass-1 system prompt so the
existing remap code is valid—choose one consistent approach and keep the
isZoomPass handling consistent with that choice.
| const finalObjectPromise = (async () => { | ||
| const generator = runIteration(); | ||
| let lastValue: any = {}; | ||
| for await (const value of generator) { | ||
| lastValue = value; | ||
| } | ||
| return lastValue; | ||
| })(); | ||
|
|
||
| return { | ||
| partialObjectStream: runIteration(), | ||
| object: finalObjectPromise | ||
| } as any; | ||
| } |
There was a problem hiding this comment.
runIteration() is invoked twice — double API calls, inconsistent results between partialObjectStream and object.
runIteration is an async function* generator. Every call creates a brand-new, independent execution:
const finalObjectPromise = (async () => {
const generator = runIteration(); // ← 1st full analysis run starts here
…
})();
return {
partialObjectStream: runIteration(), // ← 2nd full analysis run, completely independent
object: finalObjectPromise
}app/actions.tsx consumes both — the streaming loop reads from the 2nd generator, and await streamResult.object resolves from the 1st. Because these are separate executions, every AI call (streamObject) and every extractRegion crop is performed twice, doubling LLM costs and Tavily calls. Worse, the two runs can produce different results, so the geoJson rendered during streaming may not match the analysisResult used to persist the final state.
Fix: Run the generator once, yield values to consumers, and resolve the final promise from the last yielded value.
🐛 Proposed fix
- const finalObjectPromise = (async () => {
- const generator = runIteration();
- let lastValue: any = {};
- for await (const value of generator) {
- lastValue = value;
- }
- return lastValue;
- })();
-
- return {
- partialObjectStream: runIteration(),
- object: finalObjectPromise
- } as any;
+ let resolveObject!: (v: any) => void;
+ const objectPromise = new Promise<any>(res => { resolveObject = res; });
+
+ async function* singlePassStream() {
+ let lastValue: any = {};
+ for await (const value of runIteration()) {
+ lastValue = value;
+ yield value;
+ }
+ resolveObject(lastValue);
+ }
+
+ return {
+ partialObjectStream: singlePassStream(),
+ object: objectPromise
+ } as any;Note:
app/actions.tsxmust consumepartialObjectStreamfully beforeobjectPromiseresolves (i.e. thefor awaitloop must complete). The current code inprocessResolutionSearchalready does this — it exhausts the stream before callingawait streamResult.object.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@lib/agents/resolution-search.tsx` around lines 330 - 343, The problem is that
runIteration() is called twice creating two independent generator runs; fix by
instantiating a single generator instance (const generator = runIteration()) and
use that same instance for both partialObjectStream and the final object
promise: set partialObjectStream to an async generator that yields from that
single generator instance and make finalObjectPromise iterate the same generator
to capture the last yielded value (ensuring the stream is exhausted before
resolving). Reference runIteration, partialObjectStream, and
finalObjectPromise/object in your changes.
- Add bounds to CameraState interface in MapDataContext - Populate bounds in mapbox-map.tsx to support coordinate mapping - Fix type error in followup-panel.tsx Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
- Fix user query passthrough in resolution search - Add server-side image cropping utility using sharp - Implement multi-pass iterative analysis loop in resolutionSearch agent - Enhance GeoJSON schema with category and confidence fields - Implement coordinate mapping from zoomed crops to absolute Lat/Lng - Provide incremental UI feedback for summary and Mapbox layers - Fix CameraState types and populate map bounds for accurate mapping - Ensure production build passes with full type safety Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
2 participants
This PR implements a multi-pass iterative analysis loop for the resolution search feature. It fixes the user query passthrough, adds server-side image cropping using the 'sharp' library, and introduces coordinate mapping to transform detected objects from zoomed crops back to real-world coordinates. The UI is updated to provide incremental feedback during the analysis passes.
PR created automatically by Jules for task 2601202592126715211 started by @ngoiyaeric
Summary by CodeRabbit
New Features
Improvements