-
-
Notifications
You must be signed in to change notification settings - Fork 6
feat: Add Resolution Search agent for map analysis #316
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
3fb1cef
60c2b1d
9c9c8ee
4823492
6595a9f
f1a8439
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,10 +8,11 @@ import { | |
| } from 'ai/rsc' | ||
| import { CoreMessage, ToolResultPart } from 'ai' | ||
| import { nanoid } from 'nanoid' | ||
| import type { FeatureCollection } from 'geojson' | ||
| import { Spinner } from '@/components/ui/spinner' | ||
| import { Section } from '@/components/section' | ||
| import { FollowupPanel } from '@/components/followup-panel' | ||
| import { inquire, researcher, taskManager, querySuggestor } from '@/lib/agents' | ||
| import { inquire, researcher, taskManager, querySuggestor, resolutionSearch } from '@/lib/agents' | ||
| // Removed import of useGeospatialToolMcp as it no longer exists and was incorrectly used here. | ||
| // The geospatialTool (if used by agents like researcher) now manages its own MCP client. | ||
| import { writer } from '@/lib/agents/writer' | ||
|
|
@@ -21,6 +22,7 @@ import { UserMessage } from '@/components/user-message' | |
| import { BotMessage } from '@/components/message' | ||
| import { SearchSection } from '@/components/search-section' | ||
| import SearchRelated from '@/components/search-related' | ||
| import { GeoJsonLayer } from '@/components/map/geojson-layer' | ||
| import { CopilotDisplay } from '@/components/copilot-display' | ||
| import RetrieveSection from '@/components/retrieve-section' | ||
| import { VideoSearchSection } from '@/components/video-search-section' | ||
|
|
@@ -39,6 +41,80 @@ async function submit(formData?: FormData, skip?: boolean) { | |
| const uiStream = createStreamableUI() | ||
| const isGenerating = createStreamableValue(true) | ||
| const isCollapsed = createStreamableValue(false) | ||
|
|
||
| const action = formData?.get('action') as string; | ||
| if (action === 'resolution_search') { | ||
| const file = formData?.get('file') as File; | ||
| if (!file) { | ||
| throw new Error('No file provided for resolution search.'); | ||
| } | ||
|
|
||
| const buffer = await file.arrayBuffer(); | ||
| const dataUrl = `data:${file.type};base64,${Buffer.from(buffer).toString('base64')}`; | ||
|
|
||
|
Comment on lines
+52
to
+54
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
SuggestionLock the action to Node.js by exporting the runtime: export const runtime = 'nodejs';Or switch to a runtime-agnostic conversion utility (e.g., using |
||
| // Get the current messages, excluding tool-related ones. | ||
| const messages: CoreMessage[] = [...(aiState.get().messages as any[])].filter( | ||
| message => | ||
| message.role !== 'tool' && | ||
| message.type !== 'followup' && | ||
| message.type !== 'related' && | ||
| message.type !== 'end' | ||
| ); | ||
|
Comment on lines
+56
to
+62
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🛠️ Refactor suggestion | 🟠 Major Replace The type assertion Since - const messages: CoreMessage[] = [...(aiState.get().messages as any[])].filter(
+ const messages: CoreMessage[] = (aiState.get().messages as AIMessage[])
+ .filter(
message =>
message.role !== 'tool' &&
message.type !== 'followup' &&
message.type !== 'related' &&
message.type !== 'end'
- );
+ )
+ .map(msg => ({
+ role: msg.role,
+ content: msg.content
+ })) as CoreMessage[];This approach maintains type safety while still filtering and mapping to the expected 🤖 Prompt for AI Agents
Comment on lines
+56
to
+62
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This filter includes custom assistant messages of type SuggestionExclude const messages: CoreMessage[] = (aiState.get().messages as CoreMessage[]).filter( Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this change. |
||
|
|
||
| // The user's prompt for this action is static. | ||
| const userInput = 'Analyze this map view.'; | ||
|
|
||
| // Construct the multimodal content for the user message. | ||
| const content: CoreMessage['content'] = [ | ||
| { type: 'text', text: userInput }, | ||
| { type: 'image', image: dataUrl, mimeType: file.type } | ||
| ]; | ||
|
|
||
| // Add the new user message to the AI state. | ||
| aiState.update({ | ||
| ...aiState.get(), | ||
| messages: [ | ||
| ...aiState.get().messages, | ||
| { id: nanoid(), role: 'user', content } | ||
| ] | ||
| }); | ||
| messages.push({ role: 'user', content }); | ||
|
Comment on lines
+74
to
+81
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Storing the raw, base64-encoded map image in AI state will bloat memory/storage and slow persistence. Since you already add a UI-only user message in the client (ResolutionSearch component), you don’t need to persist this heavy message server-side. SuggestionAvoid appending the image-bearing user message to aiState. Rely on the UI message added by the client and only pass the constructed message to the agent: // Remove the aiState.update(...) block here. Keep using Reply with "@CharlieHelps yes please" if you'd like me to add a commit that removes this update. |
||
|
|
||
| // Call the simplified agent, which now returns data directly. | ||
| const analysisResult = await resolutionSearch(messages); | ||
|
|
||
| // Create a streamable value for the summary and mark it as done. | ||
| const summaryStream = createStreamableValue<string>(); | ||
| summaryStream.done(analysisResult.summary || 'Analysis complete.'); | ||
|
|
||
| // Update the UI stream with the BotMessage component. | ||
| uiStream.update( | ||
| <BotMessage content={summaryStream.value} /> | ||
| ); | ||
|
|
||
| aiState.done({ | ||
| ...aiState.get(), | ||
| messages: [ | ||
| ...aiState.get().messages, | ||
| { | ||
| id: nanoid(), | ||
| role: 'assistant', | ||
| content: JSON.stringify(analysisResult), | ||
| type: 'resolution_search_result' | ||
| } | ||
| ] | ||
| }); | ||
|
|
||
| isGenerating.done(false); | ||
| uiStream.done(); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. uiStream is finalized inside the resolutionSearch agent via uiStream.done(...). Calling uiStream.done() again here risks double finalization and potential runtime errors. Let the agent own the UI stream lifecycle or remove this extra call. SuggestionRemove the extra finalization here and rely on the agent to close the stream: // isGenerating.done(false) Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this change. |
||
| return { | ||
| id: nanoid(), | ||
| isGenerating: isGenerating.value, | ||
| component: uiStream.value, | ||
| isCollapsed: isCollapsed.value | ||
| }; | ||
| } | ||
|
|
||
| const messages: CoreMessage[] = [...(aiState.get().messages as any[])].filter( | ||
| message => | ||
| message.role !== 'tool' && | ||
|
|
@@ -539,6 +615,26 @@ export const getUIStateFromAIState = (aiState: AIState): UIState => { | |
| </Section> | ||
| ) | ||
| } | ||
| case 'resolution_search_result': { | ||
| const analysisResult = JSON.parse(content as string); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You're parsing assistant content as JSON without any safety net. If prior messages or persisted history contain unexpected content, this will throw and break UI rendering. SuggestionGuard JSON.parse with try/catch and provide a graceful fallback UI: let analysisResult: any ) } } Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this change. |
||
| const summaryValue = createStreamableValue(); | ||
| summaryValue.done(analysisResult.summary); | ||
| const geoJson = analysisResult.geoJson as FeatureCollection; | ||
|
|
||
| return { | ||
| id, | ||
| component: ( | ||
| <> | ||
| <Section title="Map Analysis"> | ||
| <BotMessage content={summaryValue.value} /> | ||
| </Section> | ||
| {geoJson && ( | ||
| <GeoJsonLayer id={id} data={geoJson} /> | ||
| )} | ||
| </> | ||
| ) | ||
| } | ||
| } | ||
| } | ||
| break | ||
| case 'tool': | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| 'use client' | ||
|
|
||
| import React, { useState } from 'react' | ||
| import { useActions, useUIState } from 'ai/rsc' | ||
| import { AI } from '@/app/actions' | ||
| import { Button } from '@/components/ui/button' | ||
| import { Search } from 'lucide-react' | ||
| import { useMap } from './map/map-context' | ||
| import { nanoid } from 'nanoid' | ||
| import { UserMessage } from './user-message' | ||
| import { toast } from 'react-toastify' | ||
|
|
||
| export function AnalysisTool() { | ||
| const { map } = useMap() | ||
| const { submit } = useActions() | ||
| const [, setMessages] = useUIState<typeof AI>() | ||
| const [isAnalyzing, setIsAnalyzing] = useState(false) | ||
|
|
||
| const handleResolutionSearch = async () => { | ||
| if (!map) { | ||
| toast.error('Map is not available yet. Please wait for it to load.') | ||
| return | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| setIsAnalyzing(true) | ||
|
|
||
| try { | ||
| setMessages(currentMessages => [ | ||
| ...currentMessages, | ||
| { | ||
| id: nanoid(), | ||
| component: <UserMessage content={[{ type: 'text', text: 'Analyze this map view.' }]} /> | ||
| } | ||
| ]) | ||
|
|
||
| const canvas = map.getCanvas() | ||
| const blob = await new Promise<Blob | null>(resolve => { | ||
| canvas.toBlob(resolve, 'image/png') | ||
| }) | ||
|
|
||
| if (!blob) { | ||
| throw new Error('Failed to capture map image.') | ||
| } | ||
|
|
||
| const formData = new FormData() | ||
| formData.append('file', blob, 'map_capture.png') | ||
| formData.append('action', 'resolution_search') | ||
|
|
||
| const responseMessage = await submit(formData) | ||
| setMessages(currentMessages => [...currentMessages, responseMessage as any]) | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| } catch (error) { | ||
| console.error('Failed to perform resolution search:', error) | ||
| toast.error('An error occurred while analyzing the map.') | ||
| } finally { | ||
| setIsAnalyzing(false) | ||
| } | ||
| } | ||
|
|
||
| return ( | ||
| <div className="absolute top-4 right-4 z-20"> | ||
| <Button | ||
| variant="outline" | ||
| size="icon" | ||
| onClick={handleResolutionSearch} | ||
| disabled={isAnalyzing || !map} | ||
| title="Analyze current map view" | ||
| className="bg-background/80 backdrop-blur-sm hover:bg-background" | ||
| > | ||
| {isAnalyzing ? ( | ||
| <div className="h-5 w-5 animate-spin rounded-full border-b-2 border-current"></div> | ||
| ) : ( | ||
| <Search className="h-5 w-5" /> | ||
| )} | ||
| </Button> | ||
| </div> | ||
| ) | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| 'use client' | ||
|
|
||
| import { useEffect } from 'react' | ||
| import type mapboxgl from 'mapbox-gl' | ||
| import { useMap } from './map-context' | ||
| import type { FeatureCollection } from 'geojson' | ||
|
|
||
| interface GeoJsonLayerProps { | ||
| id: string; | ||
| data: FeatureCollection; | ||
| } | ||
|
|
||
| export function GeoJsonLayer({ id, data }: GeoJsonLayerProps) { | ||
| const { map } = useMap() | ||
|
|
||
| useEffect(() => { | ||
| if (!map || !data) return | ||
|
|
||
| const sourceId = `geojson-source-${id}` | ||
| const pointLayerId = `geojson-point-layer-${id}` | ||
| const polygonLayerId = `geojson-polygon-layer-${id}` | ||
| const polygonOutlineLayerId = `geojson-polygon-outline-layer-${id}` | ||
|
|
||
| const onMapLoad = () => { | ||
| // Add source if it doesn't exist | ||
| if (!map.getSource(sourceId)) { | ||
| map.addSource(sourceId, { | ||
| type: 'geojson', | ||
| data: data | ||
| }) | ||
| } else { | ||
| // If source exists, just update the data | ||
| const source = map.getSource(sourceId) as mapboxgl.GeoJSONSource; | ||
| source.setData(data); | ||
| } | ||
|
|
||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| // Add polygon layer for fill | ||
| if (!map.getLayer(polygonLayerId)) { | ||
| map.addLayer({ | ||
| id: polygonLayerId, | ||
| type: 'fill', | ||
| source: sourceId, | ||
| filter: ['==', '$type', 'Polygon'], | ||
| paint: { | ||
| 'fill-color': '#088', | ||
| 'fill-opacity': 0.4 | ||
| } | ||
| }) | ||
| } | ||
|
|
||
| // Add polygon layer for outline | ||
| if (!map.getLayer(polygonOutlineLayerId)) { | ||
| map.addLayer({ | ||
| id: polygonOutlineLayerId, | ||
| type: 'line', | ||
| source: sourceId, | ||
| filter: ['==', '$type', 'Polygon'], | ||
| paint: { | ||
| 'line-color': '#088', | ||
| 'line-width': 2 | ||
| } | ||
| }) | ||
| } | ||
|
|
||
| // Add point layer for circles | ||
| if (!map.getLayer(pointLayerId)) { | ||
| map.addLayer({ | ||
| id: pointLayerId, | ||
| type: 'circle', | ||
| source: sourceId, | ||
| filter: ['==', '$type', 'Point'], | ||
| paint: { | ||
| 'circle-radius': 6, | ||
| 'circle-color': '#B42222', | ||
| 'circle-stroke-width': 2, | ||
| 'circle-stroke-color': '#ffffff' | ||
| } | ||
| }) | ||
| } | ||
| } | ||
|
|
||
| if (map.isStyleLoaded()) { | ||
| onMapLoad() | ||
| } else { | ||
| map.on('load', onMapLoad) | ||
| } | ||
|
|
||
|
Comment on lines
+82
to
+87
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Listener cleanup is incomplete. You attach a SuggestionAdd if (map.isStyleLoaded()) {
onMapLoad();
} else {
map.on('load', onMapLoad);
}
return () => {
map.off('load', onMapLoad);
if (map.getLayer(pointLayerId)) map.removeLayer(pointLayerId);
if (map.getLayer(polygonLayerId)) map.removeLayer(polygonLayerId);
if (map.getLayer(polygonOutlineLayerId)) map.removeLayer(polygonOutlineLayerId);
if (map.getSource(sourceId)) map.removeSource(sourceId);
};Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this improvement. |
||
| // Cleanup function | ||
| return () => { | ||
| if (map.isStyleLoaded()) { | ||
| if (map.getLayer(pointLayerId)) map.removeLayer(pointLayerId) | ||
| if (map.getLayer(polygonLayerId)) map.removeLayer(polygonLayerId) | ||
| if (map.getLayer(polygonOutlineLayerId)) map.removeLayer(polygonOutlineLayerId) | ||
| if (map.getSource(sourceId)) map.removeSource(sourceId) | ||
| } | ||
| } | ||
|
Comment on lines
+85
to
+96
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The load event handler is registered but never removed in the cleanup. If the component rerenders/unmounts before style load, multiple handlers can accumulate and fire, wasting memory and work. SuggestionRemove the load listener in the cleanup to avoid leaks: // After attaching the handler // In the cleanup function Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this fix. |
||
| }, [map, id, data]) | ||
|
|
||
| return null // This component does not render any DOM elements itself | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Nitpick | 🔵 Trivial
Lock action to Node.js runtime.
Bufferis Node.js-specific and will fail if this action runs on the Edge runtime. Since this action processes file uploads and calls agents, it should explicitly lock to the Node.js runtime.Add this export at the top level of the file (after imports):
This ensures the action always runs in an environment where
Bufferis available. Based on past review comments.🤖 Prompt for AI Agents