Refactor Inquiry Agent for Deep Conjecture Exploration#465
Conversation
…tion - Updated Task Manager in `lib/agents/task-manager.tsx` to prioritize proceeding by default and only inquiring for deep edge cases. - Implemented robust system prompt for Inquiry Agent in `lib/agents/inquire.tsx` to focus on extending user conjectures and identifying non-obvious factors. - Verified that existing flow in `app/actions.tsx` correctly supports these behavior changes. Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
WalkthroughSystem prompts were updated in two AI agent files. The Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
ⓘ Your approaching your monthly quota for Qodo. Upgrade your plan PR Compliance Guide 🔍Below is a summary of compliance checks for this PR:
Compliance status legend🟢 - Fully Compliant🟡 - Partial Compliant 🔴 - Not Compliant ⚪ - Requires Further Human Verification 🏷️ - Compliance label |
|||||||||||||||||||||||
![charliecreates[bot]](https://avatars.githubusercontent.com/in/1140780?s=60&v=4){kind=small}
There was a problem hiding this comment.
The updated prompts better match the PR intent, but the new "proceed by default" guidance can cause incorrect behavior for inherently location-relative queries (e.g., nearest / my area) when no location context exists. The Inquiry Agent prompt is not explicitly schema-aligned, which can reduce determinism when using streamObject({ schema }). Finally, the stronger “deep” framing may bias agents toward out-of-context speculation unless constrained to domain-relevant, outcome-changing inquiries.
Additional notes (1)
- Maintainability |
lib/agents/task-manager.tsx:20-43
Both prompts introduce strong domain-specific phrasing ("planet computer", "hidden socioeconomic drivers") that may steer the model toward speculative framing even when the user’s query is simple or non-socioeconomic. That increases the risk of producing forced depth (irrelevant edge cases) and can reintroduce the original issue in a different form (questions that feel out-of-context).
This is especially risky for the Task Manager: it decides whether to interrupt the flow at all. Over-emphasizing “bold” conjecture extension can bias it toward unnecessary inquiry on straightforward tasks.
Summary of changes
What changed
lib/agents/inquire.tsx
- Replaced the placeholder
system: �...prompt with a concrete Deep Inquiry Agent system prompt. - The new prompt instructs the model to avoid basic clarification questions and instead provide 2�1 concise, high-value inquiry options focused on conjecture extension, edge cases, and alternative analytical paths.
lib/agents/task-manager.tsx
- Rewrote the Task Manager system prompt from an efficiency/clarity-first approach to a conjecture-driven approach.
- The updated guidance now:
- Defaults to returning
next: "proceed"in most cases. - Reserves
"inquire"for situations where a question would add non-obvious analytical depth (edge cases / what-if scenarios).
- Defaults to returning
- Updated examples to match the new decision policy.
| const result = await generateObject({ | ||
| model: (await getModel()) as LanguageModel, | ||
| system: `As a planet computer, your primary objective is to act as an efficient **Task Manager** for the user's query. Your goal is to minimize unnecessary steps and maximize the efficiency of the subsequent exploration phase (researcher agent). | ||
| system: `As a planet computer, your primary objective is to act as a **Conjecture-Driven Task Manager**. Your goal is to lean into exploration by default, using intelligent assumptions to proceed while only pausing for inquiry when a non-obvious edge case or a significant opportunity to extend the user's conjecture is identified. | ||
|
|
||
| You must first analyze the user's input and determine the optimal course of action. You have two options at your disposal: | ||
| You must analyze the user's input and determine whether to proceed with immediate exploration or to pause for a value-added inquiry. | ||
|
|
||
| **Exploration Efficiency Principles:** | ||
| - **Principle 1: Clarity First (Inquire):** If the query is ambiguous, lacks critical context (especially for geospatial tasks), or could be significantly narrowed down with a simple question, you MUST choose **"inquire"**. This prevents the researcher from wasting tokens and time on broad, inefficient searches. | ||
| - **Principle 2: Proceed When Sufficient:** If the query is clear, specific, and ready for immediate research, choose **"proceed"**. | ||
| **Conjecture-Driven Principles:** | ||
| - **Principle 1: Proceed by Default (Proceed):** If the query allows for a reasonable analytical path, choose **"proceed"**. The researcher agent is capable of handling ambiguity by exploring multiple facets. Do not pause for basic clarifications (like missing specific coordinates or generic context) if a general search or exploration can begin. | ||
| - **Principle 2: Value-Added Inquiry (Inquire):** Choose **"inquire"** ONLY if you identify a critical edge case, a potential "what-if" scenario, or a way to significantly extend the user's conjecture that they might not have considered. The goal is to deepen the conversation, not just clear up ambiguity. | ||
|
|
||
| **Options:** | ||
| 1. **"proceed"**: Choose this if the query is specific enough for the researcher to start a focused exploration immediately. | ||
| 2. **"inquire"**: Choose this if the query is too vague, broad, or requires essential missing parameters (like location, time, or specific metrics) to ensure an efficient and high-quality response. | ||
| **Options:** | ||
| 1. **"proceed"**: Default action. Choose this to start a focused exploration immediately, even if some parameters are broad. | ||
| 2. **"inquire"**: Choose this if the query presents an opportunity to verify a specific edge case or extend the analytical context in a way that provides deeper insight. | ||
|
|
||
| **Inquiry Guidance (If "inquire" is chosen):** | ||
| - **Geospatial Queries:** If the query involves a location, you MUST clarify the following details to ensure the most efficient use of the 'geospatialQueryTool': | ||
| - **Location Specificity:** Ask for full addresses, landmark names, or precise coordinates. | ||
| - **Context:** Ask for time constraints ("during rush hour", "at 3 PM") or specific travel methods (driving, walking). | ||
| - **Output Format:** Ask for specific output formats when needed ("as a map image", "in JSON format"). | ||
| **Inquiry Guidance (If "inquire" is chosen):** | ||
| - **Extend the Conjecture:** Look beyond the immediate question. If a user asks about "pathways," an inquiry might ask about seasonal changes or specific human activities that would leave such tracks. | ||
| - **Identify Edge Cases:** Consider factors like environmental conditions, temporal shifts, or hidden socioeconomic drivers that could influence the answer. | ||
|
|
||
| **Examples for Efficiency:** | ||
| - **User:** "What are the latest news about the floods in India?" -> **Action:** "proceed" (Clear, ready for web search). | ||
| - **User:** "What's the warmest temperature in my area?" -> **Action:** "inquire" (Missing location and preferred metric). | ||
| - **User:** "Show me the nearest park." -> **Action:** "inquire" (Missing current location). | ||
| - **User:** "Tell me about the new AI model." -> **Action:** "inquire" (Too broad; ask for the model name or specific aspect). | ||
|
|
||
| Make your choice wisely to ensure that you fulfill your mission as an efficient Task Manager and deliver the most valuable assistance to the user. | ||
| **Examples for Conjecture-Driven Flow:** | ||
| - **User:** "What are the latest news about the floods in India?" -> **Action:** "proceed" (Immediate exploration). | ||
| - **User:** "What's the warmest temperature in my area?" -> **Action:** "proceed" (Make a reasonable assumption based on general location or proceed to search for regional trends). | ||
| - **User:** "Show me the nearest park." -> **Action:** "proceed" (Start exploration with available location data or broad regional search). | ||
| - **User:** "Tell me about those pathways in the Niger Delta." -> **Action:** "proceed" (Research them first, do not ask basic clarifying questions). | ||
| - **User:** "Is this area prone to flooding?" -> **Action:** "inquire" (Pause to ask if the user is interested in historical data vs. future climate projections, or specific seasonal peaks). | ||
|
|
||
| Be bold in proceeding. Only pause if the inquiry itself adds significant analytical depth to the user's original intent. | ||
| `, |
There was a problem hiding this comment.
The new Task Manager guidance explicitly recommends proceeding for queries like "Show me the nearest park" and "What's the warmest temperature in my area" by making assumptions or doing broad regional search. That can easily produce wrong or misleading results (e.g., “nearest” is undefined without a user location; “my area” is undefined) and it may degrade UX more than a single targeted clarification would.
If the goal is to avoid redundant questions, consider narrowing the “proceed by default” policy to cases where the system actually has some location signal (profile, device location, prior messages), and otherwise allow a minimal clarification for location-dependent intents.
Suggestion
Tighten the system prompt to explicitly permit a minimal location clarification when the user’s request is inherently location-relative and no location context is available. For example, adjust Principle 1 and the examples:
- Add a rule like: "Proceed by default except for inherently relative intents (e.g., nearest, in my area, from here) when no location context exists; in those cases, inquire with a single high-leverage question."
- Update examples so "nearest park" becomes "inquire" only when location context is missing, otherwise "proceed".
Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this change.
| const result = await streamObject({ | ||
| model: (await getModel()) as LanguageModel, | ||
| system: `...`, // Your system prompt remains unchanged | ||
| system: `As a planet computer, your role is to act as a **Deep Inquiry Agent**. Your goal is to extend the user's conjecture and look for non-obvious edge cases that they haven't thought about. | ||
|
|
||
| Instead of asking for basic missing information (which should have been handled by the Task Manager), you should focus on: | ||
| - **Conjecture Extension:** Propose deeper layers of exploration. (e.g., "Are we considering the impact of seasonal shifts on this data?") | ||
| - **Edge Case Verification:** Identify hidden factors that might influence the results. (e.g., "Should we account for recent local socioeconomic changes that might not be in official datasets yet?") | ||
| - **Alternative Perspectives:** Suggest different analytical paths. | ||
|
|
||
| Your inquiries should be thought-provoking and add value to the upcoming exploration phase. Each option you provide should represent a distinct analytical path or a specific edge case to verify. | ||
|
|
||
| Keep your question concise but deep. Provide 2-4 meaningful options, and always allow for user input if they want to provide their own perspective. | ||
| `, | ||
| messages, | ||
| schema: inquirySchema, | ||
| }); |
There was a problem hiding this comment.
The Inquiry Agent prompt says it should provide “2-4 meaningful options” and “always allow for user input,” but it doesn’t anchor those requirements to the actual schema fields being generated by streamObject({ schema: inquirySchema }). If the schema expects specific keys (e.g., question, options, allowCustom, etc.), the model may comply inconsistently, leading to partial objects that don’t map cleanly to the UI.
Because this function is schema-driven, the system prompt should explicitly instruct the model to output in a way that matches the schema (field names + constraints), otherwise you’ll see more retries/partial streaming noise and weaker determinism.
Suggestion
Update the system prompt to reference the schema structure explicitly (without changing runtime code), e.g.:
- "Return an object that conforms to the provided schema. Populate
questionwith a single concise question andoptionswith 2�1 items. Ensure one option represents 'Other / my own perspective'."
If you share inquirySchema, I can propose exact wording aligned to its field names. Reply with "@CharlieHelps yes please" if you'd like me to add a commit with a schema-aligned prompt update.
| model: (await getModel()) as LanguageModel, | ||
| system: `...`, // Your system prompt remains unchanged | ||
| system: `As a planet computer, your role is to act as a **Deep Inquiry Agent**. Your goal is to extend the user's conjecture and look for non-obvious edge cases that they haven't thought about. | ||
|
|
||
| Instead of asking for basic missing information (which should have been handled by the Task Manager), you should focus on: | ||
| - **Conjecture Extension:** Propose deeper layers of exploration. (e.g., "Are we considering the impact of seasonal shifts on this data?") | ||
| - **Edge Case Verification:** Identify hidden factors that might influence the results. (e.g., "Should we account for recent local socioeconomic changes that might not be in official datasets yet?") | ||
| - **Alternative Perspectives:** Suggest different analytical paths. | ||
|
|
||
| Your inquiries should be thought-provoking and add value to the upcoming exploration phase. Each option you provide should represent a distinct analytical path or a specific edge case to verify. | ||
|
|
||
| Keep your question concise but deep. Provide 2-4 meaningful options, and always allow for user input if they want to provide their own perspective. | ||
| `, |
There was a problem hiding this comment.
The system prompts now contain strong domain claims and examples (e.g., “planet computer”, “hidden socioeconomic drivers”) that may not be appropriate for all user queries and could bias the model toward speculative explanations. This is especially risky for factual questions where the correct behavior is to gather evidence rather than invent plausible-sounding drivers.
This isn’t a style nit: it can measurably increase hallucination risk by nudging the model to always look for “hidden” factors even when the query is straightforward.
Suggestion
Add a grounding constraint to both prompts to reduce speculative drift, e.g.:
- "Prefer testable hypotheses and clearly label uncertainty. Do not invent drivers; propose them as hypotheses to verify and suggest what evidence would confirm/deny them."
This preserves “deep inquiry” while keeping outputs falsifiable.
Reply with "@CharlieHelps yes please" if you’d like me to add a commit with this wording tweak.
ⓘ Your approaching your monthly quota for Qodo. Upgrade your plan PR Code Suggestions ✨Explore these optional code suggestions:
|
||||||||||||||
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@lib/agents/task-manager.tsx`:
- Around line 18-19: The call to generateObject uses (await getModel()) which
may return Grok 4.2 (grok-4-fast-non-reasoning) that is incompatible with
structured-output; update getModel usage so agents that call generateObject
request a structured-output-capable model: add a parameter to getModel like
getModel(requireVision?: boolean, requireStructuredOutput?: boolean) and, in
task-manager (where generateObject is invoked), pass
requireStructuredOutput=true so getModel returns a compatible model (e.g.,
gpt-4o) or falls back to a safe default; alternatively, if you prefer not to add
params, remove grok-4-fast-non-reasoning from the selectable models used by
task-manager before calling generateObject. Ensure references: getModel and
generateObject are adjusted accordingly.
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
lib/agents/inquire.tsxlib/agents/task-manager.tsx
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2026-01-13T13:26:30.086Z
Learnt from: ngoiyaeric
Repo: QueueLab/QCX PR: 0
File: :0-0
Timestamp: 2026-01-13T13:26:30.086Z
Learning: Workers that rely on ai.generateObject (geojsonParser, map-command-generator, feedback-analyzer) must not use the X.AI 'grok-4-fast-non-reasoning' model because X.AI’s chat/completions rejects JSON Schema structured-output arguments; prefer OpenAI gpt-4o for structured outputs.
Applied to files:
lib/agents/task-manager.tsx
🔇 Additional comments (2)
lib/agents/task-manager.tsx (1)
20-43: Clear conjecture-driven framework with well-defined decision principles.The prompt effectively implements the "proceed by default" philosophy with clear guidance on when to inquire. The examples (lines 37-41) are helpful for establishing expected behavior.
One potential ambiguity: The "flooding" example (line 41) chooses "inquire" to ask about historical vs. future projections, but this distinction could also be explored proactively during research. Consider clarifying that this inquiry adds significant value because the analytical approach differs fundamentally between historical analysis and climate modeling.
lib/agents/inquire.tsx (1)
27-37: Well-structured Deep Inquiry Agent prompt with clear directives.The prompt effectively shifts the agent from asking basic clarifying questions to proposing deeper analytical paths. The separation of concerns (Task Manager handles basic info, Inquiry Agent extends conjectures) aligns well with the task-manager changes.
The
inquirySchemasupports both key requirements:
- User input accommodation: The schema includes an
allowsInputboolean field with optionalinputLabelandinputPlaceholderfor customization, fully supporting the "always allow for user input" instruction.- Option flexibility: The schema does not enforce a specific option count, allowing the agent to provide 1-2 options for straightforward queries or 3-4+ for complex ones, while the prompt provides guidance toward 2-4 as best practice.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
| const result = await generateObject({ | ||
| model: (await getModel()) as LanguageModel, |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Find getModel implementation and its context
rg -nA 20 'function getModel|const getModel|export.*getModel' --type ts --type tsxRepository: QueueLab/QCX
Length of output: 83
🏁 Script executed:
# Check for X.AI or grok model references
rg -n 'grok|x\.ai|xai|grok-4-fast-non-reasoning' --type ts --type tsx -iRepository: QueueLab/QCX
Length of output: 83
🏁 Script executed:
# Look for model configuration or selection logic
rg -nB 3 -A 3 'getModel\(\)' lib/agents/task-manager.tsxRepository: QueueLab/QCX
Length of output: 653
🏁 Script executed:
# Find all model-related configuration files
fd -t f -e ts -e tsx | xargs grep -l 'model\|MODEL' | grep -E '(config|env|constant|model)' | head -20Repository: QueueLab/QCX
Length of output: 111
🏁 Script executed:
# Find getModel definition in ts/tsx files without type filter
rg -n 'export.*function getModel|export const getModel|const getModel.*=' --max-count=20Repository: QueueLab/QCX
Length of output: 142
🏁 Script executed:
# Search for grok and xai references without type restrictions
rg -n 'grok|x\.ai|xai' -i --max-count=30Repository: QueueLab/QCX
Length of output: 3684
🏁 Script executed:
# Check the model-selection-form file
cat -n components/settings/components/model-selection-form.tsx | head -80Repository: QueueLab/QCX
Length of output: 3014
🏁 Script executed:
# Look for model-related imports and constants
rg -n 'model.*=.*grok|model.*=.*gpt|model.*=.*openai' -i --max-count=20Repository: QueueLab/QCX
Length of output: 38
🏁 Script executed:
cat -n lib/utils/index.ts | sed -n '20,90p'Repository: QueueLab/QCX
Length of output: 3125
Add safeguard: getModel() can return grok-4-fast-non-reasoning, which is incompatible with generateObject() for structured outputs.
The function currently allows (and defaults to) Grok 4.2, which maps to grok-4-fast-non-reasoning. X.AI's API rejects JSON Schema structured-output arguments. Either:
- Add a parameter to
getModel(requireVision, requireStructuredOutput)and switch to a compatible model (e.g., gpt-4o) when structured outputs are needed, or - Remove Grok 4.2 from selectable models for agents using
generateObject.
🤖 Prompt for AI Agents
In `@lib/agents/task-manager.tsx` around lines 18 - 19, The call to generateObject
uses (await getModel()) which may return Grok 4.2 (grok-4-fast-non-reasoning)
that is incompatible with structured-output; update getModel usage so agents
that call generateObject request a structured-output-capable model: add a
parameter to getModel like getModel(requireVision?: boolean,
requireStructuredOutput?: boolean) and, in task-manager (where generateObject is
invoked), pass requireStructuredOutput=true so getModel returns a compatible
model (e.g., gpt-4o) or falls back to a safe default; alternatively, if you
prefer not to add params, remove grok-4-fast-non-reasoning from the selectable
models used by task-manager before calling generateObject. Ensure references:
getModel and generateObject are adjusted accordingly.
2 participants
User description
This change addresses the issue where the inquiry agent was asking redundant and out-of-context questions.
Task Manager Refactor: The Task Manager's system prompt has been rewritten to follow a "Conjecture-Driven" approach. It now defaults to "proceed" for most queries, trusting the researcher agent to handle ambiguity. It only pauses for an "inquire" action when it identifies a genuine opportunity to add analytical depth or explore a complex edge case.
Inquiry Agent Implementation: The Inquiry Agent now has a robust system prompt (replacing a placeholder). It is instructed to act as a "Deep Inquiry Agent," focusing on extending the user's conjecture with "what-if" scenarios, seasonal variations, and hidden socioeconomic drivers, rather than asking for basic missing parameters.
These changes ensure the system is more proactive and provides a more sophisticated, "co-thinking" experience for the user.
PR created automatically by Jules for task 30161385172256793 started by @ngoiyaeric
PR Type
Enhancement
Description
Refactored Task Manager to default to proceeding with exploration, only inquiring for deep edge cases
Implemented robust system prompt for Inquiry Agent to extend conjectures with non-obvious factors
Shifted from basic clarification-focused to conjecture-driven analytical approach
Improved user experience with proactive, value-added inquiry patterns
Diagram Walkthrough
File Walkthrough
inquire.tsx
Implement Deep Inquiry Agent system promptlib/agents/inquire.tsx
Agent instructions
alternative perspectives
request basic missing information
phase
task-manager.tsx
Refactor Task Manager to conjecture-driven approachlib/agents/task-manager.tsx
proceed-by-default strategy
metrics)
and only inquiring for edge cases
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.