Skip to content

feat: new tool - prediction websearch#66

Draft
LOCKhart07 wants to merge 4 commits intomainfrom
feat/prediction-websearch
Draft

feat: new tool - prediction websearch#66
LOCKhart07 wants to merge 4 commits intomainfrom
feat/prediction-websearch

Conversation

@LOCKhart07
Copy link
Member

@LOCKhart07 LOCKhart07 commented Aug 7, 2025

Proposed changes

Introduce a new tool prediction-websearch

Features:

  • Uses openai websearch api directly instead of relying on google search
  • Uses structured output calling from openai instead of parsing through the output manually

Notes:

  • Has extremely high token costs relative to existing tools
  • Only supports openai models for now.
  • Want to add support for gpt-5 because it costs less to use websearch api with gpt5 https://platform.openai.com/docs/pricing#built-in-tools
  • This will probably get better roi with higher bet sizes because of it's high token cost

Fixes

Ideally should have higher accuracy

Types of changes

  • Non-breaking fix (non-breaking change which fixes an issue)
  • Breaking fix (breaking change which fixes an issue)
  • Non-breaking feature (non-breaking change which adds functionality)
  • Breaking feature (breaking change which adds functionality)
  • Refactor (non-breaking change which changes implementation)
  • Messy (mixture of the above - requires explanation!)

Checklist

  • I have read the CONTRIBUTING doc
  • I am making a pull request against the main branch (left side). Also you should start your branch off our main.
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works

Further comments

Apart from the general stuff, As part of review, I'm looking for suggestions on:

  • Tool name
  • A way to test this tool against pre existing tools.
    • My current plan is to take some 30 or so markets that will close in a day or two.
    • Run all existing tools plus this one against these markets and then check results after the markets close
  • If this approach could work even with the high token costs

Comparison with Existing Approaches

Web Search Tier Multiplier vs. 1k GPT-4.1 Tokens
GPT-4.1 / GPT-4o ~3.1× more expensive
GPT-5 / o-series ~1.25× more expensive

@LOCKhart07 LOCKhart07 self-assigned this Aug 8, 2025
@LOCKhart07 LOCKhart07 added the enhancement New feature or request label Aug 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant