Tool use & Rails integration Proposal

[peterjacobson]

Discussion with Gemini & Claude:

Okay Pete, this is exciting! Refining the DX around tool usage is key to making instruct truly powerful and intuitive for Rails developers. Let's build out the proposed solution based on the handler-centric DSL and the Instruct.generate_for helper.

Motivation

Advanced Usage: Integrating with Rails Apps

instruct provides features for robust integration with Rails applications, enabling asynchronous processing, interaction with application models via tools, and a streamlined developer experience.

Core pattern

The primary pattern involves creating "handler" objects (often ActiveRecord models or service objects) that manage specific AI interactions. These handlers define the tools the AI can use and implement the corresponding actions.

1. Include Instruct::AiResponseHandler: Add this concern to your handler class. It provides a DSL for defining tools and standard callbacks.
2. Define Tools with ai_tool: Use the class-level ai_tool DSL method to declare the functions the AI can call. Specify the tool's name (matching a public instance method), description, and input parameters (using JSON Schema).
3. Implement Tool Methods: Create public instance methods on your handler class with names matching the tools defined via ai_tool. These methods contain your application logic (e.g., updating records, calling other services, broadcasting UI updates). They should accept arguments as keyword arguments based on the parameters definition and return a result (usually a String or Hash) to be sent back to the LLM.
4. Implement Callbacks (Optional): Override methods like handle_ai_text_chunk, handle_ai_finish, handle_ai_error, handle_tool_error to customize behavior during the LLM interaction lifecycle.

API

Examply in a rails model

# app/models/assessment_updater.rb
class AssessmentUpdater
  include GlobalID::Identification # Needed for background jobs
  include Instruct::AiResponseHandler

  attr_reader :assessment

  def initialize(assessment)
    @assessment = assessment
    @id = assessment.id # For GlobalID::Identification
  end

  # --- Tool Definitions ---
  ai_tool :update_assessment_instructions,
    description: "Updates the main instructions for the assessment.",
    parameters: {
      type: "object", properties: {
        new_instructions: { type: "string", description: "The revised instructions text." }
      }, required: ["new_instructions"]
    }

  ai_tool :add_comment_to_assessment,
    description: "Adds a comment to the assessment discussion thread.",
    parameters: {
      type: "object", properties: {
        comment_text: { type: "string", description: "The content of the comment." }
      }, required: ["comment_text"]
    }

  # --- Tool Implementations ---
  def update_assessment_instructions(new_instructions:)
    if assessment.update(instructions: new_instructions)
      # Example: Broadcast Turbo Stream update for the UI
      assessment.broadcast_replace_to assessment, :instructions_field,
        target: ActionView::RecordIdentifier.dom_id(assessment, :instructions_field),
        partial: "assessments/form_instructions_field",
        locals: { assessment: assessment }
      "Assessment instructions updated successfully." # Result for LLM
    else
      "Failed to update instructions: #{assessment.errors.full_messages.join(', ')}"
    end
  end

  def add_comment_to_assessment(comment_text:)
    comment = assessment.comments.create(author: "AI Assistant", content: comment_text)
    if comment.persisted?
       # Example: Broadcast Turbo Stream append
       comment.broadcast_append_to assessment, :comments,
         target: ActionView::RecordIdentifier.dom_id(assessment, :comments_container),
         partial: "comments/comment",
         locals: { comment: comment }
      "Comment added."
    else
      "Failed to add comment: #{comment.errors.full_messages.join(', ')}"
    end
  end

  # --- Optional Callbacks ---
  def handle_ai_finish(reason)
    Rails.logger.info "AssessmentUpdater finished for assessment #{assessment.id}: #{reason}"
  end

  def handle_tool_error(tool_name, exception, arguments)
     Rails.logger.error "AssessmentUpdater tool error in '#{tool_name}' for assessment #{assessment.id}: #{exception.message}"
     # Maybe add a comment indicating the tool failure
     add_comment_to_assessment(comment_text: "Sorry, I encountered an error trying to perform the action: #{tool_name}.")
  end
end

Asynchronous Generation with Instruct.generate_for

For most Rails integrations, especially those involving tool calls or potentially long-running generations, use the asynchronous helper Instruct.generate_for.

Purpose: Simplifies triggering AI generation with a handler. It automatically retrieves tools from the handler, constructs the Instruct::Prompt, and enqueues the standard Instruct::GenerateJob.
Usage: Call Instruct.generate_for(handler_instance, user_prompt:, system_prompt: nil, ...) from your controller, model, or service.
Requirements:
The handler object must include Instruct::AiResponseHandler.
The handler object must include GlobalID::Identification (so it can be passed to the background job).
ActiveJob must be configured in your Rails application.

Example in a Controller action

class AssessmentsController < ApplicationController
  def suggest_improvements
    assessment = Assessment.find(params[:id])
    handler = AssessmentUpdater.new(assessment) # Create the handler

    user_prompt = "Review the assessment instructions and suggest improvements using the available tools."

    # Trigger the background job
    Instruct.generate_for(handler, user_prompt: user_prompt)

    # Redirect or render response indicating generation has started
    flash[:notice] = "AI is generating suggestions..."
    redirect_to assessment_path(assessment)
  end
end

Background Job: Instruct::GenerateJob

This standard ActiveJob class orchestrates the AI interaction in the background.

• It locates the handler using GlobalID.
• It retrieves tool definitions from the handler.
• It instantiates Instruct::Orchestrator.
• It calls orchestrator.start, which executes prompt.call and dispatches events (tool calls, text chunks, finish/error) to the handler's methods and callbacks.
• Tool methods within the handler can trigger UI updates via Turbo Streams broadcasts.

You generally don't need to interact with this job directly; use Instruct.generate_for. You can configure its queue name and retry behavior like any other ActiveJob.

Direct prompt.call with Manual Handling

For synchronous use cases or when you need fine-grained control over the event stream without the Orchestrator, you can still use prompt.call directly with a block. If you pass tools: to gen(), prompt.call will yield Instruct::ToolCallRequest events when the LLM wants to use a tool. Your block is responsible for executing the tool and calling event.respond(result) or event.respond_error(message).

(Reference example from instruct/README.md startLine:408 endLine:442)

handler = MyToolHandler.new
prompt = p.user("...") + gen(tools: handler.ai_tools_definitions)

prompt.call do |event|
  case event
  when Instruct::ToolCallRequest
    # ... find method, execute, call event.respond/respond_error ...
  when Hash # Assuming simple hash events for text/finish
    # ... handle event[:type] == :text_chunk or :finish ...
  end
end

This approach gives maximum flexibility but requires implementing the orchestration logic manually within the block. The Orchestrator + GenerateJob pattern is recommended for most Rails applications.

Proposed implementation

Here's the proposed implementation, addressing the core components, missing details, tests, and documentation:

1. Core instruct Gem Enhancements

a) Instruct::AiResponseHandler Concern (with DSL)

This concern will live in the instruct gem and provide the DSL for defining tools alongside standard callbacks.

require 'active_support/concern'
require 'active_support/core_ext/class/attribute'

module Instruct
  # Provides a DSL for defining AI tools and standard callbacks
  # for handling LLM interactions within a class (e.g., an
  # ActiveRecord model or service object).
  module AiResponseHandler
    extend ActiveSupport::Concern

    included do
      # Stores tool definitions keyed by tool name symbol
      class_attribute :_ai_tool_definitions,
                      instance_writer: false,
                      default: {}
    end

    module ClassMethods
      # DSL method to define a tool the handler can execute.
      #
      # @param name [Symbol] The name of the tool (must match method name).
      # @param description [String] Description for the LLM.
      # @param parameters [Hash] JSON Schema definition of arguments.
      def ai_tool(name, description:, parameters:)
        unless parameters.is_a?(Hash) &&
               parameters[:type] == "object" &&
               parameters[:properties].is_a?(Hash)
          raise ArgumentError, "Parameters must be a Hash matching JSON Schema object definition"
        end

        # Store definition compatible with LLM API (e.g., OpenAI)
        self._ai_tool_definitions = _ai_tool_definitions.merge(
          name.to_sym => {
            type: "function", # Standard for OpenAI/Anthropic tools
            function: {
              name: name.to_s,
              description: description,
              parameters: parameters
            }
          }
        )
      end

      # Returns the array of tool definition hashes suitable for LLM APIs.
      #
      # @return [Array<Hash>]
      def ai_tools_specifications
        _ai_tool_definitions.values
      end
    end

    # --- Instance Methods ---

    # Returns the tool specifications for this specific handler instance.
    # Defaults to the class-level definitions. Override for dynamic tools.
    #
    # @return [Array<Hash>]
    def ai_tools_definitions
      self.class.ai_tools_specifications
    end

    # --- Standard Callbacks (Implement in including class as needed) ---

    # Called when the LLM generates a text chunk (if streaming).
    # @param content [String] The text chunk.
    def handle_ai_text_chunk(content)
      gid = self.try(:to_gid).try(:to_s) || self.class.name
      Instruct.logger.debug("Handler #{gid} received text: #{content.truncate(50)}")
    end

    # Called when the LLM interaction finishes successfully.
    # @param reason [String] The reason for finishing (e.g., 'stop', 'tool_calls').
    def handle_ai_finish(reason)
      gid = self.try(:to_gid).try(:to_s) || self.class.name
      Instruct.logger.debug("Handler #{gid} finished: #{reason}")
    end

    # Called when an error occurs during the LLM API call or streaming.
    # @param exception [Exception] The error object.
    def handle_ai_error(exception)
      gid = self.try(:to_gid).try(:to_s) || self.class.name
      Instruct.err_logger.error("Handler #{gid} AI error: #{exception.message}")
    end

    # Called by Orchestrator if a tool method raises an error.
    # @param tool_name [String] The name of the tool that failed.
    # @param exception [Exception] The error raised by the tool method.
    # @param arguments [Hash] The arguments passed to the tool.
    def handle_tool_error(tool_name, exception, arguments)
      gid = self.try(:to_gid).try(:to_s) || self.class.name
      Instruct.err_logger.error("Handler #{gid} tool error in '#{tool_name}': #{exception.message}")
      # Consider re-raising, logging, or notifying
    end

    # Called by Orchestrator if the LLM requests a tool the handler
    # doesn't respond_to?.
    # @param tool_name [String] The name of the unknown tool.
    def handle_unknown_tool(tool_name)
       gid = self.try(:to_gid).try(:to_s) || self.class.name
       Instruct.err_logger.warn("Handler #{gid} received unknown tool request: '#{tool_name}'")
    end

    # Optional: Called *before* dispatch if rate limits are checked.
    # def handle_rate_limit_exceeded(limit_details)
    #   # ...
    # end
  end
end

b) Instruct::ToolCallRequest Event Class

This class represents the event yielded by prompt.call when the LLM wants to use a tool. It needs the respond/respond_error methods.

module Instruct
  # Represents a request from the LLM to call a tool.
  # This object is yielded by `prompt.call`. The block *must* call
  # either `respond` or `respond_error` to continue the LLM interaction.
  class ToolCallRequest # We might need a base class like StreamEvent later
    attr_reader :id, :tool_name, :arguments

    # @param id [String] The unique ID for this tool call (from LLM API).
    # @param tool_name [String] The name of the tool to be called.
    # @param arguments [Hash] The arguments provided by the LLM.
    # @param response_block [Proc] A block to call with the result/error.
    def initialize(id:, tool_name:, arguments:, response_block:)
      @id = id
      @tool_name = tool_name
      @arguments = arguments
      @response_block = response_block
      @responded = false
    end

    # Call this with the successful result of the tool execution.
    # The result will be sent back to the LLM.
    # @param result [Object] The return value of the tool method.
    #                         Should typically be JSON-serializable (String, Hash, etc.).
    def respond(result)
      raise Instruct::Error, "Already responded to tool call #{id}" if @responded
      # The response_block communicates back to the core prompt.call loop
      @response_block.call(tool_call_id: @id, result: result, error: false)
      @responded = true
    end

    # Call this if the tool execution fails.
    # The error message will be sent back to the LLM.
    # @param error_message [String] A description of the error.
    def respond_error(error_message)
      raise Instruct::Error, "Already responded to tool call #{id}" if @responded
      # The response_block communicates back to the core prompt.call loop
      @response_block.call(tool_call_id: @id, result: error_message.to_s, error: true)
      @responded = true
    end

    def responded?
      @responded
    end

    # Potentially add type/event markers if inheriting from a base event class
    def type
      :tool_call_request
    end
  end
end

c) Instruct::Orchestrator (Refined)

Updated to correctly handle the ToolCallRequest and call event.respond/event.respond_error.

# Assuming StreamEvent and ToolCallRequest are loaded
require 'instruct/tool_call_request'
# require 'instruct/stream_event' # Define if needed for text/finish

module Instruct
  class Orchestrator
    # ... (initialize, attr_reader remain the same) ...
    # Reference: instruct/lib/instruct/orchestrator.rb startLine:10 endLine:18

    def start
      Instruct.logger.info("Orchestrator starting for #{handler_gid_or_name}")
      final_result = nil # Or capture differently if needed

      # Ensure tools are passed to the call method
      # The core prompt.call implementation needs to handle this 'tools:' kwarg
      final_result = prompt.call(tools: @tools) do |event|
        process_event(event)
      end

      Instruct.logger.info("Orchestrator finished for #{handler_gid_or_name}")
      final_result # Return the final result from prompt.call if any

    rescue => e
      Instruct.err_logger.error("Orchestrator error for #{handler_gid_or_name}: #{e.message}\n#{e.backtrace.first(5).join("\n")}")
      # Delegate error to handler's handle_ai_error
      handler.handle_ai_error(e) if handler.respond_to?(:handle_ai_error)
      # Re-raise or return error indicator? Depends on desired behavior.
      raise e # Or return a specific error object
    end

    private

    def process_event(event)
      # Check the type of event yielded by prompt.call
      case event
      when Instruct::ToolCallRequest
        # Handle the request to call a tool
        handle_tool_call_request(event)

      # Assuming other events might be simple Hashes or a StreamEvent class
      when Hash # Simple placeholder for other event types
        case event[:type]
        when :text_chunk
          handler.handle_ai_text_chunk(event[:content]) if handler.respond_to?(:handle_ai_text_chunk)
        when :finish
          handler.handle_ai_finish(event[:reason]) if handler.respond_to?(:handle_ai_finish)
        else
          Instruct.err_logger.warn("Orchestrator received unknown event type: #{event[:type]}")
        end
      # Add specific StreamEvent class check if defined
      # when Instruct::StreamEvent
      #    # handle based on event.type ...

      else
        Instruct.err_logger.warn("Orchestrator received unexpected event object: #{event.inspect}")
      end
    end

    # Handles the ToolCallRequest event specifically
    def handle_tool_call_request(event)
      method_name = event.tool_name.to_sym
      arguments = event.arguments || {} # Ensure arguments is a Hash

      unless arguments.is_a?(Hash)
        err_msg = "Invalid arguments for tool '#{event.tool_name}': Expected Hash, got #{arguments.class}."
        Instruct.err_logger.error(err_msg + " Raw: #{arguments.inspect}")
        event.respond_error(err_msg) # Respond with error
        handler.handle_tool_error(event.tool_name, TypeError.new(err_msg), arguments) if handler.respond_to?(:handle_tool_error)
        return
      end

      if handler.respond_to?(method_name)
        Instruct.logger.debug("Orchestrator executing tool ##{method_name} on #{handler_gid_or_name}")
        begin
          # Execute the tool method on the handler
          result = handler.public_send(method_name, **arguments.symbolize_keys)
          # Respond with the successful result
          event.respond(result)
          Instruct.logger.debug("Tool ##{method_name} executed successfully.")
        rescue => e
          Instruct.err_logger.error("Tool ##{method_name} error: #{e.message}\n#{e.backtrace.first(5).join("\n")}")
          # Respond with the error message
          event.respond_error(e.message)
          # Also call the handler's error callback
          handler.handle_tool_error(event.tool_name, e, arguments) if handler.respond_to?(:handle_tool_error)
        end
      else
        # Handler does not implement the requested tool
        Instruct.logger.warn("Handler #{handler_gid_or_name} does not implement tool method ##{method_name}")
        event.respond_error("Tool '#{event.tool_name}' is not available.") # Respond with error
        handler.handle_unknown_tool(event.tool_name) if handler.respond_to?(:handle_unknown_tool)
      end
    ensure
      # Ensure we always respond if something unexpected happened before responding
      unless event.responded?
         Instruct.err_logger.error("Tool call '#{event.tool_name}' was not responded to!")
         event.respond_error("Internal error: Tool call was not processed.")
      end
    end

    def check_handler_interface
      # Optional: Add checks to ensure handler includes AiResponseHandler
      # unless handler.is_a?(Instruct::AiResponseHandler) ...
    end

    def handler_gid_or_name
      @handler_gid_or_name ||= handler.try(:to_gid).try(:to_s) || handler.class.name
    end
  end
end

d) Instruct.generate_for Helper

The high-level API for Rails integration.

require 'instruct/prompt'
require 'instruct/ai_response_handler'
require 'instruct/generate_job' # Assuming job is defined

module Instruct
  # ... other Instruct module methods ...

  # High-level API to generate content asynchronously using a handler
  # that includes Instruct::AiResponseHandler.
  #
  # Automatically constructs the prompt, retrieves tools from the handler,
  # and enqueues the standard Instruct::GenerateJob.
  #
  # @param handler [Object] An instance that includes AiResponseHandler.
  # @param user_prompt [String] The main user query or instruction.
  # @param system_prompt [String, nil] Optional system prompt.
  # @param model [String, nil] Optional model override (defaults to Instruct.default_model).
  # @param context [String, Hash, nil] Optional additional context (structure TBD).
  # @return [ActiveJob::Base] The enqueued job instance.
  def self.generate_for(handler,
                        user_prompt:,
                        system_prompt: nil,
                        model: nil,
                        context: nil # How to best use context needs thought
                       )

    unless handler.class.include?(Instruct::AiResponseHandler)
      raise ArgumentError, "Handler must include Instruct::AiResponseHandler"
    end
    unless handler.respond_to?(:to_gid)
        raise ArgumentError, "Handler must be usable with GlobalID (include GlobalID::Identification)"
    end

    # 1. Get tools from the handler instance
    tools = handler.ai_tools_definitions
    Instruct.logger.debug("Using tools for #{handler.class.name}: #{tools.map { |t| t.dig(:function, :name) }}")

    # 2. Construct the prompt object
    #    (This might need refinement based on Instruct::Prompt capabilities)
    prompt_obj = Instruct::Prompt.new.tap do |p|
      p.system(system_prompt) if system_prompt
      # How to incorporate 'context'? Add as text? Specific method?
      p.user(user_prompt)
      # Add generation parameters, including tools
      p.gen!(model: model || Instruct.default_model, tools: tools)
    end

    # 3. Enqueue the standard background job
    job = Instruct::GenerateJob.perform_later(
      handler.to_gid, # Pass handler GID
      prompt_obj      # Pass the constructed prompt object
    )
    Instruct.logger.info("Enqueued Instruct::GenerateJob for #{handler.class.name} (Job ID: #{job.job_id})")
    job
  end

  # Placeholder for default model configuration
  def self.default_model
    ENV.fetch('INSTRUCT_DEFAULT_MODEL', 'gpt-4o') # Example
  end

  # Placeholder for loggers
  def self.logger
    @logger ||= defined?(Rails) ? Rails.logger : Logger.new($stdout)
  end
  def self.err_logger
    @err_logger ||= defined?(Rails) ? Rails.logger : Logger.new($stderr)
  end
end

e) Instruct::GenerateJob

The standard background job.

require 'active_job'
require 'globalid'
require 'instruct/orchestrator'

# Requires the including application to have ActiveJob configured
# and the handler object to include GlobalID::Identification.
class Instruct::GenerateJob < ActiveJob::Base
  queue_as :default # Or :low, or configure dynamically

  # Default retry strategy (customize as needed)
  retry_on StandardError, wait: :exponentially_longer, attempts: 5

  discard_on ActiveJob::DeserializationError # Cannot recover if args are bad

  def perform(handler_gid, prompt_obj)
    handler = GlobalID::Locator.locate(handler_gid)
    unless handler
      Rails.logger.error "Instruct::GenerateJob: Handler not found for GID: #{handler_gid}. Discarding job."
      return # Or raise specific error if discard_on isn't sufficient
    end

    handler_name = handler.try(:to_gid).try(:to_s) || handler.class.name
    Rails.logger.info "Instruct::GenerateJob starting for #{handler_name}"

    # Orchestrator gets tools directly from the handler instance now
    tools = handler.ai_tools_definitions

    # Instantiate and run the orchestrator
    orchestrator = Instruct::Orchestrator.new(
      handler: handler,
      prompt: prompt_obj,
      tools: tools # Pass tools for Orchestrator's use
    )

    # Execute the prompt and handle events via the orchestrator
    orchestrator.start

    Rails.logger.info "Instruct::GenerateJob finished successfully for #{handler_name}"

  rescue => e
    # Log error for visibility, rely on ActiveJob retries/discard
    Rails.logger.error "Instruct::GenerateJob failed for #{handler_name}: #{e.message}\n#{e.backtrace.join("\n")}"
    # Call handler's error hook if possible, though it might be the source of the error
    handler.handle_ai_error(e) if handler&.respond_to?(:handle_ai_error)
    raise e # Re-raise for ActiveJob retry/failure mechanism
  end
end

f) Core prompt.call / Adapter Changes (Conceptual)

• LLM Client Adapters (e.g., instruct/lib/instruct/adapters/openai.rb):
  • When making the API call, pass the tools array and set tool_choice="auto" (or as configured).
  • When processing the response stream:
    • If a chunk indicates a tool call (delta.tool_calls), aggregate the id, function.name, and function.arguments chunks.
    • Once a complete tool call is received, create a Proc (response_block) that knows how to send the result back to the next API call (as a tool role message).
    • yield an Instruct::ToolCallRequest.new(id:, tool_name:, arguments:, response_block:).
    • Pause processing the stream for this interaction until the response_block is called (via event.respond/event.respond_error).
    • When the response_block is called, make the next API call, including the tool result message.
    • Resume processing the stream from the LLM's new response.
  • If a chunk is regular text (delta.content), yield a text chunk event (e.g., yield { type: :text_chunk, content: chunk }).
  • If the stream finishes (finish_reason), yield a finish event (e.g., yield { type: :finish, reason: reason }).

This is the most complex part, deeply tied to the specific LLM API and the streaming logic.

2. Testing

a) Handler Unit Test Example

require 'rails_helper'

RSpec.describe RubricBuilder, type: :model do
  # Include GlobalID support for testing GID generation
  include GlobalID::Identification

  let(:workbench) { instance_double(Workbench, update_doc_content: true, start_new_message: true) }
  let(:builder) { RubricBuilder.new(workbench: workbench) }

  describe "AI Tools" do
    it "defines the :update_rubric_content tool" do
      spec = RubricBuilder.ai_tools_specifications.find { |t| t.dig(:function, :name) == "update_rubric_content" }
      expect(spec).to be_present
      expect(spec.dig(:function, :description)).to include("Updates the entire content")
      expect(spec.dig(:function, :parameters, :properties, :content, :type)).to eq("string")
    end

    it "defines the :add_chat_message tool" do
      spec = RubricBuilder.ai_tools_specifications.find { |t| t.dig(:function, :name) == "add_chat_message" }
      expect(spec).to be_present
      # ... check description and parameters ...
    end

    describe "#update_rubric_content" do
      it "calls workbench.update_doc_content" do
        expect(workbench).to receive(:update_doc_content).with("New Content")
        result = builder.update_rubric_content(content: "New Content")
        expect(result).to eq("Rubric content updated.")
      end
    end

    describe "#add_chat_message" do
      it "calls workbench.start_new_message" do
        expect(workbench).to receive(:start_new_message).with(role: "assistant", content: "Hello")
        result = builder.add_chat_message(message: "Hello")
        expect(result).to eq("Message sent to educator.")
      end
    end
  end

  describe "Callbacks" do
    it "logs finish event" do
      expect(Rails.logger).to receive(:info).with(/RubricBuilder AI generation finished: stop/)
      builder.handle_ai_finish("stop")
    end

    it "logs error and adds chat message on AI error" do
      error = StandardError.new("API Failed")
      expect(Rails.logger).to receive(:error).with(/RubricBuilder AI error: API Failed/)
      # Expect chat message to be added via the tool method
      expect(builder).to receive(:add_chat_message).with(message: /Sorry, I encountered an error/)
      builder.handle_ai_error(error)
    end
  end
end

b) Job Integration Test Example

require 'rails_helper'

RSpec.describe Instruct::GenerateJob, type: :job do
  include ActiveJob::TestHelper

  # Mock handler class
  class MockAiHandler
    include GlobalID::Identification
    include Instruct::AiResponseHandler

    attr_accessor :id, :tool_called_args

    # Define a tool for testing
    ai_tool :test_tool,
            description: "A test tool",
            parameters: { type: "object", properties: { param: { type: "string" } }, required: ["param"] }

    def initialize(id = 1)
      @id = id
      @tool_called_args = nil
    end

    # Implement the tool method
    def test_tool(param:)
      @tool_called_args = { param: param }
      "Tool result for #{param}"
    end

    # Implement callbacks if needed for testing
    def handle_ai_finish(reason); end
    def handle_ai_error(exception); end
    def handle_tool_error(tool_name, exception, arguments); end
  end

  let(:handler) { MockAiHandler.new(5) }
  let(:prompt) { Instruct::Prompt.new.user("Call test tool with param 'hello'").gen!(tools: handler.ai_tools_definitions) }
  let(:handler_gid) { handler.to_gid }

  it "enqueues the job" do
    expect {
      Instruct::GenerateJob.perform_later(handler_gid, prompt)
    }.to have_enqueued_job(Instruct::GenerateJob)
  end

  it "executes perform" do
    # Mock the Orchestrator
    mock_orchestrator = instance_double(Instruct::Orchestrator)
    expect(Instruct::Orchestrator).to receive(:new).with(
      handler: an_instance_of(MockAiHandler),
      prompt: prompt,
      tools: handler.ai_tools_definitions # Check tools are passed
    ).and_return(mock_orchestrator)

    expect(mock_orchestrator).to receive(:start)

    # Perform the job synchronously for testing
    perform_enqueued_jobs { Instruct::GenerateJob.perform_later(handler_gid, prompt) }
  end

  it "locates the handler" do
    # Stub GlobalID::Locator if not using a real DB record
    allow(GlobalID::Locator).to receive(:locate).with(handler_gid).and_return(handler)
    mock_orchestrator = instance_double(Instruct::Orchestrator, start: true)
    allow(Instruct::Orchestrator).to receive(:new).and_return(mock_orchestrator)

    perform_enqueued_jobs { Instruct::GenerateJob.perform_later(handler_gid, prompt) }
    # No error should be raised
  end

  it "handles handler not found" do
     allow(GlobalID::Locator).to receive(:locate).with(handler_gid).and_return(nil)
     expect(Rails.logger).to receive(:error).with(/Handler not found/)

     perform_enqueued_jobs { Instruct::GenerateJob.perform_later(handler_gid, prompt) }
     # Job should likely be discarded or marked failed depending on config
  end

  # More complex test: Mock Orchestrator.start to yield events
  it "Orchestrator interaction (conceptual)" do
    # This requires more involved mocking of prompt.call within Orchestrator.start
    # or directly testing Orchestrator separately.

    # Example: Mocking Orchestrator.start to simulate tool call
    mock_orchestrator = instance_double(Instruct::Orchestrator)
    allow(Instruct::Orchestrator).to receive(:new).and_return(mock_orchestrator)

    # Simulate the orchestrator calling the handler's tool method
    expect(mock_orchestrator).to receive(:start) do
      # In a real scenario, Orchestrator would call prompt.call,
      # get a ToolCallRequest, and then call handler.test_tool
      handler.test_tool(param: "simulated")
    end

    perform_enqueued_jobs { Instruct::GenerateJob.perform_later(handler_gid, prompt) }

    # Verify handler state after simulated call
    reloaded_handler = GlobalID::Locator.locate(handler_gid) # Need to ensure state persists if handler is AR
    # expect(reloaded_handler.tool_called_args).to eq({ param: "simulated" }) # This won't work directly with mocks like this
    # Better to test Orchestrator logic separately.
  end
end

3. README Updates

# Instruct Gem README.md (Additions/Changes)

## Advanced Usage: Integrating with Rails Apps

`instruct` provides features for robust integration with Rails applications, enabling asynchronous processing, interaction with application models via tools, and a streamlined developer experience.

### Core Pattern: Handlers and Tools

The primary pattern involves creating "handler" objects (often ActiveRecord models or service objects) that manage specific AI interactions. These handlers define the tools the AI can use and implement the corresponding actions.

1.  **Include `Instruct::AiResponseHandler`:** Add this concern to your handler class. It provides a DSL for defining tools and standard callbacks.
2.  **Define Tools with `ai_tool`:** Use the class-level `ai_tool` DSL method to declare the functions the AI can call. Specify the tool's `name` (matching a public instance method), `description`, and input `parameters` (using JSON Schema).
3.  **Implement Tool Methods:** Create public instance methods on your handler class with names matching the tools defined via `ai_tool`. These methods contain your application logic (e.g., updating records, calling other services, broadcasting UI updates). They should accept arguments as keyword arguments based on the `parameters` definition and return a result (usually a String or Hash) to be sent back to the LLM.
4.  **Implement Callbacks (Optional):** Override methods like `handle_ai_text_chunk`, `handle_ai_finish`, `handle_ai_error`, `handle_tool_error` to customize behavior during the LLM interaction lifecycle.

```ruby
# app/models/assessment_updater.rb
class AssessmentUpdater
  include GlobalID::Identification # Needed for background jobs
  include Instruct::AiResponseHandler

  attr_reader :assessment

  def initialize(assessment)
    @assessment = assessment
    @id = assessment.id # For GlobalID::Identification
  end

  # --- Tool Definitions ---
  ai_tool :update_assessment_instructions,
    description: "Updates the main instructions for the assessment.",
    parameters: {
      type: "object", properties: {
        new_instructions: { type: "string", description: "The revised instructions text." }
      }, required: ["new_instructions"]
    }

  ai_tool :add_comment_to_assessment,
    description: "Adds a comment to the assessment discussion thread.",
    parameters: {
      type: "object", properties: {
        comment_text: { type: "string", description: "The content of the comment." }
      }, required: ["comment_text"]
    }

  # --- Tool Implementations ---
  def update_assessment_instructions(new_instructions:)
    if assessment.update(instructions: new_instructions)
      # Example: Broadcast Turbo Stream update for the UI
      assessment.broadcast_replace_to assessment, :instructions_field,
        target: ActionView::RecordIdentifier.dom_id(assessment, :instructions_field),
        partial: "assessments/form_instructions_field",
        locals: { assessment: assessment }
      "Assessment instructions updated successfully." # Result for LLM
    else
      "Failed to update instructions: #{assessment.errors.full_messages.join(', ')}"
    end
  end

  def add_comment_to_assessment(comment_text:)
    comment = assessment.comments.create(author: "AI Assistant", content: comment_text)
    if comment.persisted?
       # Example: Broadcast Turbo Stream append
       comment.broadcast_append_to assessment, :comments,
         target: ActionView::RecordIdentifier.dom_id(assessment, :comments_container),
         partial: "comments/comment",
         locals: { comment: comment }
      "Comment added."
    else
      "Failed to add comment: #{comment.errors.full_messages.join(', ')}"
    end
  end

  # --- Optional Callbacks ---
  def handle_ai_finish(reason)
    Rails.logger.info "AssessmentUpdater finished for assessment #{assessment.id}: #{reason}"
  end

  def handle_tool_error(tool_name, exception, arguments)
     Rails.logger.error "AssessmentUpdater tool error in '#{tool_name}' for assessment #{assessment.id}: #{exception.message}"
     # Maybe add a comment indicating the tool failure
     add_comment_to_assessment(comment_text: "Sorry, I encountered an error trying to perform the action: #{tool_name}.")
  end
end

Asynchronous Generation with Instruct.generate_for

For most Rails integrations, especially those involving tool calls or potentially long-running generations, use the asynchronous helper Instruct.generate_for.

• Purpose: Simplifies triggering AI generation with a handler. It automatically retrieves tools from the handler, constructs the Instruct::Prompt, and enqueues the standard Instruct::GenerateJob.
• Usage: Call Instruct.generate_for(handler_instance, user_prompt:, system_prompt: nil, ...) from your controller, model, or service.
• Requirements:
  • The handler object must include Instruct::AiResponseHandler.
  • The handler object must include GlobalID::Identification (so it can be passed to the background job).
  • ActiveJob must be configured in your Rails application.

# Example in a Controller action
class AssessmentsController < ApplicationController
  def suggest_improvements
    assessment = Assessment.find(params[:id])
    handler = AssessmentUpdater.new(assessment) # Create the handler

    user_prompt = "Review the assessment instructions and suggest improvements using the available tools."

    # Trigger the background job
    Instruct.generate_for(handler, user_prompt: user_prompt)

    # Redirect or render response indicating generation has started
    flash[:notice] = "AI is generating suggestions..."
    redirect_to assessment_path(assessment)
  end
end

Background Job: Instruct::GenerateJob

This standard ActiveJob class orchestrates the AI interaction in the background.

• It locates the handler using GlobalID.
• It retrieves tool definitions from the handler.
• It instantiates Instruct::Orchestrator.
• It calls orchestrator.start, which executes prompt.call and dispatches events (tool calls, text chunks, finish/error) to the handler's methods and callbacks.
• Tool methods within the handler can trigger UI updates via Turbo Streams broadcasts.

You generally don't need to interact with this job directly; use Instruct.generate_for. You can configure its queue name and retry behavior like any other ActiveJob.

Direct prompt.call with Manual Handling

For synchronous use cases or when you need fine-grained control over the event stream without the Orchestrator, you can still use prompt.call directly with a block. If you pass tools: to gen(), prompt.call will yield Instruct::ToolCallRequest events when the LLM wants to use a tool. Your block is responsible for executing the tool and calling event.respond(result) or event.respond_error(message).

# (Reference example from instruct/README.md startLine:408 endLine:442)
handler = MyToolHandler.new
prompt = p.user("...") + gen(tools: handler.ai_tools_definitions)

prompt.call do |event|
  case event
  when Instruct::ToolCallRequest
    # ... find method, execute, call event.respond/respond_error ...
  when Hash # Assuming simple hash events for text/finish
    # ... handle event[:type] == :text_chunk or :finish ...
  end
end

This approach gives maximum flexibility but requires implementing the orchestration logic manually within the block. The Orchestrator + GenerateJob pattern is recommended for most Rails applications.

This comprehensive proposal provides a clear path forward, focusing on a clean developer experience within Rails while building on the existing `instruct` architecture. The next critical step is the deep implementation within `prompt.call` and the client adapters to handle the yielding and responding cycle for `ToolCallRequest


# Adaptor details:

Okay Pete, let's build out those adapters. This is where the rubber meets the road in handling the different LLM provider APIs for tool calls.

We'll start with a base adapter class and then implement one for each provider (OpenAI, Anthropic, Google Gemini).

**1. Base Adapter**

This defines the interface and can hold common logic later if needed.

```language=ruby:instruct/lib/instruct/adapters/base.rb
require 'json'
require 'instruct/tool_call_request'
# May need other requires like specific event types later

module Instruct
  module Adapters
    class Base
      # Executes the prompt against the specific LLM provider.
      # Must be implemented by subclasses.
      #
      # @param prompt_object [Instruct::Prompt] The prompt containing messages,
      #   generation parameters, and tool definitions.
      # @param &block [Proc] A block to yield events to (e.g.,
      #   ToolCallRequest, text chunks, finish events).
      # @return The final aggregated text response (or potentially more
      #   structured data depending on implementation).
      def call(prompt_object, &block)
        raise NotImplementedError, "#{self.class.name} must implement #call"
      end

      protected

      # Helper to safely parse JSON arguments from the LLM.
      # @param args_string [String] The JSON string from the LLM.
      # @return [Hash] Parsed arguments or an empty hash on error.
      def parse_arguments(args_string)
        return {} unless args_string && !args_string.empty?
        JSON.parse(args_string)
      rescue JSON::ParserError => e
        Instruct.err_logger.error("Failed to parse tool arguments JSON: #{e.message}\nInput: #{args_string}")
        {} # Return empty hash on parse failure
      end

      # Placeholder for yielding text chunks consistently
      def yield_text_chunk(content, block)
        return unless block && content && !content.empty?
        # Later, we might use a specific StreamEvent class
        block.call({ type: :text_chunk, content: content })
      end

      # Placeholder for yielding finish event consistently
      def yield_finish(reason, block)
        return unless block
        # Later, we might use a specific StreamEvent class
        block.call({ type: :finish, reason: reason })
      end
    end
  end
end

2. OpenAI Adapter

Handles the OpenAI API specifics for tool calls.

require 'openai' # Assuming the 'openai' gem is used
require 'instruct/adapters/base'

module Instruct
  module Adapters
    class OpenAI < Base
      # @param client [OpenAI::Client] An initialized OpenAI client.
      def initialize(client: nil)
        # Lazy load client if not provided
        @client = client || ::OpenAI::Client.new(access_token: ENV.fetch('OPENAI_API_KEY'))
        # Configure client for streaming responses
        @client.default_timeout = 120 # Example: Increase timeout
      end

      def call(prompt_object, &block)
        model = prompt_object.model || Instruct.default_model
        messages = build_openai_messages(prompt_object)
        tools = format_tools_for_openai(prompt_object.tools)
        tool_choice = tools.any? ? "auto" : nil # Only specify if tools exist

        final_response_text = ""
        current_messages = messages.dup # History for follow-up calls

        loop do # Loop to handle potential sequences of tool calls
          Instruct.logger.debug "OpenAI Request: Model=#{model}, Messages=#{current_messages.count}, Tools=#{tools.count}"

          response_handler = OpenAIStreamResponseHandler.new(
            client: @client,
            model: model,
            messages: current_messages,
            tools: tools.empty? ? nil : tools,
            tool_choice: tool_choice,
            &block # Pass the block for yielding events
          )

          # Process the stream, handle tool calls internally
          response_handler.process_stream

          # Accumulate text
          final_response_text << response_handler.full_text_content

          # Check if tool calls were made and responded to
          if response_handler.tool_calls_requested?
            # Add the assistant's request and the tool results to history
            current_messages << response_handler.assistant_message_with_calls
            current_messages.concat(response_handler.tool_result_messages)
            # Continue loop to send results back to OpenAI
            Instruct.logger.debug "OpenAI: Tool calls processed, continuing interaction."
          else
            # No tool calls, or final response received after tool calls
            Instruct.logger.debug "OpenAI: Interaction finished. Reason: #{response_handler.finish_reason}"
            break # Exit loop
          end
        end # End loop

        final_response_text
      end

      private

      def build_openai_messages(prompt_object)
        # Convert Instruct::Prompt structure to OpenAI's message format
        # Example: [{ role: "system", content: "..."}, { role: "user", content: "..."}]
        prompt_object.to_openai_messages # Assumes Prompt has this method
      end

      def format_tools_for_openai(tools_array)
        # Convert internal tool spec (JSON schema) to OpenAI's format
        # Example: [{ type: "function", function: { name:, description:, parameters: } }]
        return [] unless tools_array
        tools_array.map do |tool_spec|
          { type: "function", function: tool_spec[:function] }
        end
      end

      # --- Inner Class for Stream Handling ---
      class OpenAIStreamResponseHandler
        attr_reader :full_text_content, :finish_reason,
                    :tool_result_messages, :assistant_message_with_calls

        def initialize(client:, model:, messages:, tools:, tool_choice:, &block)
          @client = client
          @model = model
          @messages = messages
          @tools = tools
          @tool_choice = tool_choice
          @block = block # For yielding events (ToolCallRequest, text, finish)
          @full_text_content = ""
          @finish_reason = nil
          @pending_tool_calls = {} # { id => { name:, arguments: "" } }
          @assistant_message_with_calls = nil # Store assistant message requesting tools
          @tool_result_messages = [] # Store { role: 'tool', ... } messages
          @responded_tool_call_ids = Set.new
        end

        def tool_calls_requested?
          !@pending_tool_calls.empty?
        end

        def process_stream
          stream_params = {
            model: @model,
            messages: @messages,
            tools: @tools,
            tool_choice: @tool_choice,
            stream: true
          }.compact # Remove nil values

          @client.chat(parameters: stream_params) do |chunk, _bytesize|
            delta = chunk.dig("choices", 0, "delta")
            @finish_reason = chunk.dig("choices", 0, "finish_reason")

            # 1. Accumulate Tool Call Chunks
            if delta&.key?("tool_calls")
              delta["tool_calls"].each do |call_chunk|
                index = call_chunk["index"] # Should always be 0 for non-parallel
                id = call_chunk["id"]
                function = call_chunk["function"]

                if id # Start of a new tool call
                  @pending_tool_calls[id] ||= { name: "", arguments: "" }
                end
                # Append name/argument chunks
                @pending_tool_calls[id][:name] << function["name"] if function&.key?("name")
                @pending_tool_calls[id][:arguments] << function["arguments"] if function&.key?("arguments")
              end
              # Store the assistant message that contained the tool_calls request
              @assistant_message_with_calls ||= delta.compact # Store the delta itself? Or reconstruct? Needs care.
                                                              # Let's assume we store the final aggregated message later.
            end

            # 2. Accumulate Text Content
            if delta&.key?("content") && delta["content"]
              text_chunk = delta["content"]
              @full_text_content << text_chunk
              yield_text_chunk(text_chunk, @block)
            end

            # 3. Check for Finish
            if @finish_reason
              # Store the complete assistant message if it contained tool calls
              if @finish_reason == 'tool_calls' && @assistant_message_with_calls.nil?
                 # Need to reconstruct the full message from the final chunk if needed
                 # Example: chunk.dig("choices", 0, "message") might hold it
                 full_message = chunk.dig("choices", 0, "message")
                 @assistant_message_with_calls = full_message if full_message&.key?("tool_calls")
              end
              break # Stop processing stream chunks
            end
          end # End stream processing

          # If finished due to tool calls, yield ToolCallRequest events
          if @finish_reason == 'tool_calls'
            yield_tool_call_requests
          else
            # Normal finish or error
            yield_finish(@finish_reason, @block)
          end
        end

        private

        def yield_tool_call_requests
          @pending_tool_calls.each do |id, call_info|
            next if @responded_tool_call_ids.include?(id) # Skip if already handled (safety)

            arguments = parse_arguments(call_info[:arguments])

            # Create the response block specific to OpenAI
            response_block = ->(tool_call_id:, result:, error:) {
              # Format the result message for OpenAI API
              result_message = {
                role: "tool",
                tool_call_id: tool_call_id,
                # Content should be a string, per OpenAI docs
                content: error ? "Error: #{result}" : result.to_s
              }
              @tool_result_messages << result_message
              @responded_tool_call_ids.add(tool_call_id)
              Instruct.logger.debug "OpenAI: Prepared tool result for ID #{tool_call_id}"
            }

            # Yield the common ToolCallRequest object
            tool_request = ToolCallRequest.new(
              id: id,
              tool_name: call_info[:name],
              arguments: arguments,
              response_block: response_block
            )
            @block.call(tool_request) if @block

            # Ensure the block called respond/respond_error
            unless tool_request.responded?
              Instruct.err_logger.error "OpenAI Adapter: Block did not respond to ToolCallRequest (ID: #{id})"
              # Automatically respond with an error to avoid hanging
              tool_request.respond_error("Handler did not respond to tool call.")
            end
          end
        end

        # Delegate helpers from Base if needed
        include Instruct::Helpers::YieldingHelpers # Define this module if needed
        def parse_arguments(args_string)
          # Use Base class helper or override
          super(args_string)
        end
      end # End OpenAIStreamResponseHandler
    end
  end
end

5. Integrating Adapters into Instruct::Prompt

Now, Instruct::Prompt needs to select and use the correct adapter.

# ... other requires ...
require 'instruct/adapters/openai'
require 'instruct/adapters/anthropic'
require 'instruct/adapters/google'

module Instruct
  class Prompt
    # ... attributes like @elements, @model, @tools ...

    # Add gen! method to store generation parameters including tools
    def gen!(model: nil, tools: nil, **other_params)
      @model = model if model
      @tools = tools if tools # Store tool definitions from AiResponseHandler
      @generation_params = other_params # Store other params if needed
      self # Return self for chaining
    end

    def call(&block)
      adapter = select_adapter
      # Delegate the actual call to the selected adapter
      adapter.call(self, &block)
    end

    # --- Conversion methods needed by adapters ---
    # These need to be implemented based on how @elements stores messages

    def system_prompt
      # Find the system message in @elements
      @elements.find { |el| el[:type] == :system }&.dig(:content)
    end

    def to_openai_messages
      # Convert @elements to OpenAI's [{ role:, content: }] format
      @elements.map do |el|
         # Handle different element types (:system, :user, :assistant)
         # Ensure correct role mapping
         { role: el[:type].to_s, content: el[:content] } # Simplified example
      end.compact
    end

    def to_anthropic_messages
      # Convert @elements to Anthropic's format (excluding system)
      @elements.select { |el| el[:type] != :system }.map do |el|
         { role: el[:type].to_s, content: el[:content] } # Simplified example
      end.compact
    end

    def to_google_contents
      # Convert @elements to Google's [{ role:, parts: [{ text: }] }] format
      @elements.map do |el|
        role = (el[:type] == :assistant) ? "model" : "user" # Map roles
        { role: role, parts: [{ text: el[:content] }] } # Simplified example
      end.compact
    end

    # --- Adapter Selection ---
    private

    def select_adapter
      model_name = @model || Instruct.default_model
      # Determine provider based on model name convention or configuration
      case model_name
      when /^gpt-/
        Instruct::Adapters::OpenAI.new # Pass client if configured globally
      when /^claude-/
        Instruct::Adapters::Anthropic.new
      when /^gemini-/
        Instruct::Adapters::Google.new
      else
        # Default or raise error
        Instruct.logger.warn "Unknown model provider for '#{model_name}', defaulting to OpenAI."
        Instruct::Adapters::OpenAI.new # Or raise error
      end
    end
  end
end

Key Considerations & Next Steps:

1. Client Initialization: Adapters currently initialize clients with environment variables. This should be configurable globally or via dependency injection.
2. Error Handling: The error handling within stream processing is basic. More specific error types and potentially yielding error events would be beneficial.
3. Prompt Structure: The to_openai_messages, to_anthropic_messages, to_google_contents methods in Prompt depend heavily on how prompts are internally structured (@elements). These need careful implementation.
4. Streaming Robustness: Parsing streaming responses, especially Google's line-delimited JSON, needs to be very robust against partial data or malformed chunks.
5. Google Function Response: The exact structure required by Google in the functionResponse part needs verification against their latest API documentation.
6. Dependencies: Ensure the necessary gems (openai, anthropic, googleauth, httparty, etc.) are added to the instruct.gemspec.
7. Testing: Each adapter needs thorough testing, mocking the API calls and verifying that ToolCallRequest objects are yielded correctly and that the response_block formats data as expected for the subsequent API call.

This provides a solid foundation for multi-provider tool support using the Adapter pattern, keeping the core logic clean and isolating provider specifics.