TOML Execution Framework Specification for tomlToChain

Version: 1.0

Last Updated: March 2025

Table of Contents

1. Overview
   • 1.1. Key Features
2. Core Components
   • 2.1. Node
     • 2.1.1. Node Structure
3. Node Types
   • 3.1. Action Node
     • 3.1.1. Action Structure
     • 3.1.2. Chat Messages Structure
   • 3.2. Looper Node
     • 3.2.1. Looper Structure
     • 3.2.2. Looping Expansion
   • 3.3. Reruner Node
     • 3.3.1. Reruner Node Structure
   • 3.4. End Node
     • 3.4.1. End Node Structure
4. Global Configuration
   • 4.1. Config Block
   • 4.2. Files Block
   • 4.3. Variables Block
   • 4.4. Imports Block
   • 4.5. Tools Block
5. Execution Flow
6. Example TOML File

---

1. Overview

This specification defines a structured TOML format for configuring and executing an LLM-based chain in Python. Each TOML file represents an execution plan consisting of Nodes, which can be standalone actions, looping structures, or final outputs.

The framework is designed to be modular, composable, and extensible, ensuring scalability for future enhancements.

1.1. Key Features

• Modular Design: Nodes can be combined in various ways to create complex execution flows.
• Dynamic Execution: Conditional logic and looping structures allow for flexible processing.
• Global Configuration: Centralized settings for model selection, parameters, and variable initialization.
• File-Based Inputs: External files can be loaded into variables for processing.
• Hardcoded Variables: Static values can be defined and referenced throughout the execution.
• Import Mechanism: Allows importing external TOML files to reuse common definitions.

1.2. To run your TOML file (CLI)

• Step 1: Install the required Python packages using pip install -r requirements.txt.
• Step 2: Run the cli.py script with -h to see options.

python cli.py -h

1.2. To run your TOML file (Package)

• Step 1: Install the required Python packages using pip install -r requirements.txt.
• Step 2: Place the tomlToChain package where you wish.

Example Usage

from tomlToChain.Builder import Builder
from dotenv import load_dotenv

load_dotenv()

toml_file = "./test.toml"

builder = Builder(toml_file=toml_file)
chain = builder.get_chain()
outputs = chain.run_chain()
print(outputs)

---

2. Core Components

2.1. Node

A Node is the fundamental execution unit in this framework. Every node follows a shared structure while allowing specialized fields for different node types.

2.1.1. Node Structure

Each node must define the following core properties:

Field	Type	Required	Description
name	string	✅	Unique identifier for the node
description	string	✅	A brief description of the node's function
requires	list<string>	❌	List of node names that must execute before this node
run_if	list<Condition>	❌	Conditions that must be met for execution
model	string	❌	Override the default model for this node
llm_params	object	❌	Model parameters, passed as **kwargs
use_ollama	bool	❌	Use ollama for local models

Each node type expands on this foundation with additional properties.

2.1.2. Condition Structure

Conditions are used to determine if a node should execute based on variable values.

Field	Type	Required	Description
variable	string	✅	Name of the variable to check
equals	any	❌	Value/Variable that the variable must equal
not_equals	any	❌	Value/Variable that the variable must not equal
greater_than	number	❌	Value/Variable that the variable must be greater than
less_than	number	❌	Value/Variable that the variable must be less than
greater_or_equal	number	❌	Value/Variable that the variable must be greater or equal to
less_or_equal	number	❌	Value/Variable that the variable must be less or equal to
contains	any	❌	Value/Variable that the variable must contain
not_contains	any	❌	Value/Variable that the variable must not contain

---

3. Node Types

3.1. Action Node

An Action Node represents a discrete task that interacts with the LLM.

3.1.1. Action Structure

Field	Type	Required	Description
input	list<string>	✅	List of variable names provided as input
output	string	✅	Name of the variable that will store the output
output_format	enum["raw", "text", "json", "list", "number"]	✅	Format in which the output should be parsed
prompt_type	enum["text", "chat"]	✅	Defines whether the prompt is plain text or a structured chat
prompt	string or list<ChatMessage>	✅	The prompt text (for text) or a structured chat format
tools	list<string>	❌	A list of tool names that the action can use

3.1.2. Chat Messages Structure

If prompt_type is "chat", the prompt must be a list of structured chat messages:

Field	Type	Required	Description
role	string	✅	Role of the message (system, user, etc.)
content	string	✅	Message content

---

3.2. Looper Node

A Looper Node allows iteration over a list variable, executing a set of actions for each element.

3.2.1. Looper Structure

Field	Type	Required	Description
list_variable	string	✅	The variable containing a list to iterate over
iterator	string	✅	The name of the variable representing the current iteration item
actions	list<Action>	✅	The list of actions to execute per iteration
output	string	✅	Variable to store results from all iterations

3.2.2. Looping Expansion

Within actions inside loopers, the {iterator} token can be used dynamically in any string field, enabling context-aware modifications.

Example:

[[loopers.actions]]
name = "evaluate_{competency}"
output = "{competency}_score"

---

3.3. Reruner Node

A Reruner Node can rerun any amount of Nodes in the chain.

3.3.1. Reruner Node Structure

Field	Type	Required	Description
targets	list<string>	✅	List of node names that will be rerun
max_reruns	number	❌	Max number of reruns

---

3.4. End Node

An End Node consolidates outputs from previous nodes into a final result.

3.4.1. End Node Structure

Field	Type	Required	Description
input	list<string>	✅	List of variable names that provide input
output	string	✅	Name of the variable storing the final output
output_format	enum["raw", "text", "json", "list", "number"]	✅	Defines how the final result should be formatted
prompt_type	enum["text", "chat"]	✅	Determines the type of LLM interaction
prompt	string or list<ChatMessage>	✅	The final prompt used to generate the summary/output
tools	list<string>	❌	A list of tool names that the action can use

---

4. Global Configuration

4.1. Config Block

Defines global settings for the execution environment.

Field	Type	Required	Description
model	string	✅	The default LLM model to use
llm_params	object	❌	Default parameters for the model
use_ollama	bool	❌	Use ollama for local models

Example:

[config]
model = "gpt-4"
llm_params = { temperature = 0.7, max_tokens = 1000 }
use_ollama = false

---

4.2. Files Block

Defines files that should be loaded into variables.

Field	Type	Required	Description
variable_name	string	❌	Path to the file

Example:

[files]
cv_text = "cv_oscargomez.txt"
competencies = "competences_bilingual_indexed.json"

---

4.3. Variables Block

Defines hardcoded variables.

Field	Type	Required	Description
variable	any	❌	Static values that can be referenced

Example:

[variables]
competency_codes = ["C01", "C02", "C03"]

---

4.4. Imports Block

Allows importing other TOML files.

Field	Type	Required	Description
imports	list<string>	❌	List of TOML files to import

Example:

imports = ["shared_config.toml"]

4.5. Tools Block

The [tool_imports] block allows importing external Python functions (tools) that can be called inside Actions.

Field	Type	Required	Description
tool_name	string	✅	Name of the tool that can be referenced in an Action
file_path	string	✅	Path to the Python file where the tool is defined

Example: Tool Definition

[tool_imports]
get_rating = "tools.py"

This registers a tool named "get_rating" that is located in "tools.py".

---

5. Execution Flow

1. Configuration Loading:

   • The framework first loads the [config] block.
   • It applies global model parameters and imports external files.

2. Variable Initialization:

   • Files and hardcoded variables are loaded.

3. Node Execution:

   • Nodes execute in dependency order, respecting requires and run_if.
   • Loopers iterate over lists, running their contained actions dynamically.

4. Final Consolidation:

   • The [end] node gathers results from all nodes and generates the final output.

6. Example TOML File

# ===================================================================
# TOML Execution Framework Template for `tomlToChain`
# Version: 1.0
# ===================================================================

# ---------------------------------------------------------------
# 1. Global Configuration
# ---------------------------------------------------------------
[config]
model = "gpt-4"
llm_params = { temperature = 0.7, max_tokens = 1000 }
use_ollama = false

# ---------------------------------------------------------------
# 2. File-Based Variables (Loaded from External Files)
# ---------------------------------------------------------------
[files]
document_text = "input_document.txt"
reference_data = "reference.json"

# ---------------------------------------------------------------
# 3. Hardcoded Variables (Accessible Throughout Execution)
# ---------------------------------------------------------------
[variables]
threshold = 0.85
categories = ["Science", "Technology", "Philosophy"]

# ---------------------------------------------------------------
# 4. Imports (External TOML Files)
# ---------------------------------------------------------------
[imports]
imports = ["common_definitions.toml", "additional_rules.toml"]

# ---------------------------------------------------------------
# 5. Actions: Individual Processing Steps
# ---------------------------------------------------------------

[[actions]]
name = "extract_keywords"
description = "Extracts key topics from the document text."
input = ["document_text"]
output = "keywords"
output_format = "json"
prompt_type = "chat"

[[actions.prompt]]
role = "system"
content = "You are an expert in text analysis and keyword extraction."

[[actions.prompt]]
role = "user"
content = "Extract a structured list of keywords from the following text: {document_text}"

[[actions]]
name = "categorize_text"
description = "Classifies the document based on predefined categories."
requires = ["extract_keywords"]
input = ["keywords", "categories"]
output = "category_labels"
output_format = "json"
prompt_type = "chat"

[[actions.prompt]]
role = "system"
content = "You are an AI trained to categorize text based on provided themes."

[[actions.prompt]]
role = "user"
content = "Based on these extracted keywords: {keywords}, classify the document into one or more of these categories: {categories}."

# ---------------------------------------------------------------
# 6. Loopers: Iterating Over Data
# ---------------------------------------------------------------

[[loopers]]
name = "score_text"
description = "Iterates over categories to assign a confidence score."
list_variable = "categories"
iterator = "current_category"
requires = ["categorize_text"]
output = "category_scores"

[[loopers.actions]]
name = "score_{current_category}"
description = "Evaluates the document's confidence level for {current_category}."
requires = ["categorize_text"]
input = ["document_text", "current_category"]
output = "{current_category}_score"
output_format = "json"
prompt_type = "chat"

[[loopers.actions.prompt]]
role = "system"
content = "You analyze text and provide confidence scores for given categories."

[[loopers.actions.prompt]]
role = "user"
content = """
Evaluate the document for the category: {current_category}.
- Score it from 0 to 1.
- Justify the score with evidence from the document.
"""

# ---------------------------------------------------------------
# 7. Conditional Execution (Dynamic Routing)
# ---------------------------------------------------------------

[[actions]]
name = "high_confidence_summary"
description = "Generate a summary if confidence is above the threshold."
requires = ["score_text"]
input = ["document_text"]
output = "high_confidence_summary"
output_format = "text"
prompt_type = "chat"
run_if = [{ "variable" = "threshold", "greater_or_equal" = 0.85 }]

[[actions.prompt]]
role = "system"
content = "You generate summaries for well-classified documents."

[[actions.prompt]]
role = "user"
content = "Summarize the document because it is well-classified: {document_text}."

[[actions]]
name = "low_confidence_feedback"
description = "Provide feedback if confidence is too low."
requires = ["score_text"]
input = ["document_text"]
output = "low_confidence_feedback"
output_format = "text"
prompt_type = "chat"
run_if = [{ "variable" = "threshold", "less_than" = 0.85 }]

[[actions.prompt]]
role = "system"
content = "You provide feedback on poorly classified documents."

[[actions.prompt]]
role = "user"
content = "Provide feedback on how to improve the classification accuracy for: {document_text}."

# ---------------------------------------------------------------
# 8. End Node: Final Processing Step
# ---------------------------------------------------------------

[end]
name = "final_output"
description = "Compile the final report with classification and scoring."
requires = ["high_confidence_summary", "low_confidence_feedback"]
input = ["category_labels", "category_scores", "high_confidence_summary", "low_confidence_feedback"]
output = "final_report"
output_format = "text"
prompt_type = "text"
prompt = """
Generate a structured final report summarizing:
- Categories assigned: {category_labels}
- Confidence scores: {category_scores}
- Summary (if applicable): {high_confidence_summary}
- Feedback (if applicable): {low_confidence_feedback}
"""