Tips Improvement - #20
Conversation
📝 WalkthroughWalkthroughRefactors tip generation to use structured Pydantic models and constrained decoding, updates LLM settings/defaults, rewrites the tip prompt template, enriches trajectory/guideline metadata in Phoenix sync and MCP, and adds an errors field to SyncResult. Documentation updated accordingly. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Parser as Parser\n(parse_openai_agents_trajectory)
participant Prompter as Prompt Builder
participant DecoderCheck as Litellm\nsupports_response_schema
participant LLM as LLM Model
participant Validator as Pydantic\nTipGenerationResponse
Client->>Parser: send messages[]
Parser-->>Prompter: trajectory dict
Prompter->>DecoderCheck: check schema support
alt schema supported
DecoderCheck-->>Prompter: true (supports_response_schema)
Prompter->>LLM: generate(prompt, response_schema=TipGenerationResponse)
LLM-->>Prompter: constrained JSON
else schema not supported
DecoderCheck-->>Prompter: false
Prompter->>LLM: generate(prompt)
LLM-->>Prompter: JSON/text
Prompter->>Prompter: clean/normalize response
end
Prompter->>Validator: parse response as TipGenerationResponse
Validator-->>Prompter: list[Tip]
Prompter-->>Client: return list[Tip]
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Comment |
| Examine the following agent trajectory. Extract guidelines or tips that can be used in the future to optimize the trajectory. | ||
| This includes remembering sequences of tool calls, or entities that will make the reasoning more efficient and avoid errors. | ||
| You are analyzing an AI agent's execution trajectory to extract actionable tips. | ||
|
|
||
| The tips should be short sentences that guide future iterations of the agent to perform better or faster, avoid errors, or learn from experience. | ||
| Generate as many tips as you think is needed. Output as a JSON formatted list of strings. | ||
| # Task Information | ||
| **Task:** {{task_instruction}} | ||
| **Status:** UNKNOWN | ||
| **Steps Taken:** {{num_steps}} | ||
|
|
||
| {{markdown_trajectory}} No newline at end of file | ||
| # Agent Trajectory | ||
| {{trajectory_summary}} | ||
|
|
||
| # Your Task | ||
| Extract 3-5 actionable tips from this trajectory that would help AI agents perform similar tasks better. | ||
|
|
||
| **Guidelines:** | ||
| 1. Focus on patterns that worked or mistakes that were made | ||
| 2. Be specific to what you observed in this trajectory | ||
| 3. Each tip should have: | ||
| - Clear description of what to do (or avoid) | ||
| - Why it matters | ||
| - When to apply it | ||
|
|
||
| {% if not constrained_decoding_supported %} | ||
| **Output Format (JSON):** | ||
| ```json | ||
| { | ||
| "tips": [ | ||
| { | ||
| "content": "Clear, actionable tip", | ||
| "rationale": "Why this tip helps", | ||
| "category": "strategy|recovery|optimization", | ||
| "trigger": "When to apply this tip" | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| Generate tips now. Return ONLY the JSON, no other text. | ||
| {% endif %} No newline at end of file |
There was a problem hiding this comment.
This is where most of the changes are.
| trajectory_data = parse_openai_agents_trajectory(messages) | ||
| prompt = Template(prompt_file.read_text()).render( | ||
| task_instruction=trajectory_data["task_instruction"], | ||
| num_steps=trajectory_data["num_steps"], | ||
| trajectory_summary=trajectory_data["trajectory_summary"], | ||
| constrained_decoding_supported=constrained_decoding_supported, | ||
| ) | ||
|
|
There was a problem hiding this comment.
The new tips prompt made by jayaram wants us to preprocess the openai trajectory directly. This implementation differs slightly from his in that the data format we expect is different from the one he used. Apparently OpenAI's format is not a standard.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Fix all issues with AI agents
In `@kaizen/llm/tips/tips.py`:
- Around line 73-79: The agent_steps entries are being appended with "type":
"actions" but the formatter checks for "type" == "action", causing action steps
to be skipped; update the append to use "type": "action" in the agent_steps
construction (the block that builds the dict with "content":
f"{assistant_response['function']['name']}({args_display})", "raw":
assistant_response) so it matches the formatter's check, or alternatively change
the formatter's condition to check for "actions" — prefer making the append use
"action" for consistency with the existing formatting logic.
- Around line 60-71: The code assigns args_str with a trailing comma which makes
args_str a tuple; change the assignment so args_str =
assistant_response["function"]["arguments"] (no comma) so isinstance(args_str,
str) works, then parse with json.loads(args_str) inside a try/except that
catches JSONDecodeError and TypeError, set args to the parsed dict when
successful, and build args_display from args.items() if args is a dict,
otherwise set args_display = str(args_str) to avoid assigning a tuple; reference
the symbols args_str, assistant_response["function"]["arguments"], json.loads,
JSONDecodeError, TypeError, args, and args_display when making the change.
In `@kaizen/schema/tips.py`:
- Around line 5-9: The Tip model's trigger attribute currently uses Field("When
to apply this tip") which sets that string as the default rather than a
description; update the Tip class so trigger is declared with
Field(description="When to apply this tip") (or if you intended no default make
trigger: Optional[str] = None with Field(description=...)) to match the pattern
used for content and rationale and avoid an unintended default value.
🧹 Nitpick comments (5)
kaizen/frontend/mcp/mcp_server.py (1)
88-110: Consider error handling for partial success scenarios.If
generate_tips()fails (LLM error, timeout, etc.) on line 93, the trajectory entities are already persisted (lines 88-92), but no tips are stored. This leaves the system in a partial state where the trajectory exists without generated tips.Depending on the desired behavior, consider:
- Wrapping tip generation in try/except to return partial results with a warning
- Using a transaction-like pattern where both succeed or fail together
kaizen/llm/tips/tips.py (3)
80-87: Consider more graceful handling of unknown content types.While raising exceptions ensures strict validation, this could cause entire trajectories to fail processing due to a single unexpected content type. Consider logging a warning and skipping unknown types instead, or at minimum ensure callers handle
KaizenExceptionappropriately.The static analysis hint about long exception messages (TRY003) is a minor style concern - the current messages are informative for debugging.
159-159: Add error handling for JSON parsing and validation failures.This line can raise
JSONDecodeError(if LLM returns malformed JSON) orpydantic.ValidationError(if schema doesn't match). Consider wrapping in try-except to provide clearer error messages or graceful degradation.♻️ Proposed improvement
- return TipGenerationResponse.model_validate(json.loads(clean_response)).tips + try: + return TipGenerationResponse.model_validate(json.loads(clean_response)).tips + except (JSONDecodeError, ValidationError) as e: + raise KaizenException(f"Failed to parse tip generation response: {e}") from eAdd
ValidationErrorimport:from pydantic import ValidationError
135-148: Global state mutation may cause race conditions in concurrent scenarios.Setting
litellm.enable_json_schema_validationglobally (lines 136, 148) is not thread-safe. Ifgenerate_tipsis called concurrently from multiple threads, the setting toggled by one thread could affect another. LiteLLM does not provide a context manager or per-call parameter to avoid this. Consider protecting the global setting mutation with a threading lock, or restructure the code to avoid toggling this flag at runtime.kaizen/sync/phoenix_sync.py (1)
88-92: Consider making the entity search limit configurable.The hardcoded
limit=10000could be insufficient for large deployments with many processed trajectories, potentially causing re-processing of older spans.💡 Suggestion
Consider making this configurable via settings or implementing pagination to handle arbitrarily large numbers of processed spans:
+ PROCESSED_SPAN_LIMIT = 10000 # Class constant or from config + def _get_processed_span_ids(self) -> set[str]: """Get span_ids that have already been processed.""" try: entities = self.client.search_entities( namespace_id=self.namespace_id, filters={"type": "trajectory"}, - limit=10000, + limit=self.PROCESSED_SPAN_LIMIT, )
📜 Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
README.mdkaizen/config/llm.pykaizen/frontend/mcp/mcp_server.pykaizen/llm/tips/prompts/generate_tips.jinja2kaizen/llm/tips/tips.pykaizen/schema/tips.pykaizen/sync/phoenix_sync.pypyproject.toml
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2026-01-15T19:05:07.833Z
Learnt from: CR
Repo: AgentToolkit/kaizen PR: 0
File: demo/workdir/CLAUDE.md:0-0
Timestamp: 2026-01-15T19:05:07.833Z
Learning: Before carrying out any task, always read the list of guidelines from the guidelines MCP server. Be sure to follow these guidelines to help complete the task. It's very important.
Applied to files:
kaizen/frontend/mcp/mcp_server.py
🧬 Code graph analysis (2)
kaizen/llm/tips/tips.py (3)
kaizen/utils/utils.py (1)
clean_llm_response(3-14)kaizen/schema/exceptions.py (1)
KaizenException(6-8)kaizen/schema/tips.py (2)
TipGenerationResponse(12-13)Tip(5-9)
kaizen/sync/phoenix_sync.py (1)
kaizen/schema/core.py (1)
Entity(17-21)
🪛 Ruff (0.14.11)
kaizen/llm/tips/tips.py
36-36: Avoid specifying long messages outside the exception class
(TRY003)
81-83: Avoid specifying long messages outside the exception class
(TRY003)
85-87: Avoid specifying long messages outside the exception class
(TRY003)
🔇 Additional comments (12)
pyproject.toml (1)
42-46: LGTM!The simplified marker remains functional with pytest. The README.md provides adequate documentation about the Phoenix tests purpose.
README.md (1)
56-64: Documentation updates look good.The configuration table accurately reflects the updated defaults in
LLMSettingsand provides helpful guidance for users configuring custom LLM providers with proxy setups.kaizen/schema/tips.py (1)
12-13: LGTM!Clean container model for aggregating tips.
kaizen/frontend/mcp/mcp_server.py (3)
25-29: LGTM!Good defensive pattern to ensure the namespace exists before operations.
32-56: LGTM!The namespace pre-check is a good addition to prevent errors on first use.
112-116: LGTM!The return correctly queries for the saved trajectory entities.
kaizen/config/llm.py (1)
5-13: LGTM!The updated defaults (
gpt-4owithout provider prefix) and the newcustom_llm_providerfield provide flexibility for both direct OpenAI usage and proxy configurations. The README documentation clearly explains the usage pattern.kaizen/llm/tips/prompts/generate_tips.jinja2 (2)
1-21: LGTM - Clear prompt structure with appropriate placeholders.The template structure aligns well with the
parse_openai_agents_trajectoryreturn values used intips.py. The hardcoded "UNKNOWN" status is acceptable as a placeholder, though consider making it a template variable if task status becomes available in the future.
22-38: Conditional logic is correct for fallback JSON formatting.The
{% if not constrained_decoding_supported %}block correctly provides explicit JSON format instructions only when the LLM provider doesn't support constrained decoding (JSON schema validation). When constrained decoding is supported, the schema enforcement handles the output format automatically.kaizen/sync/phoenix_sync.py (3)
28-35: LGTM - Good addition of error tracking.The
errors: list[str]field enables proper error reporting from the sync operation. This integrates well with the error collection in thesyncmethod (lines 385, 422).
330-344: LGTM - Rich metadata for guideline entities.The tip entity construction properly extracts structured fields from the
Tipmodel (content,category,rationale,trigger) and includes source traceability. This aligns well with theEntityschema and the newTipmodel fromkaizen/schema/tips.py.
407-422: Appropriate error handling for resilience.The try-except block ensures a single failed trajectory doesn't halt the entire sync operation. Errors are both logged and collected for the
SyncResult, providing good observability.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
| args_str = (assistant_response["function"]["arguments"],) | ||
| try: | ||
| args = ( | ||
| json.loads(args_str) | ||
| if isinstance(args_str, str) | ||
| else args_str | ||
| ) | ||
| args_display = ", ".join( | ||
| f"{k}={json.dumps(v)}" for k, v in args.items() | ||
| ) | ||
| except JSONDecodeError: | ||
| args_display = args_str |
There was a problem hiding this comment.
Bug: Trailing comma creates a tuple instead of a string.
Line 60 has a trailing comma that creates a single-element tuple instead of assigning the string value. This causes:
isinstance(args_str, str)on line 64 to always beFalsejson.loads(args_str)to raiseTypeError(notJSONDecodeError), which is uncaught- If it somehow reached line 71,
args_displaywould be assigned a tuple
🐛 Proposed fix
- args_str = (assistant_response["function"]["arguments"],)
+ args_str = assistant_response["function"]["arguments"]
try:
args = (
json.loads(args_str)
if isinstance(args_str, str)
else args_str
)
args_display = ", ".join(
f"{k}={json.dumps(v)}" for k, v in args.items()
)
- except JSONDecodeError:
+ except (JSONDecodeError, TypeError):
args_display = args_str🤖 Prompt for AI Agents
In `@kaizen/llm/tips/tips.py` around lines 60 - 71, The code assigns args_str with
a trailing comma which makes args_str a tuple; change the assignment so args_str
= assistant_response["function"]["arguments"] (no comma) so isinstance(args_str,
str) works, then parse with json.loads(args_str) inside a try/except that
catches JSONDecodeError and TypeError, set args to the parsed dict when
successful, and build args_display from args.items() if args is a dict,
otherwise set args_display = str(args_str) to avoid assigning a tuple; reference
the symbols args_str, assistant_response["function"]["arguments"], json.loads,
JSONDecodeError, TypeError, args, and args_display when making the change.
| if constrained_decoding_supported: | ||
| litellm.enable_json_schema_validation = True | ||
| clean_response = ( | ||
| completion( | ||
| model=llm_settings.tips_model, | ||
| messages=[{"role": "user", "content": prompt}], | ||
| response_format=TipGenerationResponse, | ||
| custom_llm_provider=llm_settings.custom_llm_provider, | ||
| ) | ||
| .choices[0] | ||
| .message.content | ||
| ) | ||
| else: | ||
| litellm.enable_json_schema_validation = False | ||
| response = ( | ||
| completion( | ||
| model=llm_settings.tips_model, | ||
| messages=[{"role": "user", "content": prompt}], | ||
| custom_llm_provider=llm_settings.custom_llm_provider, | ||
| ) | ||
| .choices[0] | ||
| .message.content | ||
| ) | ||
| clean_response = clean_llm_response(response) |
There was a problem hiding this comment.
We use constrained decoding if available (litellm's "json_mode") which guarantees a JSON output, ensuring a valid response without requiring retries or other messy error handling/retry logic.
| class Tip(BaseModel): | ||
| content: str = Field(description="Clear, actionable tip") | ||
| rationale: str = Field(description="Why this tip helps") | ||
| category: Literal["strategy", "recovery", "optimization"] | ||
| trigger: str = Field("When to apply this tip") | ||
|
|
||
|
|
||
| class TipGenerationResponse(BaseModel): | ||
| tips: list[Tip] |
There was a problem hiding this comment.
I'm currently unconvinced that we need this much structure for tips. A natural language sentence should be enough?
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@kaizen/llm/tips/tips.py`:
- Around line 134-147: The function generate_tips toggles the module-level flag
litellm.enable_json_schema_validation per-call (controlled by
constrained_decoding_supported), which is not thread-safe; either remove
per-call mutation and ensure the flag is set once at process startup (set
litellm.enable_json_schema_validation based on desired global behavior) or
serialize access by guarding the toggling + call to completion(...,
response_format=TipGenerationResponse) with a thread/process-safe lock (e.g., a
module-level threading.Lock) so only one request mutates the flag at a time;
update generate_tips to reference constrained_decoding_supported and
completion/TipGenerationResponse without directly flipping the global flag per
request, or document that the function is single-threaded if you choose not to
change behavior.
♻️ Duplicate comments (1)
kaizen/llm/tips/tips.py (1)
61-70: Bug: UncaughtAttributeErrorif arguments JSON is not a dict.If
args_strcontains valid JSON that isn't a dictionary (e.g., an array"[1,2]"or primitive"null"),json.loadssucceeds butargs.items()on line 64 raisesAttributeError, which isn't caught.🐛 Proposed fix
args_str = assistant_response["function"]["arguments"] try: - args: dict = json.loads(args_str) - args_display = ", ".join( - f"{k}={json.dumps(v)}" for k, v in args.items() - ) + args = json.loads(args_str) + if isinstance(args, dict): + args_display = ", ".join( + f"{k}={json.dumps(v)}" for k, v in args.items() + ) + else: + args_display = json.dumps(args) function_description = f"{assistant_response['function']['name']}({args_display})" - except JSONDecodeError: + except (JSONDecodeError, TypeError): function_description = ( f"{assistant_response['function']['name']}({args_str})" )
🧹 Nitpick comments (1)
kaizen/llm/tips/tips.py (1)
158-158: Add defensive handling for potentialNoneresponse content.If the LLM returns a response where
.message.contentisNone,json.loads(clean_response)will raiseTypeError. While rare, this can happen with certain API error conditions.🔧 Proposed fix
+ if not clean_response: + raise KaizenException("LLM returned empty response content") return TipGenerationResponse.model_validate(json.loads(clean_response)).tips
📜 Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
kaizen/llm/tips/tips.py
🧰 Additional context used
🧬 Code graph analysis (1)
kaizen/llm/tips/tips.py (3)
kaizen/utils/utils.py (1)
clean_llm_response(3-14)kaizen/schema/exceptions.py (1)
KaizenException(6-8)kaizen/schema/tips.py (2)
TipGenerationResponse(12-13)Tip(5-9)
🪛 Ruff (0.14.11)
kaizen/llm/tips/tips.py
36-36: Avoid specifying long messages outside the exception class
(TRY003)
80-82: Avoid specifying long messages outside the exception class
(TRY003)
84-86: Avoid specifying long messages outside the exception class
(TRY003)
🔇 Additional comments (2)
kaizen/llm/tips/tips.py (2)
88-110: LGTM!The step formatting logic with the 50-step limit, content truncation, and structured return dictionary is well-implemented. The "observation" type handling provides forward compatibility.
113-132: LGTM on constrained decoding detection.The logic for detecting
response_formatsupport andresponse_schemasupport using litellm's helper functions is well-structured. The trajectory parsing and prompt rendering integration looks correct.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
| if constrained_decoding_supported: | ||
| litellm.enable_json_schema_validation = True | ||
| clean_response = ( | ||
| completion( | ||
| model=llm_settings.tips_model, | ||
| messages=[{"role": "user", "content": prompt}], | ||
| response_format=TipGenerationResponse, | ||
| custom_llm_provider=llm_settings.custom_llm_provider, | ||
| ) | ||
| .choices[0] | ||
| .message.content | ||
| ) | ||
| else: | ||
| litellm.enable_json_schema_validation = False |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's find and examine the exact file and lines in question
find . -name "tips.py" -type f | head -5Repository: AgentToolkit/kaizen
Length of output: 112
🌐 Web query:
litellm enable_json_schema_validation per-request configuration thread safety
💡 Result:
Short answer: litellm.enable_json_schema_validation is a global/module-level flag (not a per-call parameter) and therefore is not safe to toggle concurrently from multiple threads for per-request behavior. Use per-request validation controls (where available) or avoid mutating the global flag at runtime. [1][2]
Details and evidence:
- The official config docs show enable_json_schema_validation is a boolean in the global settings and “If true, enables json schema validation for all requests.” — i.e., module-level/global. [1]
- There is an open/closed feature request asking to convert litellm.enable_json_schema_validation into a per-call parameter; the issue confirms the current behavior is global and the request was closed as “not planned” (so no built-in per-request flag yet). This implies toggling the global at runtime is the only built-in option today. [2]
- Some providers/features support per-call enforcement (e.g., response_format/enforce_validation for Vertex provider), so prefer using per-request enforcement where the provider SDK supports it instead of changing the global flag. [3]
Recommended practical approaches:
- Don’t flip litellm.enable_json_schema_validation inside concurrent threads. Treat it as a process-wide setting.
- If you need per-request control:
- Use provider-specific per-request options (response_format with enforce_validation / structured-output options) when available. [3]
- Run separate worker processes each with the desired global setting (process isolation).
- If you must toggle the global flag in a single process, serialize changes with a lock so only one thread modifies/uses it at a time (but this serializes those requests and is brittle).
Sources:
- liteLLM config settings — enable_json_schema_validation description. [link] [1]
- GitHub issue requesting per-call parameter for enable_json_schema_validation (feature request, closed/not planned). [link] [2]
- liteLLM Structured Outputs / per-request enforce_validation example (Vertex provider). [link] [3]
Would you like code examples for a locking approach or a per-process worker setup?
🏁 Script executed:
# Examine the tips.py file around lines 113-158 to see full context
head -160 ./kaizen/llm/tips/tips.py | tail -60Repository: AgentToolkit/kaizen
Length of output: 2455
🏁 Script executed:
# Get full context of the function to understand imports and setup
head -160 ./kaizen/llm/tips/tips.py | cat -nRepository: AgentToolkit/kaizen
Length of output: 7588
🏁 Script executed:
# Search for calls to generate_tips to understand usage patterns
rg "generate_tips" --type pyRepository: AgentToolkit/kaizen
Length of output: 2638
🌐 Web query:
litellm response_format Pydantic model automatic validation enable_json_schema_validation
💡 Result:
Short answer: LiteLLM supports using a Pydantic BaseModel as response_format and can perform client‑side JSON Schema validation, but validation is enabled globally via litellm.enable_json_schema_validation (module setting) and there are known edge cases when converting Pydantic -> JSON Schema so some users convert models manually before passing them.
How to use it
- Pass a Pydantic model as response_format to completion(...). Example in the docs shows passing a BaseModel class directly. [1]
- To enable client-side validation, set litellm.enable_json_schema_validation = True (this uses jsonvalidator to validate LLM JSON responses). [1][4]
Known caveats / workarounds
- Validation is currently a global/module setting (not a per-call parameter). There’s an open feature request to make it per-call (closed/not planned). [2]
- Some users have observed buggy or imprecise conversion from Pydantic models to the JSON schema LiteLLM sends to models; recommended workaround is to generate the schema yourself (e.g., model.model_json_schema()) and pass a {"type":"json_schema","json_schema":...} dict instead of relying on the built-in conversion. See related bug reports/discussions. [3]
Sources
- LiteLLM docs: Structured Outputs / JSON mode. [1]
- LiteLLM config settings (enable_json_schema_validation). [4]
- GitHub issues (feature request to make validation per-call; Pydantic -> JSON Schema conversion bugs). [2][3]
Thread-safety concern: Global state mutation.
Setting litellm.enable_json_schema_validation globally on lines 135 and 147 can cause race conditions in concurrent/multi-threaded environments where multiple calls to generate_tips could interfere with each other. LiteLLM's enable_json_schema_validation is a module-level flag that cannot be toggled safely per-request, making this pattern unsafe for concurrent use.
🔧 Suggested approach
Consider one of these alternatives:
- Use a lock to serialize access to this global state
- Set this once at application startup rather than per-call
- Use process isolation (separate workers each with their own global setting)
+import threading
+
+_litellm_lock = threading.Lock()
+
def generate_tips(messages: list[dict]) -> list[Tip]:
# ... setup code ...
if constrained_decoding_supported:
+ with _litellm_lock:
+ litellm.enable_json_schema_validation = True
+ clean_response = (
+ completion(
- litellm.enable_json_schema_validation = True
- clean_response = (
- completion(
model=llm_settings.tips_model,
messages=[{"role": "user", "content": prompt}],
response_format=TipGenerationResponse,
custom_llm_provider=llm_settings.custom_llm_provider,
)
.choices[0]
.message.content
+ )
)
else:
+ with _litellm_lock:
+ litellm.enable_json_schema_validation = False
+ # ... completion call ...
- litellm.enable_json_schema_validation = False
- response = (
- completion(
- ...
- )
- )Alternatively, if this function is only called single-threaded, document that assumption explicitly.
🤖 Prompt for AI Agents
In `@kaizen/llm/tips/tips.py` around lines 134 - 147, The function generate_tips
toggles the module-level flag litellm.enable_json_schema_validation per-call
(controlled by constrained_decoding_supported), which is not thread-safe; either
remove per-call mutation and ensure the flag is set once at process startup (set
litellm.enable_json_schema_validation based on desired global behavior) or
serialize access by guarding the toggling + call to completion(...,
response_format=TipGenerationResponse) with a thread/process-safe lock (e.g., a
module-level threading.Lock) so only one request mutates the flag at a time;
update generate_tips to reference constrained_decoding_supported and
completion/TipGenerationResponse without directly flipping the global flag per
request, or document that the function is single-threaded if you choose not to
change behavior.
2 participants
This PR replaces the initial tips implementation with something slightly more tested.
It also ran
ruffon the repo.Summary by CodeRabbit
Documentation
New Features
Configuration Updates
Improvements
✏️ Tip: You can customize this high-level summary in your review settings.