Skip to content

AI: LLM-assisted material preset prompts #406

Description

@fernandotonon

Parent epic: #397

Goal

Use the existing LLMManager (llama.cpp) to translate natural-language descriptions into PBR preset parameter values that feed the slice-E preset templates.

Example: user types "rusty bronze, weathered, slightly metallic" → LLM returns a JSON object with metallic, roughness, albedo, normalIntensity, etc., which the Material Editor applies live.

Why

The slice-E PBR work added solid preset templates and parameter sliders, but new users don't know which numbers correspond to which look. A locally-run LLM is perfect for this — small models (3B–8B) handle structured-output prompts well and we already ship the runtime.

Scope

  • Reuse LLMManager — no new dependency.
  • Add a "Describe material" prompt input in the Material Editor's preset area.
  • Use a constrained-output prompt template (JSON schema in the system prompt) to coerce the LLM into producing a valid preset patch.
  • Apply the patch through the existing PBR preset application path so it's consistent with qtmesh material --preset behavior.
  • CLI: qtmesh material model.fbx --describe "rusty bronze armor" -o out.fbx.
  • MCP: describe_material tool.

Acceptance Criteria

  • Natural-language prompts produce valid preset patches that don't break the existing preset code path.
  • Works with at least the default bundled LLM.
  • Falls back with a clear error if no LLM model is loaded.
  • CLI and MCP parity.
  • Sentry breadcrumb ai.assist.describe_material recorded.

Effort

~3 days.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-assistLocal-AI-assisted 3D workflows (epic prefix: AI:)enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions