Skip to content

Failed to get model definition wit /compact + Docker Model Runner #2800

@k33g

Description

@k33g

When running docker-agent against a model served by Docker Model Runner (provider dmr), the manual /compact command always fails with the error: Failed to get model definition

Additionally, automatic compaction never runs.

Environment

  • Provider: dmr (Docker Model Runner, local inference via llama-server)
  • Example model: huggingface.co/unsloth/qwen3.5-4b-gguf:Q4_K_M
  • provider_opts.context_size: 32768 is set in the agent YAML
  • base_url: http://localhost:12434/engines/v1

Steps to reproduce

  1. Configure an agent that uses a local DMR model (see agent.yaml snippet below).
  2. Start a conversation and let it grow.
  3. Run /compact at any point.
  models:
    qwen-local:
      provider: dmr
      model: huggingface.co/unsloth/qwen3.5-4b-gguf:Q4_K_M
      base_url: http://localhost:12434/engines/v1
      max_tokens: 24576
      provider_opts:
        context_size: 32768

Expected behavior

  • /compact should produce a summary of the conversation and free up context.
  • When the conversation approaches the configured context_size (32768), proactive compaction should trigger automatically.

Actual behavior

  • /compact emits Failed to get model definition and exits without compacting anything.
  • The conversation continues to grow until the context window overflows; automatic compaction never fires.

Version of docker-agent: 1.50.0

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions