Skip to content

LLM seems to ignore the grammar I'm setting for it #404

@HossamBasion

Description

@HossamBasion

I'm trying to force my LLM to only output 5 to 15 words Arabic output using the attached file (It's a gbnf file but I uploaded it as a .txt file so I can attach it here)

WordCount.txt

And if it helps, this is my system prompt: "I will give you a topic. Respond with one short sentence in Egyptian Arabic language. The sentence must be between five words and fifteen words ONLY.
Just produce a simple, everyday Egyptian sentence that mentions the topic naturally.
Return only the sentence and nothing else.
Under no circumstance should you ever respond in more than fifteen words . And your response must always be more than five words. This is the most important thing. And always make sure to respond in Egyptian Arabic Language
Remember a word is defined as letters surrounded by whitespace"

It does generate Arabic output correctly. But no matter what I do, it keeps outputting all kinds of word counts including a lot of two word responses and sometimes even one word responses.

This is an example of something it generated: السجاد جميل (translates to "the carpets are beautiful" but it's only two words)

I don't even use the SetGrammar function. I just use the loadGrammar button to load the grammar document I attached here.

Steps to reproduce

  1. Load any LLM gguf and keep all default settings the same.
  2. Use the System prompt and grammar file I posted.
  3. Create a for loop and run the llm for let's say 20 times: response = await Allam.Chat(topic, null, null, false);
  4. Observe how many sentences ignore the grammar file word count requirement.

LLMUnity version

6000.0.50

Operating System

Windows

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions