Please add gemma2 text encoder arch

[btaskel]

Lumina2 requires gemma2 2b(5.2GB) as a text encoder, but it still takes up a lot of memory on some mobile devices. We hope to increase support for the gemma2 architecture🙏.

[spawner1145]

Hi, btaskel. I have tested the GGUF format for the Gemma2-2B model using the code repository at https://github.com/spawner1145/ComfyUI-GGUF/tree/gemma2test and identified a critical issue: significant quality loss in generated outputs when using the GGUF-quantized version of the model.

pre-quantized models in https://huggingface.co/neta-art/gemma2-2b-gguf

1. Root Cause of Compatibility Issue

ComfyUI requires a specific metadata key named "spiece_model" to enable model integration with downstream tasks. However, this key is not included in the official implementation of llama.cpp. As a result, quantized GGUF models generated via standard llama.cpp workflows fail to be recognized and passed to subsequent processing steps in ComfyUI.

2. Solution Attempt: Custom Conversion Script

To resolve the metadata compatibility issue, I developed a custom conversion script that processes the official Safetensors files provided by ComfyUI. This script ensures the required spiece_model metadata is retained in the final GGUF file.

How to Use the Conversion Script

Run the following command in the repository to convert a Gemma2-2B Safetensors file to GGUF format. The script supports several parameters to control the conversion process:

python tools/convert_gemma2.py --src gemma_2_2b_fp16.safetensors --dst gemma_2_2b_fp16.gguf --quantize q4_k

Parameter Explanations:

1.--src (required)- Specifies the path to the source Safetensors file containing the Gemma2-2B model

• Example: --src gemma_2_2b_fp16.safetensors
• This parameter is mandatory and must point to a valid existing file

2.--dst (optional)- Defines the output path and filename for the converted GGUF file

• If not specified, the output filename will be generated automatically based on the source filename and quantization type (e.g., gemma_2_2b_fp16-Q4_K.gguf)
• Example: --dst custom_output_name.gguf

3.--quantize / --quant / -q (optional)- Specifies the quantization type to apply during conversion

• Available options:
  • Full precision: f32, f16, bf16
  • 8-bit quantization: q8_0
  • 4-bit quantization: q4_0, q4_1, q4_k
  • 5-bit quantization: q5_0, q5_1, q5_k
  • 2/3/6-bit quantization: q2_k, q3_k, q6_k
• If not specified, the script will automatically determine the quantization based on the source model's dominant dtype
• Example: -q q4_k or --quantize f16

Special Quantization Rules:

The conversion script applies intelligent quantization with these rules:

• Embedding layer (token_embd.weight) is always kept at F16 precision to preserve quality
• Normalization layers (including attention norms and output norms) are kept at F32 precision for numerical stability
• All other layers (attention and MLP components) use the specified quantization type

Example Commands:

# Basic conversion with automatic quantization
python tools/convert_gemma2.py --src gemma_2_2b_fp16.safetensors

# Convert to 4-bit quantization (K type) with custom output path
python tools/convert_gemma2.py --src gemma_2_2b.safetensors --dst gemma2_2b_quantized.gguf -q q4_k

# Convert to full 16-bit floating point precision
python tools/convert_gemma2.py --src gemma_2_2b.safetensors --quantize f16

After conversion, the output GGUF file will include the required "spiece_model" metadata for compatibility with ComfyUI, while applying the specified quantization to balance file size and model performance.

After conversion:

1. Use the GGUF CLIP Loader (included in this repository) to load the generated gemma_2_2b_fp16.gguf file.
2. Integrate the loader into your ComfyUI workflow as a replacement for the standard model loader.

3. Quality Loss Observation

Despite resolving the compatibility issue, the GGUF-quantized Gemma2-2B model exhibits severe quality degradation compared to the original Safetensors model. Below are side-by-side comparisons of generated outputs under identical parameters.

Test Configuration

• Sampler: Euler A
• Scheduler: smooth scheduler
• Steps: 24
• CFG Scale: 4.0
• TeaCache Settings: TeaCache L1 Threshold = 10.0

3.1 Basic Prompt Test

Prompt:
You are an assistant designed to generate anime images based on textual prompts. <Prompt Start>,,,1girl,red hair,blue eyes,,

Model Type	Generated Image
Original (Safetensors)
GGUF Quantized

Observation: The GGUF model produces outputs with blurred details, inconsistent color rendering, and loss of fine features (e.g., hair texture, eye clarity) compared to the original.

3.2 Character-Specific Prompt Test

The GGUF model also suffers from knowledge loss, failing to recognize some character-specific attributes.

Prompt:
You are an assistant designed to generate anime images based on textual prompts. <Prompt Start>,,,1girl,emilia (re:zero),,

Model Type	Generated Image
Original (Safetensors)
GGUF Quantized

Observation: The original model correctly generates Emilia (from Re:Zero) with her iconic silver hair and purple eyes. The GGUF model, however, fails to recognize the character—producing a generic female figure with few resemblance to Emilia.

But you can still find some features of some charas,like:

Prompt:
You are an assistant designed to generate anime images based on textual prompts. <Prompt Start>,,,aris (blue archive),,

Model Type	Generated Image
Original (Safetensors)
GGUF Quantized

4. Recommendation

Based on the extensive testing, it is strongly recommended not to use the GGUF-quantized version of the Gemma2-2B model for ComfyUI workflows. The quantized model suffers from irreversible quality loss and knowledge degradation that cannot be mitigated by compatibility fixes.

For optimal performance and output quality, continue using the original Safetensors version of Gemma2-2B in ComfyUI.