Refresh skill docs for current multimodal/CUDA behavior (#282)

Align skill docs with current mobius behavior: encoder inputs are
declared with config.dtype, KV-shared Gemma4 GQA emits empty K/V with
runtime dependency on ORT KV-shared support, and quality checklist now
points to search.past_present_share_buffer in genai_config.json.

---------

Signed-off-by: Justin Chu <justinchu@microsoft.com>
Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Author: 4 people

.agents/skills/debugging-memcpy/SKILL.md

@@ -65,11 +65,12 @@ pkg = build(model_id, execution_provider='cuda', dtype='float16')
 tmpdir = tempfile.mkdtemp(prefix="memcpy_profile_")
 
 for name, model in pkg.items():
-    # IMPORTANT: lower opset to 23 — CUDA EP doesn't register many ops
-    # (including Reshape, Cast) at opset 24.  Without this, you'll see
-    # hundreds of false-positive Memcpy from ops that work fine at opset 23.
-    if model.opset_imports.get("", 0) > 23:
-        model.opset_imports[""] = 23
+    # NOTE: ORT ≤1.24.x didn't register CUDA kernels for some opset 24
+    # standard ops.  If profiling on older ORT, lower opset to 23 to
+    # avoid false-positive Memcpy nodes.  On current ORT this is
+    # unnecessary — the correct fix is updating ORT kernel registration.
+    # if model.opset_imports.get("", 0) > 23:
+    #     model.opset_imports[""] = 23
 
     path = os.path.join(tmpdir, f"{name}.onnx")
     ir.save(model, path, external_data=f"{name}.onnx.data")
@@ -120,14 +121,16 @@ This gives you the exact node names causing Memcpy.  Common categories:
 - **Equal/Cast on token IDs** — usually low-impact (small tensors)
 - **CumSum on INT64** — inherent, no GPU kernel
 
-### Critical: opset 24 false positives
+### Critical: opset 24 false positives (ORT ≤1.24.x)
 
-**Always lower opset to 23 before profiling.**  ORT CUDA EP (≤1.24.x)
-does not register kernels for many standard ops at opset 24, including
-`Reshape`, `Cast`, and others.  A Gemma4 decoder at opset 24 shows
-**280 Memcpy** nodes; at opset 23, it shows **4**.  The
-`ort_lower_opset_for_ep` flag in `_flags.py` handles this at runtime,
-but you must apply it manually when profiling with raw ORT sessions.
+**On ORT ≤1.24.x**, CUDA EP did not register kernels for many standard
+ops at opset 24, including `Reshape`, `Cast`, and others.  A Gemma4
+decoder at opset 24 showed **280 Memcpy** nodes; at opset 23, just **4**.
+The correct fix is to update ORT kernel registration for the missing
+opset versions — not to lower the model's opset.  The
+`ort_lower_opset_for_ep` flag in `_flags.py` is available as a
+workaround (disabled by default, opt-in via
+`MOBIUS_ORT_LOWER_OPSET_FOR_EP=1`).
 
 ## CPU-only op reference (ORT CUDA EP)
 
@@ -223,6 +226,11 @@ and the `GreaterOrEqual` for causality.
 with `is_causal=1`.  Not applicable to `GroupQueryAttention` (GQA handles
 masking internally via `local_window_size`).
 
+> **Note**: For complex attention patterns (e.g. dual head_dim, KV-shared
+> layers, mixed sliding/full attention), prefer **float additive bias** over
+> bool masks.  Float masks are batch-safe and work correctly with MEA
+> (Memory Efficient Attention) on CUDA.
+
 ### Pattern 3: Use GQA's built-in local_window_size
 
 **Problem**: Sliding-window attention requires an explicit mask (CumSum-based)
@@ -303,12 +311,14 @@ at opset 23.  Results after optimization:
 | Decoder | 4 | `input_ids` (input), 2× `Equal` (token masks), `Where` (bool mask) |
 | Embedding | 3 | `input_ids` (input), 2× `Equal` (token masks) |
 
-### Opset 24 trap
+### Opset 24 historical note (ORT ≤1.24.x)
 
-Without opset lowering, the decoder showed **280 Memcpy** nodes because
-CUDA EP doesn't register `Reshape`, `Cast`, and other standard ops at
-opset 24.  The `ort_lower_opset_for_ep` flag (enabled by default) fixes
-this at runtime.  Always lower to opset 23 before profiling.
+On ORT ≤1.24.x, the decoder showed **280 Memcpy** nodes because
+CUDA EP didn't register `Reshape`, `Cast`, and other standard ops at
+opset 24.  This has been fixed in newer ORT versions.  The
+`ort_lower_opset_for_ep` flag (disabled by default, opt-in via
+`MOBIUS_ORT_LOWER_OPSET_FOR_EP=1`) is available as a workaround for
+older ORT builds.
 
 ## Impact assessment
 

.agents/skills/debugging-multimodal/SKILL.md

@@ -254,9 +254,28 @@ ORT bug: microsoft/onnxruntime#28107
 
 ### Opset 24 kernel registration
 
-ORT ≤1.24.x CUDA/TRT EPs don't register kernels for opset 24.
-**Fix:** Use the `ort_lower_opset_for_ep` feature flag (enabled by
-default). See `src/mobius/_flags.py`.
+ORT ≤1.24.x CUDA/TRT EPs didn't register kernels for opset 24.
+This has been fixed in newer ORT versions.  The `ort_lower_opset_for_ep`
+feature flag is available as a workaround (disabled by default, opt-in
+via `MOBIUS_ORT_LOWER_OPSET_FOR_EP=1`).  See `src/mobius/_flags.py`.
+
+### Encoder input dtype alignment
+
+Encoder task inputs should be declared with `dtype=config.dtype` so
+entry tensors match the model compute dtype (float32/float16/bfloat16).
+In the current codebase, multimodal encoder task builders set encoder
+inputs directly to `config.dtype` (there is no `_cast_encoder_input()`
+helper in `src/mobius/tasks/_base.py`).
+
+### GQA for KV-shared layers
+
+Gemma4 KV-shared layers now emit `GroupQueryAttention` with empty K/V
+inputs (`kv_sequence_length=0`) and borrowed source-layer KV wired via
+`past_key`/`past_value`, avoiding extra Transpose/Reshape cache ops.
+
+Runtime support depends on ORT having KV-shared GQA support (tracked in
+microsoft/onnxruntime#28242; still upstreaming as of this writing). On
+ORT builds without that support, this path can fail at runtime.
 
 ### NaN for large head_dim (> 256)
 

.agents/skills/quality-checklist/SKILL.md

@@ -135,10 +135,19 @@ python examples/<model>_text_generation.py --compare-hf --dtype bf16
       overflow: microsoft/onnxruntime#28107)
 - [ ] No NaN from large head_dim > 256 (tracked in
       microsoft/onnxruntime#28195, #28196)
-- [ ] `ort_lower_opset_for_ep` flag handles opset 24→23 lowering for
-      CUDA EP (enabled by default in `src/mobius/_flags.py`)
+- [ ] `ort_lower_opset_for_ep` flag available for opset 24→23 lowering
+      on older ORT builds (disabled by default in `src/mobius/_flags.py`,
+      opt-in via `MOBIUS_ORT_LOWER_OPSET_FOR_EP=1`)
 - [ ] Dead graph inputs removed after EP-aware optimization
       (`RemoveDeadGraphInputsPass` in Stage 4 of `optimize_model()`)
+- [ ] For models with dual head_dim (e.g. Gemma4: 256 for sliding,
+      512 for full attention), verify whether ORT GenAI requires
+      `search.past_present_share_buffer=false` because GenAI allocates
+      uniform KV cache shapes; if so, explicitly override the generated
+      `genai_config.json` before runtime validation rather than assuming
+      the default generated setting is correct
+- [ ] Encoder inputs (vision/audio) are declared with `dtype=config.dtype`
+      at sub-model entry (no stale float32-only cast guidance)
 
 ### 9. ORT GenAI runtime