Refactor data type handling and fix convolution bias initialization

[tadani3]

This pull request primarily focuses on improving numerical stability and dtype consistency in the codebase, particularly for operations involving mixed-precision (such as float16). The changes ensure that constants used in computations match the data types of the operands, which helps prevent potential issues during inference or training. Additionally, there is a small fix to avoid setting function overloads that could interfere with serialization.

Improvements for dtype consistency and numerical stability:

• Updated all constant values in convolution and normalization operations to use op.CastLike, ensuring they match the dtype of associated tensors (e.g., float16). This affects the bias in convolution, scaling in attention, and epsilon in normalization.

Model and function configuration updates:

• Removed the explicit overload attribute from the ir.Function definition in causal_conv1d_with_state to prevent issues with node-function matching during serialization.

[github-actions]

Performance Comparison

Model	Metric	Baseline	Current	Delta
falcon	model_size_bytes	364 KB	364 KB	+0.0%	⚪
falcon	num_nodes	66	66	+0.0%	⚪
gemma2	model_size_bytes	428 KB	428 KB	+0.0%	⚪
gemma2	num_nodes	109	107	-1.8%	⚪
gpt2	model_size_bytes	388 KB	388 KB	+0.0%	⚪
gpt2	num_nodes	60	60	+0.0%	⚪
llama	model_size_bytes	425 KB	425 KB	+0.0%	⚪
llama	num_nodes	68	68	+0.0%	⚪
llama (static-cache-text-generation)	model_size_bytes	425 KB	425 KB	+0.0%	⚪
llama (static-cache-text-generation)	num_nodes	58	58	+0.0%	⚪
phi3	model_size_bytes	421 KB	421 KB	+0.0%	⚪
phi3	num_nodes	68	68	+0.0%	⚪
phi3 (static-cache-text-generation)	model_size_bytes	421 KB	421 KB	+0.0%	⚪
phi3 (static-cache-text-generation)	num_nodes	58	58	+0.0%	⚪
qwen2	model_size_bytes	425 KB	425 KB	+0.0%	⚪
qwen2	num_nodes	68	68	+0.0%	⚪
qwen2 (static-cache-text-generation)	model_size_bytes	425 KB	425 KB	+0.0%	⚪
qwen2 (static-cache-text-generation)	num_nodes	58	58	+0.0%	⚪

No performance regressions.

[github-actions]

🏗️ Architecture Diff

Model	Sub-model	Changes	Status

No architecture changes detected. ✅

---

Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed)

[Copilot]

Pull request overview

This PR improves dtype consistency for mixed-precision execution (notably float16) by ensuring scalar constants are cast to the same dtype as their operand tensors, and fixes an ONNX ir.Function configuration detail that can break function-node matching during serialization.

Changes:

• Removed the overload field from the CausalConv1DWithState ir.Function definition to avoid serialization mismatches with call sites.
• Cast scalar constants via op.CastLike in Gated DeltaNet (conv bias, attention scale, L2-norm epsilon) to match operand dtypes.
• Updated the Qwen3.5 text-generation example to build the text-only model in f16 by default.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
src/mobius/functions/causal_conv.py	Removes function overload to ensure ONNX serializer can match nodes to the function definition.
src/mobius/components/_gated_deltanet.py	Uses CastLike on scalar constants to keep computations dtype-consistent under mixed precision.
examples/qwen35_text_generation.py	Switches default dtype for the text-only example build from f32 to f16.

---

You can also share your feedback on Copilot code review. Take the survey.

[justinchuby]

Thanks!

[justinchuby]

For some reason I still get onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from /tmp/tmpej7mlcbd/model.onnx failed:Type Error: Type parameter (T) of Optype (Mul) bound to different types (tensor(float) and tensor(float16) in node (model/layers.0/linear_attn/Mul_node_94)