refactor: Deduplicate language-specific model files#492
Conversation
Consolidates ~700 lines of duplicated boilerplate across three language-specific model files into a generic implementation. Changes: - Add ParakeetLanguageModels<Config> generic struct (337 lines) - Refactor CtcJaModels.swift: 229 → 22 lines (config + typealias) - Refactor CtcZhCnModels.swift: 265 → 22 lines (config + typealias) - Refactor TdtJaModels.swift: 237 → 22 lines (config + typealias) - Make Repo enum Sendable for concurrency safety - Add joint model validation in TdtJaManager Pattern: Protocol-based configuration with generic implementation. The ParakeetLanguageModelConfig protocol defines language-specific settings (blankId, repository, model files, int8 support). Type aliases maintain backward compatibility. Reduces codebase by 328 lines (~45% reduction) while maintaining identical functionality. All CI tests pass. Resolves #457
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
VAD Benchmark ResultsPerformance Comparison
Dataset Details
✅: Average F1-Score above 70% |
✅ Japanese ASR Benchmark Results (CTC)Status: Passed
✅ Benchmark completed successfully. The TDT Japanese hybrid model (CTC preprocessor/encoder + TDT decoder/joint) is working correctly. View benchmark log |
Offline VBx Pipeline ResultsSpeaker Diarization Performance (VBx Batch Mode)Optimal clustering with Hungarian algorithm for maximum accuracy
Offline VBx Pipeline Timing BreakdownTime spent in each stage of batch diarization
Speaker Diarization Research ComparisonOffline VBx achieves competitive accuracy with batch processing
Pipeline Details:
🎯 Offline VBx Test • AMI Corpus ES2004a • 1049.0s meeting audio • 225.8s processing • Test runtime: 3m 50s • 04/07/2026, 05:01 AM EST |
PocketTTS Smoke Test ✅
Runtime: 0m29s Note: PocketTTS uses CoreML MLState (macOS 15) KV cache + Mimi streaming state. CI VM lacks physical GPU — audio quality may differ from Apple Silicon. |
Parakeet EOU Benchmark Results ✅Status: Benchmark passed Performance Metrics
Streaming Metrics
Test runtime: 1m39s • 04/07/2026, 04:59 AM EST RTFx = Real-Time Factor (higher is better) • Processing includes: Model inference, audio preprocessing, state management, and file I/O |
Kokoro TTS Smoke Test ✅
Runtime: 0m43s Note: Kokoro TTS uses CoreML flow matching + Vocos vocoder. CI VM lacks physical ANE — performance may differ from Apple Silicon. |
Speaker Diarization Benchmark ResultsSpeaker Diarization PerformanceEvaluating "who spoke when" detection accuracy
Diarization Pipeline Timing BreakdownTime spent in each stage of speaker diarization
Speaker Diarization Research ComparisonResearch baselines typically achieve 18-30% DER on standard datasets
Note: RTFx shown above is from GitHub Actions runner. On Apple Silicon with ANE:
🎯 Speaker Diarization Test • AMI Corpus ES2004a • 1049.0s meeting audio • 40.2s diarization time • Test runtime: 1m 52s • 04/07/2026, 05:03 AM EST |
Qwen3-ASR int8 Smoke Test ✅
Performance Metrics
Runtime: 3m36s Note: CI VM lacks physical GPU — CoreML MLState (macOS 15) KV cache produces degraded results on virtualized runners. On Apple Silicon: ~1.3% WER / 2.5x RTFx. |
Sortformer High-Latency Benchmark ResultsES2004a Performance (30.4s latency config)
Sortformer High-Latency • ES2004a • Runtime: 1m 57s • 2026-04-07T09:06:04.520Z |
ASR Benchmark Results ✅Status: All benchmarks passed Parakeet v3 (multilingual)
Parakeet v2 (English-optimized)
Streaming (v3)
Streaming (v2)
Streaming tests use 5 files with 0.5s chunks to simulate real-time audio streaming 25 files per dataset • Test runtime: 5m58s • 04/07/2026, 05:12 AM EST RTFx = Real-Time Factor (higher is better) • Calculated as: Total audio duration ÷ Total processing time Expected RTFx Performance on Physical M1 Hardware:• M1 Mac: ~28x (clean), ~25x (other) Testing methodology follows HuggingFace Open ASR Leaderboard |
Replace force unwrap with guard let statement and proper error handling. This follows project guidelines which prohibit force unwrapping. Changes: - Replace models.joint! with guard let jointModel = models.joint - Throw ASRError.processingFailed if joint model is missing - Remove precondition from init (guard let provides better error handling) Addresses review feedback on PR #492.
Fixed Force Unwrap Violation ✅Addressed the review feedback by removing the force unwrap at . Changes
BeforejointModel: models.joint!, // Safe to force unwrap - validated in initAfter// Validate joint model is present (required for TDT)
guard let jointModel = models.joint else {
throw ASRError.processingFailed("TDT models require a joint model")
}
// ...
jointModel: jointModel,Why This is Better
Tests
The refactoring now follows all project code standards. |
✅ Japanese ASR Benchmark Results (CTC)Status: Passed
✅ Benchmark completed successfully. The TDT Japanese hybrid model (CTC preprocessor/encoder + TDT decoder/joint) is working correctly. View benchmark log |
Replace force unwrap with guard let statement and proper error handling. This follows project guidelines which prohibit force unwrapping. Changes: - Replace models.joint! with guard let jointModel = models.joint - Throw ASRError.processingFailed if joint model is missing - Remove precondition from init (guard let provides better error handling) Addresses review feedback on PR #492.
1 participant
Summary
Consolidates ~700 lines of duplicated boilerplate across three language-specific model files into a generic implementation. This addresses the architectural debt noted in #457.
Changes
New Files
ParakeetLanguageModels.swift- Generic implementation (337 lines)Refactored Files
CtcJaModels.swift: 229 → 22 lines (config + typealias)CtcZhCnModels.swift: 265 → 22 lines (config + typealias)TdtJaModels.swift: 237 → 22 lines (config + typealias)Supporting Changes
RepoenumSendablefor Swift 6 concurrency safetyTdtJaManager(TDT requires joint model)Architecture
Uses a protocol-based configuration pattern:
Three lightweight configs capture the differences:
CtcJaConfig- Japanese CTC (blankId: 3072, 3 models)CtcZhCnConfig- Chinese CTC (blankId: 7000, 3 models + optional int8 encoder)TdtJaConfig- Japanese TDT (blankId: 3072, 4 models with joint)Type aliases maintain backward compatibility:
Impact
Test Plan
Resolves #457