Conversation
…es with custom generation options
There was a problem hiding this comment.
Pull request overview
This PR refactors LlamaLanguageModel to move implementation-specific properties (contextSize, batchSize, threads, seed, temperature, topK, topP, repeatPenalty, repeatLastN) from model-level initialization parameters to per-request custom generation options. This follows up on #50 to improve the API design by allowing these parameters to vary per request rather than being fixed at model initialization.
Key changes:
- Properties previously set during initialization now default to standard values and can be overridden per request via
GenerationOptionscustom options - Original initializer parameters are deprecated but remain functional (ignoring passed values)
- New internal
ResolvedGenerationOptionsstruct consolidates option resolution logic - Refactored sampling logic into a dedicated
applySamplingmethod
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| Sources/AnyLanguageModel/Models/LlamaLanguageModel.swift | Adds custom option properties to CustomGenerationOptions, deprecates model-level properties, introduces ResolvedGenerationOptions for option resolution, refactors initializers and generation methods to use resolved options, extracts sampling logic into applySampling method |
| Tests/AnyLanguageModelTests/LlamaLanguageModelTests.swift | Updates tests to reflect new initialization API, adds tests for custom options round-trip and deprecated initializer behavior, expands test for custom generation options with new parameters |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Follow up to #50