Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions examples/speculative_decoding/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ For small base models that fit in GPU memory, we can collocate them with draft m
```bash
./launch_train.sh \
--config ../../modelopt_recipes/general/speculative_decoding/eagle3.yaml \
model.model_name_or_path=meta-llama/Llama-3.2-1B \
model.model_name_or_path=meta-llama/Llama-3.2-1B-Instruct \
data.data_path=input_conversations/train.jsonl \
training.output_dir=ckpts/llama-3.2-1b-online
```
Expand Down Expand Up @@ -123,7 +123,7 @@ Once we finish dumping hidden states, launch offline training pointing to the hi
```bash
./launch_train.sh \
--config ../../modelopt_recipes/general/speculative_decoding/eagle3.yaml \
model.model_name_or_path=meta-llama/Llama-3.2-1B \
model.model_name_or_path=meta-llama/Llama-3.2-1B-Instruct \
data.offline_data_path=$HIDDEN_STATES_DIR \
training.output_dir=ckpts/llama-3.2-1b-offline
```
Expand Down
7 changes: 6 additions & 1 deletion modelopt/torch/utils/plugins/transformers_dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,12 @@ def __init__(

self._post_process_tokenizer()
if self.tokenizer.chat_template is None:
raise ValueError("No valid chat template!")
raise ValueError(
"No valid chat template! The tokenizer for this model has no chat_template, "
"which is common for base / pretrained checkpoints. Use an instruction-tuned "
"model (e.g. a '*-Instruct' variant), or pass a custom chat_template via the "
"training config or LanguageDataCollator(chat_template=...)."
)

if self.answer_only_loss:
self._verify_generation_tags()
Expand Down
Loading