Reminder
System Info
transformers version: 4.41.1
Platform: Linux-5.15.0-107-generic-x86_64-with-glibc2.35
Python version: 3.10.12
Huggingface_hub version: 0.23.2
Safetensors version: 0.4.3
Accelerate version: 0.29.3
Accelerate config: - compute_environment: LOCAL_MACHINE
- distributed_type: DEEPSPEED
- mixed_precision: bf16
- use_cpu: False
- debug: True
- num_processes: 3
- machine_rank: 0
- num_machines: 1
- rdzv_backend: static
- same_network: True
- main_training_function: main
- enable_cpu_affinity: False
- deepspeed_config: {'gradient_accumulation_steps': 2, 'zero3_init_flag': False, 'zero_stage': 0}
- downcast_bf16: no
- tpu_use_cluster: False
- tpu_use_sudo: False
- tpu_env: []
PyTorch version (GPU?): 2.2.1+cu121 (True)
Tensorflow version (GPU?): not installed (NA)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?: True
Using distributed or parallel set-up in script?: False
Reproduction
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \ --stage dpo \ --pref_loss simpo \ --simpo_gamma 1.0 \ --do_train True \ --model_name_or_path /home/ubuntu/date/llama_ckpts/llama_lx3_ckpts/BAdam_llama3_random_lr1e-6/checkpoint-9600 \ --preprocessing_num_workers 16 \ --finetuning_type full \ --template default \ --flash_attn auto \ --dataset_dir data \ --dataset ultrafeedback_binarized \ --split train \ --cutoff_len 2048 \ --learning_rate 2e-7 \ --num_train_epochs 1.0 \ --max_samples 10000000 \ --per_device_train_batch_size 1 \ --per_device_eval_batch_size 1 \ --gradient_accumulation_steps 8 \ --lr_scheduler_type cosine \ --max_grad_norm 1.0 \ --logging_steps 1 \ --save_steps 100 \ --warmup_ratio 0.1 \ --optim adamw_torch \ --packing False \ --report_to none \ --use_badam True \ --output_dir saves/LLaMA3-8B/full/train_2024-06-05 \ --pure_bf16 True \ --plot_loss True \ --use_badam True \ --badam_mode layer \ --badam_switch_mode random \ --badam_switch_interval 100 \ --val_size 0.05 \ --evaluation_strategy steps \ --eval_steps 20
Same to "#4085"
Expected behavior
ultrafeedback_binarized 是研究中常用的对齐数据集，能否提供支持？
Others
No response
Reminder
System Info
transformers version: 4.41.1
Platform: Linux-5.15.0-107-generic-x86_64-with-glibc2.35
Python version: 3.10.12
Huggingface_hub version: 0.23.2
Safetensors version: 0.4.3
Accelerate version: 0.29.3
Accelerate config: - compute_environment: LOCAL_MACHINE
PyTorch version (GPU?): 2.2.1+cu121 (True)
Tensorflow version (GPU?): not installed (NA)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?: True
Using distributed or parallel set-up in script?: False
Reproduction
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \ --stage dpo \ --pref_loss simpo \ --simpo_gamma 1.0 \ --do_train True \ --model_name_or_path /home/ubuntu/date/llama_ckpts/llama_lx3_ckpts/BAdam_llama3_random_lr1e-6/checkpoint-9600 \ --preprocessing_num_workers 16 \ --finetuning_type full \ --template default \ --flash_attn auto \ --dataset_dir data \ --dataset ultrafeedback_binarized \ --split train \ --cutoff_len 2048 \ --learning_rate 2e-7 \ --num_train_epochs 1.0 \ --max_samples 10000000 \ --per_device_train_batch_size 1 \ --per_device_eval_batch_size 1 \ --gradient_accumulation_steps 8 \ --lr_scheduler_type cosine \ --max_grad_norm 1.0 \ --logging_steps 1 \ --save_steps 100 \ --warmup_ratio 0.1 \ --optim adamw_torch \ --packing False \ --report_to none \ --use_badam True \ --output_dir saves/LLaMA3-8B/full/train_2024-06-05 \ --pure_bf16 True \ --plot_loss True \ --use_badam True \ --badam_mode layer \ --badam_switch_mode random \ --badam_switch_interval 100 \ --val_size 0.05 \ --evaluation_strategy steps \ --eval_steps 20Same to "#4085"
Expected behavior
ultrafeedback_binarized 是研究中常用的对齐数据集，能否提供支持？
Others
No response