Create a set of trainer scripts, in directory similar to drivers/ or just one generlized trainer script (whichever is a better practice), that can load, finetune, and save a huggingface model based on a provided dataset. This dataset should follow the same annotations JSON format used in the pipeline for consistency. Models should be saved and loaded to a specified directory of the user's choice.
Create a set of trainer scripts, in directory similar to
drivers/or just one generlized trainer script (whichever is a better practice), that can load, finetune, and save a huggingface model based on a provided dataset. This dataset should follow the same annotations JSON format used in the pipeline for consistency. Models should be saved and loaded to a specified directory of the user's choice.