[docs] WeightConverter

[stevhliu]

Adds docs for WeightConverter from #41580

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[LysandreJik]

Sound good! You might want to intro the weight converter with some docs about why it's needed and how dynamic weight loading works

I detailed it in the release notes: https://github.com/huggingface/transformers/releases/tag/v5.0.0rc0

[Cyrilvallez]

This is not entirely true: as soon as we add the tensor to the thread_pool to load it asynchornously, the thread_pool is free to load it as soon as it has an available thread to do it.
So when we enter conversion for a weight, we make sure all needed tensors are materialized by waiting for them if they are not already loaded, but they could very well all be loaded already.
This is why the theoretical memory footprint is model_size + all_largest_params_needed_in_a_single_conversion as I mentioned here.

This is however somewhat of an "unlucky" outcome: it either means the more memory demanding conversion is the last to happen, or that all other params were already loaded by the thread_pool when entering the more demanding conversion.
So in practice, the memory peak can be even lower than that, closer to model_size!

Hope this is clear enough, IMO those considerations are interesting to add in the doc as well, as it makes people understand more how it really works!

EDIT: was talking about the "They're kept as lazy references until needed to defer memory allocation." and "
Tensors are materialized once all Future objects are collected. Operations are batched together and applied to return the transformed tensors." bits here!

[stevhliu]

thanks so much for the clarification!

[stevhliu]

You might want to intro the weight converter with some docs about why it's needed and how dynamic weight loading works

reworked the intro to start with motivation and then brief explanation of how it works!

[Cyrilvallez]

Hey @stevhliu! Just pushed some changes to (hopefully) clarify a bit and add more details, let me know what you think! Feel free to change a bit the language etc if you think something is not so nice (I'm not a native english speaker 🙃), or maybe use tip instead of note etc... Feel free to merge then!

[stevhliu]

Nice, thanks for adding more concrete examples! I think that'll make it easier to understand.

Also big thanks for iterating with me on this, super cool learning about our new loading system ❤️

[github-actions]

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42636&sha=626ca5