How to get the data for pretraining stage?

[chrisgao99]

Hello,

thank you for your great work on "Driving with LLMs".

I'm trying to reproduce the pre-training part but I'm confused with the data you used.

My understanding is that for each vector, there should be some QA pairs regarding it and in total there should 100k QA pairs.

My question is:

How do you get the 100k QA pairs related to the vector?

What is the difference between the captioning data from LanGen and the 100k QA pairs?

For each vector, how many QA did you generate?

Thank you for your time and for any clarification you can provide.

[MLgeek7325]

您好，您的邮件我已收到！

[lyqyingmu]

Do you have successfully reproduced it?

[MLgeek7325]

您好，您的邮件我已收到！