FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
Paper
•
2506.20920
•
Published
•
75
Dataset
-
-
HuggingFaceFW/finewiki
Viewer • Updated • 61.6M • 13.4k • 266 -
nhagar/fineweb_urls
Viewer • Updated • 24.5B • 328 • 1 -
PleIAs/common_corpus
Viewer • Updated • 470M • 35.1k • 320 -
google-research-datasets/mbpp
Viewer • Updated • 1.4k • 3.54M • 190 -
allenai/c4
Viewer • Updated • 10.4B • 667k • 487 -
HuggingFaceFW/fineweb
Viewer • Updated • 52.5B • 193k • 2.52k