[Oras]

Hard time? What value does adult videos description, views and comments add to small (7,32B) models?

[andy99]

It says it’s common crawl, I interpret it to mean this is a generic web scrape dataset, presumably they filter stuff out they don’t want before pretraining. You’d have to do do some ablation testing to know what value it adds

[ccgreg]

Common Crawl is a particular dataset. commoncrawl.org

[khimaros]

what if that's where they learned how to utilize the double entendre? hard times indeed.