Hacker News new | ask | show | jobs
by bingo-bongo 171 days ago
AI companies scrape to:

- have data to train on

- update the data more or less continuously

- answer queries from users on the fly

With a lot of AI companies, that generates a lot of scraping. Also, some of them behave terribly when scraping or is just bad at it.

1 comments

Why don’t they scrape once though?
1) It may be out of date 2) Storing it costs money
Bandwidth costs more money.