Hacker News new | ask | show | jobs
by zump 3406 days ago
Numerai is generating an unprecedented amount of hype articles.
2 comments

i'm guessing they're submarine'ing, maybe hired a PR firm
I wonder if they are actually making any money...
Exactly, I played Numerai many months, now it is full of cheating robots.
How does that make sense? In an AI competition you expect robots to compete to the best of their ability.
The bots are used to scrape data from the leaderboard and make adjustments to improve their score by overfitting the public leaderboard data.

Essentially this:

http://blog.mrtz.org/2015/03/09/competition.html

You might say: but that won't change who actually wins on the private board. However when you use techniques like that you can get identify data in the test set and use it for training. It does help a some, but not a lot. The payout isn't very large for the over fitters (maybe 10-20% of payout).

Another negative is regular competitors also have no idea how they are actually doing compared to others. The leaderboard has impossible (for market data) logloss values of 0.2 - 0.4, sometimes even 0.00. They keep trying to fix it, but they haven't in about 2 months.

In this case, bots are automated cheating.

I'm curious, how do the robots cheat?
There are people submitting a ton of models that are all very similar, basically overfitting the public board.
Even my cheating robot is bored of Numerai.