|
|
|
|
|
|
by anythingbot
3313 days ago
|
|
|
> reproducibility is difficult in science generally but can be insanely difficult for machine learning it is a computer algorithm, so by definition it is trivial to reproduce results, you just run the program again. > I ran into this recently by accident when writing a simple RL example. With two weight matrices to learn, the first weight matrix was given correct gradients, the last weight matrix was only supplied with partial information. Surprise, still works, and I only discovered the bug _after_ submitting to OpenAI's Gym with quite reasonable results. so you want to say that you coded a bug, but you don't have a method of testing whether you have a bug. So you didn't code a bug. And if you didn't code a bug, you can't reproduce a bug. So yes, reproducing a bug is difficult when you have no means of determining whether or not you have a bug... ...maybe you should look into choosing a means of determining whether or not you have a bug. |
|
|
Not really. Deep learning is still quite a lot of dark voodoo where random initialization and data shuffling can matter significantly. People also adapt hyperparameters manually during training, stop early with no clear metric, and don't share their code for preprocessing the data or even the exact architecture of the network.
It's certainly better than in other fields, but it's not trivial.