[kvathupo]

As someone with a little experience in ML research, I feel like outside of the top 5-10 conferences, there rarely is a code repo [1]. I think an even bigger issue is that even when a repo is provided, it's often poorly documented, contains dead links, or _hard to reproduce on other machines_ without code-wrangling.

Tangentially related, even popular repos can have implementations of models that differ from the paper cited. So, who knows how far errors can propagate ¯\_(ツ)_/¯

[1] - https://www.guide2research.com/topconf/

[buildbot]

Somethings that’s personally bothered me for a long time is that as far as I know, there’s now current way to check the test accuracy on imagenet, as the test server has been gone for awhile. Yet somehow people still report a test accuracy…