2 researchers from the Brown university were able to reproduce the full, and never released GPT-2 model a couple of days ago. They wrote an amazing blog post and released the weights!! Here's the blog with all the details: https://medium.com/@vanya_cohen/opengpt-2-we-replicated-gpt-...
For those of you who don't know what GPT-2 is, here's the simplest & fastest way with a bit of humor: https://blog.floydhub.com/gpt2/
They spent $500k doing it (in research credits for compute). Not sure if this is the best way to spend so much money, considering they made no novel contribution to NLP field, and OpenAI would have most likely released it soon anyway.