They didn't cause problems in the way you're implying. We took the time to polish hackathon projects before releasing them. We didn't just shove them out the door as soon as they were done.
That said, one of them was an unmitigated disaster. Leadership loved it, users hated it, you know the drill. The company eventually gave in and turned the feature off three years later.
* It had to directly relate to the product.
* You could not use the time to work on some existing project.
* You had to be able to finish it by the end of the hackathon.
Hackathon projects were sometimes adopted by a permanent team, if there was a good fit.