[brudgers]

Have any major platforms failed due lax moderation policies?

Yes, that's why you haven't heard of them.

They fail before they get big enough to be popular.

File sharing platforms - defined broadly enough to include Youtube - will attract adversaries with asymmetric incentives.

It will be easy for people trying to upload illegal content to upload illegal content to the degree you make it easy for people to upload content. [1]

When your file sharing server starts serving child-pornography - and it will - law enforcement will contact you and your best option will be shutting down.

You will never get to the point where radicalization videos are the problem.

[1]: What is illegal depends on which legal jurisdiction. What can legally be said about Atatürk is different in Turkey than Mexico. Libel in the UK is more broad than the US. Pictures of naked humans engaged in activity vary in legality around the world.