I've been tempted to stop using it, but instead just shut off notifications. I really like being able to pull it up quickly to see when code has been pushed up, pull requests created / merged, or whatever automated CI action is going on for a particular project.
The chatter can burn a lot of time though. You're absolutely right there.
We solved this by creating a separate channel for gifs and news. Those interested can participate, those not interested can just mute the channel and ignore it.
Working as a publisher and having channels to talk to each of our clients is amazing and increases productivity and communication way beyond just email and phone. But that's just my particular use-case :)