That's one frustration I have with Tmux. I want mouse scrollback, but don't want the other mouse controls. If I remember correctly I would accidentally go into copy-mode when I click-dragged my mouse by accident.
That's the largest gripe I've had. Eventually, I got used to not relying on GUI scrollback - that actually sped things up for me. In a pinch I just `hotkey, [` to activate scroll, then just use my mouse or page-up & page-down.
Even if you don't use tmux, get used to piping anything you'd want to read through `less`. It'll speed your workflow up.
I do use less sometimes, but sometimes I want to double check a command I typed a shirt while ago, or a command ends up outputting slightly more than I expected.
What I really want us an intelligent terminal, which attaches programming output to the program which produced it, let's me fold up long outputs, easily escape a long running program and store it output in a buffer for later checking, etc. But that seems annoyingly impossible.