Also try reducing mousewheel.min_line_scroll_amount if changing the other setting doesn't help. Try a value of 16.
The default is 100, which seems to work fine most of the time, but for the OP's issue, reducing it by a factor of ten might be a good starting point.