I am definitely willing to concede I may be in a bad situation but are there really teams in Agile frameworks with fibonacci pointing where you can pick up a 3-5 point ticket, finish it before the end of a sprint and not work on anything else?
I don't think you're supposed to do nothing at that point. Leisurely improve your tools, read to improve knowledge/performance, etc. This compounds your efficiency over time, without the stress of more deadlines.
I've seen plenty where a point-a-day pace or somewhat under (maybe 0.8pt/day) would be a totally normal pace, and is absolutely achievable with under half a day of actual work per day, including meetings, if you know what you're doing.
But then again, points are incommensurable between teams.