I found a easy solution to this one a while ago: middle seat gets both armrests to compensate for it being the middle seat. When I get an aisle or window seat, I relinquish use of one of my armrests to the middle seat.
Everyone knows, Window gets an armrest and a wall. Middle gets two armrests. Aisle gets an armrest and a little bit of extra leg. We’re not fucking animals! We live in a society!