Not to take wind out from the sails of OP, but there are [edit: not CSS only, a couple bytes of inline JS] alternatives that remove some of that issue, and usage ambiguity by using an explicit slider (at the cost of some other features like zoom).
The ask is that swipe up/down on elements that only want left/right bubble those up/down events to the page so it scrolls. I believe it's not specific to any one device.
Though it is more pronounced with a screen where the picture fills most of the screen. You currently have to find a border or space between the pictures to swipe on to scroll up or down.
https://picturepan2.github.io/spectre/experimentals/viewer-3...