[kvh]

You could just map 5 to something farther away, like 6. In fact, this is how most ordinal inference techniques work anyways: by taking an interval method and learning cutoffs for your ordered categories. Learning more parameters comes with a big cost though, which is why in practice the cutoffs are often fixed from the get-go. Obviously, ordinal methods have been tried in the literature. There is a reason they are not used in practice though, and that's because the trade-off (being harder to learn vs modelling the data more accurately) is not favorable.