But is the psychovisual output any good or do they only target, say, speed? Because back in the days, both Videolan projects x264 and x265 (especially x264) had much better psychovisual quality than commercial encoders.
Back in those days, x264 felt to me like a software written by aliens from the future.
> Back in those days, x264 felt to me like a software written by aliens from the future.
Indeed it may be :) H.264 might not be the latest and best video coding standard anymore, but in my opinion x264 is, and will always be by far the best encoder ever written for a video format.
Which brings me to...
> Because back in the days, both Videolan projects x264 and x265 (especially x264) had much better psychovisual quality than commercial encoders.
Unfortunately x265 isn't a Videolan project (its developed by MulticoreWare Inc.) and it's a very mediocre encoder which doesn't hold a candle to x264 and IMO kind of a shame considering its legacy.
Also after 2018 it's practically became maintainenance-only and was surpassed by proprietary encoders in MSU encoder tests in the following years. That's a big loss considering x264 was still seeing significant efficiency and performance improvements as late as 2013 (when the H.264 format was 10 years old), so when compared to x264 I assume a good 4-5 years of potential improvements have been left at the table for x265.
I think the big issue is x264 was very obviously a labor of love from some very talented developers. I just haven't seen that sort of love dumped into other encoders.
Newer codecs have been relying on the format to provide more obvious tools for compression (and mostly giving benefits for HD+ resolutions).
I wouldn't necessarily say x265 bad. Rather, they simply haven't been taking it to the extreme levels of optimization that the x264 devs took x264. [1]
Up until the end of development, x264 was hyper focused on getting the best possible subjective quality with the smallest possible bitrate. To date, the x264 CRF metrics are (IMO) unparalleled in consistency. With other codecs a similar CRF mode is simply, well, shit. I can't just set stuff to "CRF 20" and expect the output to hit roughly the same level of quality. VP9, in particular, is terrible with this. In VP9 CRF is more closely related to the bitrate than the actual quality of the scenes being encoded.
To be clear, even with these critiques you SHOULD choose x265, vp9, or AV1 over x264 for your encoding choices. They have better specs that allow for better compression. However, they are also leaving a lot on the table for what they COULD do.
I current do VP9 + vmaf on each scene to set a CRF value (using my own thing similar to AV1AN). That gives good consistent results at minimal bitrates. It's just a little terrible (IMO) that I have to do so much work that the encoder should theoretically be able to do better.