[ant6n]

This says it works on just-noticeable-differences. Would this work well if the quality of the compressed audio is very poor? Could one for example compare two speech codecs at 8Khz, 4bit against the original source to find out which one sounds better?

Or should one just... I dunno, calculate the mean squared error in some sort of continuous frequency domain, perhaps weighted by some hearing curve.

[mrob]

Audibility of error (and sound in general) depends on what other audio is playing at the same time, with both frequency domain and time domain effects:

https://en.wikipedia.org/wiki/Auditory_masking

Here's a two-part lecture with audio demonstrations by Bernhard Seeber of the Audio Information Processing Group at the Technical University of Munich:

https://www.youtube.com/watch?v=R9UZnMsm9o8

https://www.youtube.com/watch?v=bU0_Kaj7cPk

A simple weighed frequency domain error calculation is not very useful for comparing lossy audio codecs, because effectively exploiting auditory masking to hide the errors is a major factor in codec quality.

[jononor]

PEAQ/PESQ and visqol is worth trying for that. In principle they operate as you suggest. I keep a short overview of audio quality methods/tools here: https://github.com/jonnor/machinehearing/blob/master/audio-q...