|
|
|
|
|
by ttoinou
30 days ago
|
|
Ask a SOTA LLM when Newton was born without any access to internet : the answer is Lossless for our shared culture understanding of this question. Not Near-lossless, lossless. Ask the same LLM when YOU were born, the answer is just wrong for almost anyone in the world, not lossy. Between the two there is a whole new field of Lossyness to study. 90% depends entirely on what the measure means here, do you understand what "Normalized Discounted Cumulative Gain at rank 10" means to the set of data that we are comparing ? Sometimes coming up with new codecs (compressors decompressors) means coming up with new ways to interpret artifacts of the real world. And this is exactly why LLM are so powerful and they are like a giant Lossy (but Near-Lossless for various use cases) ZIP file / Database of the whole knowledge of the training data. Nobody is trying to manipulate you here, humanity just has to find new explanations for complex topics. Lossy-ness is binary
Lossless is binary in pure information theory. to quote my other comment :Lossless is objective for information theory. To get from the real world to digital world you need an analog to digital converter, this process is by definition lossy. We are interested in the real world, and information is pure but never represents exactly reality.
Lossyness is baked into our problem statement here. Using terms like near lossless means we think we are very close to reality for what we’re trying to do |
|