| I don't remember where I first saw it, but the best definition of "understanding" I've seen is "being able to encode and compress". For example, imagine a system that has as input the picture of a human face in RAW format. If the system runs the picture through JPEG compression, for example, and returns something substantially smaller, it has shown some understanding of the input (color, spatial repetition, etc). A more advanced system, with more understanding, may recognize it as a human face, and convert it to a template like the ones used for facial recognition. It doesn't care about individual pixels anymore, or the lighting, just general features of faces. It understands faces. An even more advanced system may recognize the specific person and compress the whole thing to a few bits. I would say that an OCR scanner understands the alphabet and how text is laid out, GPT-2 understands the relationship between words and how text is written. And a physics simulator understands basic physics because it can approximately compress a sequence of object movements into only initial conditions and small corrections. Lossy compression makes this concept non-trivial to measure, but it's still a world's away from the normal philosophical arguments. |