|
To store the result of a coin toss requires 1 bit of information, I can either give you a 0 or a 1. But implicit in me communicating that to you is that it is 1 'out of' something, namely 1 'out of' 2.
To store a trit, ie a 0, 1 or 2, requires 2 bits. You will recieve either a 00, 01, or 10. You will never recieve a 11, because that is not a valid trit.
Huffman compression reduces the ammount of bits needed to send a block of data, by prematurely terminating the data communicated once a specific sequence has passed. So the huffman compression for a trit value of 0, would not be '00', it would be '0'. We pass fewer bits, because some sequences carry an implication that this is the end of the sequence. The reciever has to know which sequences indicate a termination.
A trit translates to 2 bits, with a wastage of .5 bits. The concept of 'compression' only makes sense when person A throws a sequence of bits at you, and when they stop, you have to interpret what they just said in order to work out what it meant. In truth, the extra bits may not have been sent, but they are implied.
When I write '1' on a piece of paper and slide it across the table to you, I am cheating, I am being ambiguous. 1 out of what? shold be the correct response. A bit is more accurately communicated as 1 out of 2. A 'double' would be 1 out of a million ish. You can only store numbers in a binary computer, out of something. A 1 stored in a bit field is a different quantity than a 1 out of a million ish. In lazy human parlance a '1' represents 1 out of infinity, but this is not technologically possible to store. No computer hard drive can store enough digits to contain the information that 1 out of infinity represents.
If I toss a coin and tell you it is a 1 out of 2, this is valid. But if I have cheated, and tossed a double headed coin then from my perspective the '1' aka 'heads' is 100% predictable, and therefore, for me, does not contain any information. For you, ignorant of my cheating coin, the data I communicate to you is 1 out of 2. What is the entropy of that information? Is it 1 bit, as you would believe, or 0 bits, as I would believe? The answer is that ALL MEASUREMENTS ARE NOT PROPERTIES OF OBJECTS, BUT OF RELATIONSHIPS BETWEEN A MEASURING SYSTEM AND AN OBJECT. The entropy of that coin toss does not live inside the data communicated, it lives inside my head (where the entropy is 0 bits) and also in your head (where the answer is 1 bit). The coin toss measurement of '1' does not contain either 0 or 1 bits because a piece of data in isolation does not have any meaning. Only in my head or yours can it's meaning be measured.
|