The comment below mine, about "bit" being used as "piece", was my first thought too. But then the author contrasts with a quote using the actual entropy definition of a bit.
And chunking doesn't work as an explanation either, even with Huffman encoding, unless half of all words you ever use are "cat".
Note that even if the word was used incorrectly, it makes no difference on the author's argument.
If "cat" is a foreign language word that you just learned, it is definitely more than a bit of information. If you are a native speaker of English, "cat" is now for you a single "chunk", or piece of information.
The comment below mine, about "bit" being used as "piece", was my first thought too. But then the author contrasts with a quote using the actual entropy definition of a bit.
And chunking doesn't work as an explanation either, even with Huffman encoding, unless half of all words you ever use are "cat".
Note that even if the word was used incorrectly, it makes no difference on the author's argument.