Perhaps the etymology of the word will help with the connection to the taste. In Dutch (another Germanic language) mealy = 'melig', which is close to 'meel' (: flour). There you have the connection to the sawdust-taste. Sawdust and flour, i think, taste pretty much the same.