Diffraction of light was only understood in the mid 19th century. They simply had no idea what to look for. In Galilei's time, making optical instruments was mostly based on trial and error.
OK one easier way to confirm that Airy disk isn't a "star". Speed of Moon movement vs stars was well known and is around 0.5'' per second, and Airy disk was 4-5'' wide. So, when a star is occulted by Moon, it should have disappeared slowly as the Moon "slides" over it, during ~10 seconds. In fact, it disappears instantly, because Airy disk isn't a star...
They wouldn't have needed to know what to look for if they had noticed that the "width" they ascribed to the stars changed with every measurement. Every single telescope size would have led them to a different figure, and if they had cared about pursuing quantitative inconsistencies they would have not only realized that they weren't measuring the width of the star, but also that the determining factor was the telescope's aperture.