Open source doesn't necessarily mean that you can put it on the device. I'm sure a lot of Yubico's critics would be happy with seeing the code even if it can't be flashed.
Yes, it would technically meet the Open Source Initiative's definition (https://opensource.org/osd), but if there was no way to re-flash the device, no way to verify the binary on the device, or possibly even no way build a binary (which may require proprietary tools under NDA from the chip manufacturer) -- I think a lot of critics would still be critics, but I could be wrong.
If Yubico did this it would be very interesting to see the reaction.
The general issue is when all hardware has software in, in the end it has to be open source. Going even further: The distinction between hardware, firmware, and software is logically irrelevant in terms of trust.
If Yubico did this it would be very interesting to see the reaction.