Yes! In fact, philosophers have spent a lot of time thinking about this exact problem:
https://plato.stanford.edu/entries/qualia-inverted/
https://plato.stanford.edu/entries/qualia-knowledge/