[justsee]

I find it curious you would not consider education a significant variable.

Education and intelligence mean there's a significant variation in people's abilities to infer, to the point that particularly talented people get write-ups in 'Guinness Book of World Records.' and human-angle stories at the end of nightly news bulletins.

So the assumption that it's a binary answer to Molyneux's problem seems to be the first error. There would be certain individuals, who when adjusted to sight enough to work out the ratios of this color to that color could find enough data to make a choice that's better than a random guess.

However the fascination with this question isn't around those individuals who'd pass the test. It's fascination with the idea that most of us wouldn't, because as you outline the absence of input through the visual cortex mean the brain would not be able to make simple mappings sighted people feel are inherently 'natural'.