Yea, in fact I think it's the second law (no entropy decrease) rather than first law (energy conservation) which is the restriction. Devices up-converting photons to higher frequencies require power input.
But you can turn three infrared photons into a single red photon, if the energy levels add up.
That would, for instance, allow you to make safety goggles for detecting hot spots. Could be useful for firefighters or factory workers.
The problem with a passive device would be that if you walked past something hot enough your goggles would white out (well, red out), and you might be effectively blind. At least with the passive device you can turn it off briefly so you can see where you're going.
> But you can turn three infrared photons into a single red photon, if the energy levels add up.
You can't do this with a passive device, because the single visible photon has less total entropy than the three infrared photons. Yes, it satisfies the first law of thermo, but it violates the second.
Thanks for emphasizing this, and I agree I need to qualify my statement. I haven't completely understood the restrictions yet, but the effectiveness of this device has to be conditional on the fact that the IR laser is a laser, or otherwise is in some low-entropy coherent state. Otherwise, it could be used to transform part of a uniform IR bath into high-frequency photons, passively generating a temperature gradient from nothing.
The incoherent case is the relevant situation when you're talking about passive glasses for viewing a scene in ambient IR light.