[aifooh7Keew6xoo]

I think the issue is that when the heat exchange occurs one way, the HRV is effectively a condenser.

Moisture builds on the heat exchange surfaces of the heat exchanger, particulates, spores, etc. in the air deposit nutrients and viable life in the crud, it thrives and multiples right into the incoming airstream for your residence.

When the window is opened the air mixes without a surface for condensation and there's much less immediate deposition so there's not an enriched medium to sustain accelerated growth entirely in the air stream, but it still probably occurs to some extent in a more diffuse manner that produces a significantly lower ambient spore/toxin/particulate load