It's not just space, it's mass. A solar panel can be made very low in mass. The physical limit comes from the absorption of light in a thin layer of semiconductor. For CdTe, this would be about 1 micron. PV in space could be gossamer thin sheets of thin film semiconductors, with tremendous power/mass.