A single sidewinder is almost 200 pounds and GBU-31/32/38 JDAMs are 500-2,000 pounds each, so accounting for the pilot and ejection seat, you're only getting a small bomb's worth of weight.
Making them smaller significantly reduces the volume available for weapons, sensors, and fuel too much IMO. See the Predator, which doesn't have a large weapons loadout and still has the cockpit dome for the radar.
Making them smaller significantly reduces the volume available for weapons, sensors, and fuel too much IMO. See the Predator, which doesn't have a large weapons loadout and still has the cockpit dome for the radar.