He’d have a long way to go to get down to the number of federal agencies that existed in 1900, let alone the number in 1800, yet the US had its constitution, then, too.
Since 1800 congress has used its constitutional power to establish agencies. It is congress who has the constitutional power to shut them down, not the president. The president executes the laws passed by Congress.