[suppressingfire]

Depends on the problem and how you distribute data.

e.g., each node can generate its own subset to check, so you only need to distribute the program and input. That can be done using a multicast tree of some kind which brings distribution costs back down by a log.

[dhume]

But if you only have a finite amount of hardware available (which is probably the case), each node's data set grows exponentially with input size.

If your node population increases exponentially with input size, you're still running into the speed-of-light delay problem mentioned above (cube root of exponential is still exponential). If your solution is polynomial time with polynomially many nodes, your space cost is also polynomial.