Brain size in modern humans is also only weakly correlated with intelligence [1]. So they might not even have been stupid due to their smaller brain size.
Sperm whales don't have much of a chance to manipulate their environment, but they have the largest brains on earth [2].