That's a stretch. One can hold the view that division of labour is a useful economical principle, but also that oligopolies represent a dangerous concentration of power.
If that were true, you'd expect some correlation between amount of colonisation done in the past, and wealth today.
(And even more, you'd expect to see a causal connection. So you'd need to exclude the alternative explanation that rich countries engage in a bit of colonisation as a rich man's hobby. And there's plenty of other correlations without causation.)
We could run a formal statistical analysis. But just have a look at Portugal and Spain. They ran extensive colonial empires, but aren't exactly rich these days. Germany and the Nordic countries were rather less involved in colonies, and are rather rich.
Russia ran and runs an extensive colonial empire, and they are comparatively poor (and we know that what wealth they have comes from oil and gas).
The Netherlands ran a big colonial empire and they are rich.
> I’m talking about my home, the dear old US of A.
The core of what became the US was already rich before they even stopped being colonies, and long before they had any colonies of their own.