You do need enough free time to start said small business, though, as well as enough capital (even if not "massive loans"). For many people those requirements are pretty tough.
And don’t forget how many people have clauses in their employment agreements saying that any IP they develop on or off the clock belongs to the company.