This is just not true. Google for example routes the profit it makes in France to other countries to avoid paying taxes. That's where the current tax systen doesn't work and a new one is needed.
Yes, they avoid paying some French corporate taxes. The French have been fully aware of this for decades. Even if they somehow missed everything else, the Lisbon Treaty was rejected by the Irish until they got assurances from the EU that it won't affect taxation. Part of the reason the EU exists in the form it does is due to this taxation.