For Switzerland it would be significantly worse since they are surrounded by Germany, France, Italy, and Austria. So all trucks would be stopped at border. Food prices are already super expensive and this would make it worse.
They think it's a failure because their political class refused to use the new powers to implement the will of the people on immigration, not due to economics (which, again, were not affected).
Brexit was a referendum on immigration. It was the number one issue. Yet, successive governments since Brexit have turned immigration up to 11. No wonder they think it's a failure, the will of the people was ignored (and it's having real political blowback, with both major parties about to get reduced to dust according to polling).