Great point. In general it differs from country to country. In Australia, they have a random testing program that allows police to breathalyze without a warrant or probable cause. However, in the US roadside breathalyzer tests are voluntary and police require probable cause (visible evidence of swerving, smell alcohol on the breath, etc) to request a test.
Even when driving, you have the right to revoke it (other bad things happen, like taking your license away, but you can still refuse).
Taking a random standerby and breathalyzing him without consent or a warrant would be wrongful search and seizure.