If this is happening on iOS, then your peripheral is sending a Play command upon connection. (I don't know about Android but I assume the same.) This is often desirable, but I could see why it would piss you off.
I have a pair of such wireless speakers, and experience it, and the engineers at the speaker company which the the customer support escalated me to is adamant that they do not send any such command. It stops if I change the category of the device in iOS Settings, leading me to believe the culprit is indeed Apple.