It's meant to listen locally all the time. If Google/Amazon wanted to be evil and snoop on us it would take exactly the same amount of effort.
Besides, many phones listen all the time too.
The only real difference is that the Echo has better microphones than your phone and is powered on all the time so you don't have to worry about battery life. On the other hand you probably say more sensitive things to your phone.
It would be funny to let it listen to a Numbers Station[1] all day long. If they are actually listening and processing, it might make for a fun feedback loop particularly of any theories about those stations are true.
or setup your own number station from a russian VPS and waste millions of tax dollars as intelligence agencies around the globe try to crack your garbage data
"You're violating our anti-AI confusion End User Agreement, so you are banned for life from Google. Anyone associate d with you will have their GMail account deleted as well. Please contact Google support if have any questions".