As a Swede living abroad this creates a headache when going back. When combined with swish needs a phone number but to get a phone number you need to have a registered address, etc.
This is a real and important problem: lots of countries have a System, like Vipps or WeChat Pay, which works brilliantly for local residents but is inaccessible to non-residents. Even if (as it sounds like in your case) you're a national but not a resident.
One example is flee markets, or different type of second hand. Christmas markets etc. Some do accept debit / credit cards but a lot is swish only.
For cards, depends on the bank. I know my bank at-least do send the card abroad to my current home and as long as i keep my digital authenticator ID (bank ID) i am able to access that, renew cards. Do most banking services etc.
I also managed to get a new bank authenticator by going to the embassy and get signed papers etc, however it took about 5 months or so if you don't have any cash that might become a issue. :)
Lots of smaller cafes or people running various stall style businesses only accept Swish, especially out the in more rural areas. I've been to a couple of smaller festivals where the only way to pay for a ticket at the 'door' was with Swish.
Plus of course any time you need to give money to an actual person and not a business.
If you're living abroad and visiting you'd probably already have a debit card, no need to open a new one for a visit. This is only relevant for Swish because it's not something someone living outside of Sweden would likely already have.
I'm also a Swede living abroad. I have signed up with Swish using a non-Swedish phone number (but with Swedish bank account), it works fine in most cases. I do have an address though, but not in Sweden.