Plenty of isps that provide internet over regular ethernet. But it's a ye olde telecom provider term that referred to the phone, that you also didnt own yourself. Doesn't always apply cleanly these days.
Usually it's "something else" that turns into RJ45 (as ethernet has a maximum length) - now if you're in a datacenter you likely can get raw RJ45 Internet).
And what if you just have rj45? Is a ethernet card also special? Transceivers aren't particularly special or hard to get either. Point is that's not what makes it a CPE, ownership does.
It's an old term used by telecom to refer to the phone they owned that's in the customers home. It has been used after by internet providers if they put a device in your home. If it's your own device it's not a CPE as seen from the isp perspective.