Usually it's "something else" that turns into RJ45 (as ethernet has a maximum length) - now if you're in a datacenter you likely can get raw RJ45 Internet).
And what if you just have rj45? Is a ethernet card also special? Transceivers aren't particularly special or hard to get either. Point is that's not what makes it a CPE, ownership does.
It's an old term used by telecom to refer to the phone they owned that's in the customers home. It has been used after by internet providers if they put a device in your home. If it's your own device it's not a CPE as seen from the isp perspective.