FastMail supports CalDAV. I use my FastMail calendar with Thunderbird (Lightning) and on my iPhone; works great. They also support CardDAV for contacts. /satisfied FM customer since ~2008 or so
Why is jjawssd's (sister) comment dead? Davdroid works great and is free (as in beer and speech), though I would encourage people to donate if it's useful to you.
I use Radicale and find it great, but there's no UI, so you need to use whatever client you want that supports CalDAV (I use Lightning and the calendar on my phone). Lately I've been liking Nextcloud a lot, and that's a one-stop solution for lots of things, so nowadays I would recommend that if you have a home server or want to pay someone to host it.
Yeah, you do. As I said above, Fastmail's calendar is very good too, and you can load your self-hosted/CalDAV calendars into it, so that's a good option.