In my experience they aren't too bad. I typically use an exacto knife and just scrape around the bottom of the port for a while until I stop pulling lint out.
I use a sewing needle to pull out lint. Using an exacto knife would scare me, but I'm glad to hear that it works - the ports are more durable than I assumed.