I've been working on an ActivityPub implementation and there are so many edge cases and SHOULD vs MAY recommendations, it's ridiculous. I'm planning a separate blog post on the intricacies of the standard.
I banged my head against this issue quite a bit last year. Specifically the thing that tripped me up, is how many fields can either have single value or be a list of values, and figuring out what is meant semantically when it's a list or a single value.
This is exactly what frustrates me the most and what most of my logic checks. I'm writing tests for each of these semantic cases, but it seems silly to me that a standard leaves something so ambiguous.