The circle bit looks like a monkey tail, simple as that. And there was no need for a shorter word, as it really wasn't used in common speech or writing (the French "à" was/is used in the commercial sense).
My guess used to be that it started as a "Klammer-A" ("a in a bracket") and that at some point the joking extension from A to Affe (monkey or ape, everyday German does not make a distinction) went viral, supported by the tail-like shape. Now that I know that the monkey connection even exists in languages without the a -> Affe link I'm not so confident anymore.
I can only speculate but I believe it is due to the fact that it kind of looks like a paper clip shaped like a monkey ear (speculation on my part but I've heard that before).
Klammer is short for Büroklammer (= paper clip) and Affe is monkey (or ape).