The only downside I've found is visual: Sugru includes some tips to give it a good-looking aspect, but of course it won't look as cool as if it wasn't there. Still, I'm very happy with it.
I've read some comments about Sugru not being flexible: in my experience it's been very flexible and durable and will keep using it for every new Apple adapter I buy.