I'm not sure that was the main factor in their decline. I think it was more that they were internally divided over what direction the party should take. The members that wanted the full-fat fash street violence left for the EDL. The remnants weren't able to effectively compete with UKIP's money and ground game at elections.