You absolutely were being an annoying prick. The best thing you can do is own up to it and apologize. Definitely not what you're doing right now, what with the whole "jees, what is your issue" thing.
The reason he's so touchy about it is he's a narcissistic annoying prick. He only did it for attention since people pay attention to bullies, and it requires no work or intelligence unlike being paid attention to for contributing constructively. Call him out as an obnoxious bully and his ego can't stand it.