I've stopped drinking coffee as it would frequently give me anxiety and sometimes panic attacks. I now drink tea or water at work. However, I still need to work harder on cutting down on sugar.
I suspected coffee was giving me anxiety which bordered on paranoia. I stopped drinking coffee and it went away immediately. A week later (today) I had a cup to see how I'd respond. Severe anxiety came back within half an hour. Very interesting.
I found that small amounts of alcohol have a similar effect. Keep drinking though and the depressant effects then start to kick in and the anxiety/paranoia subsides.