Personally I've linked caffeine with anxiety. The more I take the more anxious I feel, particularly if I take caffeine a few days in a row. So now I'm only occasionally having some, like on weekends or on family holidays.
I tend to drink caffeine with I need to recruit mental energy. I find I don't get compulsive cravings to drink coffee and I can drink a single cup of coffee and be fine for the entire day. If anyone knows why this is, I'm all ears, I'm honestly shocked.