Lucky you. Some people are more effected by caffeine than others. My eldest daughter reports similar ups and downs as I do, my other daughter reports no large effects.
"Some people get jumpy after drinking a single cup of coffee, while others can gulp down a Venti Americano without feeling a thing. Part of that variability is due to the development of tolerance by regular coffee drinkers; but there are genetic differences in how people metabolize caffeine as well.
"
https://www.23andme.com/health/Caffeine-Metabolism/
---
"Coffee, CYP1A2 Genotype, and Risk of Myocardial Infarction"
http://jama.ama-assn.org/content/295/10/1135.long