Stress is not just a mental state. There are real, physiological changes, such as rising cortisol levels. To say that it is just a "psychology dependent" situation is to say it is all in your head.
I developed an extreme sensitivity to caffeine after overdosing on another stimulant.
Before, I could drink 5-6 espressos without a problem, after the incident, ~25mgs of caffeine gets pretty uncomfortable.