Built-up tolerance? After drinking triple espressos like water in university I could drink coffee at 10 pm and still fall asleep an hour later. Of course, I'd wake up feeling more tired than before with a splitting headache, but that's what the triple espressos were for.
For me I've found that it takes a while to take full effect. I normally go to sleep around 9:30. If I drink a cup at 9pm I'll fall asleep okay at 9:30. If I drink a cup at 6pm, I'm going to have problems sleeping.