I'd definitely say a t-shirt if you can afford them. I have t-shirts from multiple hackathons and wear them all the time. They get far more mileage than any sticker or pen I've received.
They need to be good t-shirts though, which means a decent design, good fabric, light-on-dark, well fitting (hint: female attendees have different body shapes to male attendees).
All of which can push the expense up quite a bit.