They're charging $7.99 for one-hour delivery. That's about on-par with on-demand courier fees that I've seen in NYC. Presumably the two-hour free delivery will be phased out once they've reached some metric.
If we assume that a) the 1h delivery is break even for them and 2) the 2h delivery would allow the courier to do a couple of extra deliveries on the same trip, then I think the per item cost for 2h delivery could be only a couple of dollars, making it competitive with UPS and thus maybe free for prime members indefinitely.