[tomschlick]

The only header that I can think of that might slow down a site is Content-Security-Policy. Even that is negligible as long as you don't have 1000 entries.