It traditionally leads to bigger executables and longer compile times. With modern compilers, both of these are non-issues for "most" reasonably-sized projects.
I can see it either way, but for a tiny program this could be handy. For a larger program that already has multiple source files, it seems like it'd be better to keep it in a separate file (to make the git diffs easier to follow, for example).
It would be better if it was namespace'd in some way, at least by using a common unusual prefix for all of the non-local symbols. It wouldn't be hard to clash with the chosen names.
> I thought it was poor form to have a lot of code in headers.
It is mostly because it makes it harder to reuse compiled objects when you make a change and rebuild. For a drop-in library that you're probably not editing, this shouldn't be an issue.