The times here seem unreasonably bad even with the bad algorithm. Something else has got to be going on. Maybe kind of hidden factorial complexity when it tries every combination?
Yes, I've never written a line of Swift, but these cases don't seem to be of the usual variety that cause Hindley-Milner to blow up. It seems like the Swift compiler source is available, and these test cases are small. This is encouragement for someone to spend a small amount of time digging into this, just for the curiosity of it. And I mean, just something like, fire it up in the debugger, start the compiler on one of these test cases, interrupt the when it looks like it is "hung", but before it times out. Step through the code a bit, to identify the handful of functions that that we're looping through, and then report out what you find, and your best guess, the algorithm is implemented correctly and is hopelessly intractable, or "hey, didn't we already check down this branch previously?". I'll give you my next upvote on HN.
Another thing I would add to swift as a flag is to make imports based on specific files vs. an abstract "module", there is a lot of repeated work that happens because of that last time I looked.
https://static.aminer.org/pdf/20170130/pdfs/popl/o8rbwxmj6h2...
https://github.com/apple/swift