[KomoD]

(IANAL) It can be both legal and illegal

If you don't respond to takedowns, that's probably leaning towards being illegal*

If you respond to takedowns and blacklist the hashes, you're most likely fine*

*obviously depends on the jurisdiction and on whether matching hashes to IP:PORT is considered distribution/facilitation/whatever (take TPB's case as an example)

I know someone who ran a pretty large tracker for years, when he received a takedown he just blacklisted the hashes and he's been fine so far.