One person under you for 3-5 years with a four attritions in that period (this latest one being Person #5?) constitutes an incredibly high attrition rate. That’s worth introspecting on. Average tenure of under a year?!
It seems likely someone managing a handful of employees doesn't decide salaries. If his unders left mostly for salary reasons, I don't see how that reflects badly on him.