I'm not sure I understand this question. If I hand someone a book, we're both interacting with the book as well as each other, so why wouldn't this also hold true for particles?
I don't see why the answer to that question affects whether particles can interact by exchanging particles with each other. To be clear, I'm not taking a stance on whether that's an accurate description of how particles work, but I don't think that it's somehow logically incoherent.
The point wasn't that particles can interact that way, but that that is how they interact. But that requires infinite regression. (That needn't necessarily be a problem, but it does need a justification.)