It's not very helpful to study the ideas that form in human brains, though, since so many of them are nonsense. It would largely be a study of misconceptions and biases.
I think you could just as well have said "I believe our ideas are often formed on paper, when we write therm down. And that writing is part of the physical world. So I think the schism you presented would be void."
Does this get us into the question of whether anything mathematical exists before anyone (or anysomething) first thought of it, and exists even when no-one is currently thinking of it? If you answer yes, then mathematics has an existence independent of thought. If you answer no, then that conditional existence presumably extends to physics as well, especially if Max Tegmark is right about math being the reality of physics. In that case, then it seems to follow that the universe only exists when it is being thought about, a conclusion that appears to have a bootstrap problem...
I think I can avoid the whole issue by observing that I can imagine things that do not physically exist, and they are not made to physically exist by dint of a conceptual representation of them physically existing in my brain.
I think you could just as well have said "I believe our ideas are often formed on paper, when we write therm down. And that writing is part of the physical world. So I think the schism you presented would be void."