The thinking language doesn't have to be the same thing as the expression language.
It can still have a language form (manipulation of groups of symbolic structures, terms, and associations), but doesn't have to be English, or even at the conscious "internal monologue" level.
How can you know that it's symbolic if you have no conscious access to it? Processes happen in the brain, some results in explicit symbolic representation, others not so much. Bringing "language" into this does not achieve much besides the fact that language we use to communicate plays some role in internal monologue.
It can still have a language form (manipulation of groups of symbolic structures, terms, and associations), but doesn't have to be English, or even at the conscious "internal monologue" level.