[colanderman]

I would never say either of those sentences.

"I passed by the man in the red hat."

"I threw out all of my old, useless papers."

Both of those not only keep the preposition away from the end of the sentence and keep it near the verb to which it belongs, but most importantly they sound more natural.

Your example of "Who am I speaking with?" follows this same pattern -- "with" is kept next to the verb "speaking".