That means they disagree with you on the definition of the word, but it doesn't make it a synonym of figuratively.