I find this a remarkable statement, what do you take empathy to be? Do you think that people can realistically care for others in a purely cognitive, non-emotional sense?
So if you saw your goldfish being clearly uncomfortable or in pain, would you have an emotional reaction to it, making you feel bad for it? That’s empathy. You don’t need to imagine the goldfish's daily routine to experience this. In the absence of this emotion, there is no incentive to perform the mechanistic actions that constitute “caring for” it.
Empathy is the process of correctly evaluating another's motives and internal state. While it's frequently assumed, it doesn't follow that one must necessarily feel sympathy for them.
You need empathy to outwit an opponent, for example.
I cannot empathize with a goldfish because I can't know what it's like to be a goldfish but i can care for it because I know its needs.
I can empathize with someone who is ill because they are human and I've been ill but i cannot care for them because I don't know their needs.