With higher memory usage you have less free memory for caching so you hit the disk/SSD more often. If it gets too high you start swapping which is really bad.
It isn't a surprise when you consider indirect power costs: CPU use filling that memory and reading it back later, extra IO because the memory consumed is not available for caching purposes (or worse because paging out to more power-expensive storage is happening), and so forth.
It is like when people say page fragmentation in SQL Server isn't important any more because random access is so fast due to SSDs and other IO subsystem improvements – pages are held in RAM in the storage format so if they are only ⅔ filled you are wasting ⅓ of the allocated memory which can have significant performance effects if your common working set is not smaller than the memory available. Though in fairness, I would agree that this issue is usually quite far down the list of things that need addressing in poorly performing database & applications.