Exactly. And I haven't found cheap charge controllers that support the voltage range of Li-ion packs (like 12.6V for a 3S pack or 16.8V for a 4S pack).
There are actually quite a few well-developed IC's specifically intended for this application. They act both as buck converters for the output device, BMS for the lithium cell / cells, and a charge controller for the cells.
Andreas Spies has a great video on comparing some modern IC's for this -> https://www.youtube.com/watch?v=KaiQDS9jObY