Yes, that brings back the memory of working through books by Andre LaMothe and implementing little games in DOS with C and a little bit of Assembler. I believe there was a very primitive graphics library included in Borland C, but it was not that useful for this task.
Andre LaMothe showed me the wonder of alternative graphics memory layout like Mode 13h and Mode Z. Though it confused teenage me at the time why you would have four memory segments each dealing with (offset %4) bytes in a line-oriented graphics buffer, it was magical when it worked and was one of the first times I had to let the code just work and move on.
Double buffering was an understandable neat trick and one that would have taken me a bit longer to discover on my own. He also had a custom controller wired through the printer port and some C+ASM for interfacing with it.
I had three of his books, I think the Black Art book was constantly on my desk in the 90s.
You do call a routine stored in VGA ROM (interrupt 0x10) to set up the mode, then do some port I/O to configure VGA registers and then access VGA memory directly. No "system libraries" from DOS involved as such (they are needed for things like filesystem access, allocating memory and dealing with command-line arguments and returning to the system, though).
Borland's dev tools came with "BGI" (Borlands Graphics Interface), but that's not necessary and wasn't really used for many games -- it provides abstract high-level drawing routines, like lines, circles, etc... that can be made to work on different graphics devices (CGA, VGA, ...). This was not necessary for direct graphics card access that most games used.