My first computer was a TRS-80 Color Computer which had a tiny set of badly chosen colors!
Back then you did what you could with graphics and it wasn't a lot. After I got a PC I had indexed color for a long time and working with indexed color was pretty rough because anything physics-based like rendering or raytracing was going to be difficult. You could render a photo pretty well with 256 carefully chosen colors and dithering but if you wanted to, say, composite two photos and do general sorts of things you'd need to convert to "true color", do the math there, then re-quantize for display.
I had a herc clone on the 286 machine I bought around 1987 and later added a Super VGA card. One cool thing about the IBM PC was that the monochrome and color graphic systems were sufficiently different in terms of memory map and ports so you could plug in two graphics cards and two monitors and that's what I had.
I think this was true for MDA (text only), but the Hercules had 64k of video memory at B0000, the latter half of which would overlap with the CGA card, and also VGA's text mode.
Back then you did what you could with graphics and it wasn't a lot. After I got a PC I had indexed color for a long time and working with indexed color was pretty rough because anything physics-based like rendering or raytracing was going to be difficult. You could render a photo pretty well with 256 carefully chosen colors and dithering but if you wanted to, say, composite two photos and do general sorts of things you'd need to convert to "true color", do the math there, then re-quantize for display.