That wouldn't be too hard if GPU's would have a stable interface. Try programming a GPU in Assembly language and see how that goes. The experience sucks, but that's the level that needs to be targeted in case of an OS.
For example, in the past Amiga computers had a 'GPU' (although much less powerful than todays GPUs) with a stable interface. It was a first class citizen in its OS. It also was incredibly easy to target in Assembly language.