[chungus_khan]

> Won't the screen-framebuffer be on gpu?

Not in X11. The front buffer needs to be in memory because it gets populated on-the-fly from windows via the Expose event, and the processes that those windows belong to need to be able to draw to it as a result. The protocol was also designed before there was any modern concept of "GPU memory", and couldn't assume the presence of anything more than very primitive 2D drawing hardware.

Another thing that can cause some extra memory use is applications caching pixmaps in the X server. X11 lets you register pixmaps server-side so that they don't have to be repeatedly transmitted across a network. This is great for X forwarding over networks, but largely unnecessary on modern hardware where both the server and clients reside on the same machine, and can cause the server's memory usage to balloon, especially with applications like web browsers.