I had approximately the same comment in a thread under a submission about a new CPU <https://news.ycombinator.com/item?id=44201707> and the replies highlighted my understanding of exactly how slow using sticks of RAM versus on-die VRAM. I'd still enjoy trying it, but I have not yet loaded enough of the FPGA ecosystem in my head to know how many hours I am away from watching the bits flow
Not exactly what you specified but close