Not exactly what you are looking for, but if you haven't seen it, firefox has some really interesting blog posts about how their css engine works, e.g. https://hacks.mozilla.org/2017/08/inside-a-super-fast-css-en... that i personally found really enlightening
I've been considering writing such a thing. Although I've only implemented Flexbox and CSS Grid so far. The CSS specifications for those algorithms are worth reading if you're interested in this kind of thing. They're a challenging read, but not an impossible one.