[rxi]

Not even that: SDL just provides a pixel buffer, the application draws everything itself per-pixel. Lite uses a technique I refer to as "cached software rendering" which allows the application code to be written as if it's doing a fullscreen-redrawn when it wants to update, the renderer cache (rencache.c) then works out which regions actually need to be redrawn at the end of the frame and redraws only those. You can call `renderer.show_debug(true)` to show these redraw regions: https://youtu.be/KtL9f6bksDQ?t=50

I wrote a short article detailing the technique here: https://rxi.github.io/200402.html

[c-smile]

That's not anyhow different from typical GDI, CoreGraphics, GTK/Cairo way of doing rendering.

Windows, MacOS and GTK maintain internal pixmap buffer for a window.

And when needed you call InvlidateRect(wnd,rc) and receive WM_PAINT with cumulative rect to update.

Personally, I would create an abstraction that wraps couple of functions of GDI, CoreGraphics and Cairo and use it instead manual pixmap rendering. Will be faster and more flexible. All that UI can be rendered by just 2..3 functions FillRect, DrawText, MeasureText.

[barrkel]

It's quite different, actually, both in application programming logic and in expressive power of drawing primitives.

With dirty rectangles, it's the application's responsibility to minimize much of its drawing; the renderer will at best avoid copying bits where the target of the bits lies outside the drawing rectangle. The renderer can only optimize in situations where the entire call is understood as a full primitive, and it knows that the result will lie outside the dirty rectangle.

With rxi's approach, the application gets to define the commands which update the UI - which may be as complex as desired, as long as they have a calculable rectangle - and the cost of calculating that rendering can be skipped, without needing to query for dirty rectangles or doing any application-side conditional logic, beyond the layer that rxi wrote.

It's particularly powerful if the rendering primitives are higher level than those provided by the native APIs.

[frank2]

Interesting. Do you know any software whose source code is available that uses such an abstraction to paint the screen?

[c-smile]

Any classic Windows, MacOS or GTK application does that.

Check https://docs.microsoft.com/en-us/windows/win32/learnwin32/pa...

If your question is about unified wrapper for multiple platforms then wxWidgets will qualify : https://wiki.wxwidgets.org/Painting_your_custom_control

As my Sciter where I wrapped Direct2D/DirectX, Skia/OpenGL, CoreGraphics, Cairo, GDI+ into class graphics abstraction so rest of code is isolated from particular paltform/backend used: https://github.com/c-smile/sciter-sdk/blob/master/include/be...

[badsectoracula]

FWIW this is basically "dirty rectangles" which was a very common technique for avoiding full screen updates in games back when the hardware wasn't fast enough to do that.

[rxi]

You're equating the final stage of this approach to the entire approach. The point of this technique is that you get the benefits you typically would from dirty rectangles without the burden of the bookkeeping you would traditionally have. Using this technique your application "redraws" everything as if it's drawing it fresh each frame and the renderer cache takes care of determining what's actually changed.

Typically with dirty rectangles you would have to manage this state in the application code, for example, determining that line X was edited then updating the region for that line, or determining that view Y moved and updating a dirty rectangle based upon it's previous and current positions.

[badsectoracula]

From the description at least it does sound there is book-keeping for the tiles so i'm not sure why you think there isn't such a burden.

[barrkel]

It isn't. Normally the application needs to be aware of dirty rectangles; it fetches them from the window compositor, and it needs to limit its drawing calculations based on the dirty rectangle, in order to get the full benefit.

(Dirty rectangles are still perfectly common in desktop apps for when you e.g. drag a window back on screen after overlapping the edge, or if you scroll a window.)

[badsectoracula]

You are thinking about regions, dirty rectangles were a common thing in games to avoid redrawing the entire thing and rarely seen outside of them.

[derefr]

Nah, it's tile-based concurrent precompositing, like web browsers do. Each tile knows what's in it (i.e. what set of DOM elements); and subscribes to state-change events for those DOM elements; and any such state-change event will trigger the tile to re-render its cached texture. Then, on each frame, all you have to do to draw everything, is to grab the latest cached texture from each tile, and blit those (or set them up as a grid of flat-projected rects in screen-space, if you're in 3D-semantics land.)

You can get additional benefits from this approach, by doing multiple layers of it (e.g. having scrollable surfaces have their own tiles that precompute the inner-document-viewport-space rather than the outer-viewport-space, such that the inner tiles aren't invalidated by scrolling the outer viewport.) This technique ends up forming a tree of tiles, where tiles higher up the tree, when invalidated, re-render trivially by compositing tiles further down the tree into themselves. Thus, another name for this approach is a "precompositing tree."

The difference between this approach and dirty rects, is in the direction of information flow. In tile-based precompositing, the information only flows in one direction—from the user, through view-controller, into the DOM, to the tiles, and then out the display. Dirty rects, meanwhile, are a signal sent backwards, from the display system to the program, essentially telling it that the display system lost/discarded the information needed to re-draw an area, so could the program please send it over again. (The program doesn't even have to re-render in response; some dirty-rects implementations, like X11's DAMAGE extension, just involve the client application re-transmitting pixbuf data to the server from its own precomposited buffer.)

Also, dirty rects / screen-damage doesn't solve the problem of hardware not being fast enough; it solves the problem of hardware not having enough VRAM to do per-frame compositing from undamaged intermediates. In low-VRAM conditions, you can only keep around the final pre-composited image; and so any time you "damage" / make "dirty" a region of the screen (e.g. by removing an overdrawn element, which should have the semantics of revealing whatever was there before that element overdrew it) then you need to propagate a request back to the renderer to re-draw (and, for efficiency, re-draw just that region), because you don't just have an intermediate texture laying around for that window/stage-layer/etc. to re-source it from. If you did, then dirty-rects would never come into play, since you'd just re-composite everything each frame. (Which is cheap even on old-school CPU-only blitters—you just have to alternate which pixmap pointer you're basing your LOADs off of using either a rect-overlap check, or a mask-bitmap [which gets you 8 pixels' mask-states per LOAD.] Even the Gameboy can do it!)

[badsectoracula]

Like the other guy, you are thinking about regions (which is what the X11 DAMAGE / WM_PAINT / etc stuff use). Dirty rectangles is a method used in older games where the game kept track of -usually- sprites on screen and whenever something changed (e.g. a sprite moving) that part of the screen was marked as dirty (often implemented as a list of non-overlapping rectangles, hence the name). Dirty rects also flows only in one direction.

[bitexploder]

Just an aside. Everyone should play with SDL and write a simple game that you have to draw your own pixels. It’s extremely satisfying. Do it in C. It’s pretty simple and fun.

[ZoomZoomZoom]

Thanks for the write-up. I can see it being beneficial for rust-audio community, which is looking for the right approach to VST GUI.

[tambourine_man]

> …the application draws everything itself per-pixel

That’s what I meant, yes. Interesting approach and impressive.

[eyelidlessness]

Sounds like the pixel grid equivalent of a "virtual DOM"?

[konsumer]

I think this is correct, like in the sense that react uses a VDom. When you make changes, you sort of pretend that you are changing everything, but the rendering engine figures out the differences to the real DOM, based on the in-memory changes, and makes minimal edits to it. This is why you can use react with all kinds of things that aren't DOM or even web-related (react-native, react-blessed, react-babylonjs, etc.) I contributed to react-blessed & react-babylonjs, and wrote the main chunk of react-babylonjs's current fiber system. You essentially just use the VDOm to describe the full graph, and that graph doesn't have to be DOM at all.

[c-smile]

Not even close.

[eyelidlessness]

I'm open to the possibility that I'm that wrong in my understanding, but this didn't help me understand any better at all.

The technique does sound similar to me. Both (as I understand it) maintain a representation in memory of the final rendering and use a diff to determine which parts of the rendering to perform. The "virtual DOM" technique isn't strictly tied to a browser DOM, though the term is a reference to that, and React's (in particular) has been adapted to many other rendering targets.

I'd be happy to learn more if you'd be kind enough to explain what I misunderstood.

[c-smile]

virtual DOM implies DOM existence.

DOM based systems use so called retained mode rendering. But this one uses something that can be classified as immediate mode rendering.

Check https://docs.microsoft.com/en-us/windows/win32/learnwin32/re...

[barrkel]

A virtual DOM doesn't necessarily imply DOM existence; I don't buy that. It really depends on how persistent you want your "virtual" to be.

A virtual DOM, to me, means that the application renders by, every time, constructing data structures which are handed off to be reconciled with the display.

If, as in an HTML application, you render by means of a retained mode real DOM, then the reconciliation is via comparison of the virtual with the real. But that's not the only way to handle the output of the construction of a virtual DOM; it could figure out how those structures intersect with the dirty rectangle/s, and only render the subtree of the DOM which applies.

rxi's technique resembles a virtual DOM of depth 2 (1 root and everything is a child) and absolute positioning, though it's even closer to an OpenGL display list or combined vertex & command buffer. For that reason, I think it's a little bit of a stretch; not on the virtual DOM angle, but on the not particularly DOM-like nature of the drawing commands.

[c-smile]

> A virtual DOM doesn't necessarily imply DOM existence;

That's wrong. virtual DOM is always a parallel structure to real DOM - projection of it. That's by definition of it.

From vDOM authors: https://reactjs.org/docs/faq-internals.html

> The virtual DOM (VDOM) is a programming concept where an ideal, or “virtual”, representation of a UI is kept in memory and synced with the “real” DOM

[eyelidlessness]

> virtual DOM implies DOM existence

No, it doesn't. I addressed this in the comment above. React's virtual DOM has been used to render:

- Plain HTML (e.g. server-side rendering) - Native UI framework objects (e.g. React Native) - Text-based interfaces (e.g. Ink) - Smart TV devices (e.g. Netflix's Gibbon) - Browser `<canvas>` elements - Markdown formatted text

And a whole bunch of other targets.

> DOM based systems use so called retained mode rendering. But this one uses something that can be classified as immediate mode rendering.

This seems orthogonal to the question? I'm not trying to be difficult, I sincerely don't understand why this would mean the two are "not even close".

[c-smile]

> Native UI framework objects (e.g. React Native)

React Native uses DOM, the only nuance is that that DOM is a tree of native widgets/windows which is a perfect DOM.

Again, virtual DOM is a projection of real DOM in one form or another. It could be a tree of anything that can be represented by attributed nodes and leaves.

DOM tree has nothing with rendering and pixels, that's why "not even close". By using virtual DOM you can update (by diffing) some tree that even has no visual representation in principle - it is pure data structure. Think about abstract XML config that can be reconciliated with its virtual DOM.

[tasty_freeze]

Sounds like curses.