| A few pointers: The first bit is that practically all that is argued in for here, is in Android - and has been so for a long time (~2013 last I worked on the topic in that context) and it is still not enough - they will go (and are going) higher refresh rate. Even at the time there were crazy things like tuning CPU governor to wake up when touch screen detects incoming 'presence'(before you can get an accurate measurement of where and how hard the finger hit) - it's not like you are going to change your muscle intention mid flight. Input events were specifically tagged so that the memory would be re-used rather than GCed and reallocated. A sunny frame could go from motion to photon in 30ms. Then something happens and the next takes 110ms. That something is often garbage collectors and other runtime systems that think it is safe to do things as there is no shared systemic synchronization mechanism around for these things. The second is that this is -ultimately- a systemic issue and the judge, jury and executioner is the user's own perception. That's what you are optimising for. Treating it as a graphics only thing is not putting the finger on something, it is picking your nose. The input needs to be in on it, audio needs to be in on it, and the producer needs to be cooperative. Incidentally, the guitar hero style games and anything VR are good qualitative evaluation targets but with brinelling like incremental loads. (Larger scale)
1. communicate deadlines to client so the renderer can judge if it is even worth doing anything.
2. communicate presentation time to client so animations and video frames arrive at the right time.
3. have a compositor scheduler than will optimize for throughput, latency, minimizing jitter or energy consumption.
4. inform the scheduler about the task that is in focus, and bias client unlock and event routing to distinguish between input focus and the thundering herd.
5. type annotate client resources so the scheduler knows what they are for.
6. coalesce / resample input and resize events.
7. align client audio buffers to video buffers.
8. clients with state store/restore capabilities can rollback, inject and fastforward (http://filthypants.blogspot.com/2016/03/using-rollback-to-hi...) There is a bunch of more eccentric big stuff after that as well as the well known quality of life improvements (beyond the obvious gamedev lessons like don't just malloc in your renderloop and minimize the amount of syscall jitter in the entire pipeline), but baby steps. Oh, and all of the above++ is already in Arcan and yet it is still nowhere near good enough. |
It tells just how much more garbage been thrown into the linux stack since 2007, that even, effectively, kernel based, and hardware accelerated animation stutters today.