[londons_explore]

If you wanted to do this 'opensource', your best bet is to try and get the GPU to do the work. You'd need hardware/HAL support for streaming raw data at ~15Gbits to GPU accessible memory, and after that you can probably do most of the rest using software without hardware/kernel involvement.

At a minimum, the GPU needs to be able to generate image scale pyramids, align them using optical flow, have a sensor noise model to detect misalignment, and take a running-mean of every non-misaligned pixel in each frame seen. This technique only needs enough ram for a handful of frames.

That should get you a good chunk of the way to a decent image.

As usual, the actual math is quite simple in python, but when you need it to run at 15 Gbits you'll be spending a really long time optimising assembly code for whatever GPU you're using...

[verall]

The SoC in the fairphone has an ISP, they can just use that.

[londons_explore]

The ISP is locked down and can only run qualcomm-code. Perhaps with the right licensing agreement they'd let you run your own code on it.