[scottdupoy]

Interesting to see something like this!

My computer science masters thesis was based on the same goal. I used a 2D convolution which meant you can merge images with inexact overlaps. I had to run a high-pass filter first to limit the image details to their edges only or else the convolution incorrectly matched bright areas.

In reality merging pictures is further complicated because the source images may be slightly rotated relative to each other and also due to the images being slightly curved due to lens distortion.

My supervisor wanted me to do a PHD on the topic!

[gsliepen]

I used this for several applications. Note that 2D convolution can be done efficiently using FFTs, and filtering can be combined with this very efficiently: if you see your high-pass filter as a convolution of its own, you can pre-calculate its FFT, and just multiply it almost for free in the frequency domain with the two images you want to convolve.

[scottdupoy]

That's exactly how it worked, hand rolled FFT and filtering following the method in "Numerical Recipes for C"

[gsliepen]

Oh, Numerical Recipes is nice but their algorithm implementations are not really state-of-the-art. I highly recommend using FFTW (https://fftw.org/) as it will likely give you a substantial performance improvement.

[C-Naoki]

Thank you for your comments! For sure, the CNN is expressive for learning the characteristics of images. However, in this development, I tried to not use deep-learning because I believe that it is important to provide fast, consistent results without the need for training data. If you are particularly interested in this app, I would be glad if you could create a pull request to extend the algorithm.

[jdhwosnhw]

The parent comment said nothing about using deep learning. Convolution is not the same as using a CNN. I interpreted their comment as meaning they used a 2D convolution (presumably a 2D cross correlation, actually) to find regions of overlap

[scottdupoy]

Yes you're right it was a 2D cross-correlation which is very analogous to a convolution

[r_hanz]

If memory serves… the only difference is that one of the kernels being convolved is reversed for convolution.

[sitkack]

The images might not be coplanar and the overlapping composition should be 2d planes in 3d space or go full gaussian splat.