Good idea! I'd want to do some research and simulations to figure out how much of an altitude hit I'd take with the added weight of an additional controller.
This[0] is the ESP32 module that I keep on hand. The seller doesn't list any information that would help with your simulations, but the photos can give you a rough idea of the physical size. Other than a bare ESP32 module[1] I don't think I've seen a physically smaller ESP32 board.
You can program these with the Arduino IDE, so they're quite easy to use and there are tons of libraries available for interfacing with sensors, etc.
I've used TinyPICO Esp32 modules. They're even smaller than D1 Minis! They lack the metal cover, though I don't quite know what that's for (mumble EMI?).
Looks like the pocketbeagle has two 'programmable real-time units' PRUs @200MHz which could be used to run the sensors while the main CPU handles video+I/O. Could be a good option for 'multithreading' without having to insert another module. (caution: I've kept to the rpi for weird little side projects, so haven't actually tried this. Double super big caution: Never add hardware before profiling your current system to figure out where the bottleneck is... it might just be a matter of changing your 'main loop' a bit.)
You can program these with the Arduino IDE, so they're quite easy to use and there are tons of libraries available for interfacing with sensors, etc.
[0] https://www.aliexpress.com/item/32834982479.html
[1] https://www.aliexpress.com/item/1005001271041790.html