[johnjones4]

I'm not totally certain. I could do a quick C++ POC to see if that improves throughput a bit, but I've got doubts. My other thought is that the SPI interface only can go up to a certain speed. If anyone has deeper knowledge of that protocol I'd love to hear their thoughts.

[chrisdalke]

You should try doing some profiling on each part of your system and find where the bottlenecks are.

anecdotally, I built a telemetry system for an electric boat with very similar architecture (RPi, python script for telemetry collection, 433Mhz telemetry radio) and were pretty easily able to push on the scale of ~100 data points at 100hz. Part of that was efficiently (and manually) packing each data point into the minimum space required, pretty much just the raw data and a checksum. But from what you’re describing I definitely think you can push your system further without rewriting significantly.

[johnjones4]

Really helpful! Thank you! Would you happen to have your source code available anywhere for me to take a look?

[chrisdalke]

Yeah, this is the source code: https://github.com/URSolarSplash/Telemetry-Server. The version uploaded there is transmitting at ~4hz but we cranked it up to a much faster transmit loop when necessary. A few interesting files:

The main server loop: https://github.com/URSolarSplash/Telemetry-Server/blob/maste... The device implementation for our hand-rolled serial protocol: https://github.com/URSolarSplash/Telemetry-Server/blob/maste...

In classic "college engineering project" fashion, the project is rather disorganized and very specialized to the technology we used on our boat at the time. Our telemetry server had an in-memory cache of latest data points which were relayed over serial with all the telemetry nodes. We used the telemetry system to carry throttle and all other control signals for the boat. We replicated all the data points to a secondary server running on a shore computer, so we had two separate sources of data logs.

I will note that since building that system, my philosophy has changed several times -- Today, I would likely not roll our own telemetry protocol, instead using something like https://msgpack.org/index.html or going all-in and using ROS on a larger project.

If you're interested in chatting more, feel free to send me an email (my username @ gmail). It's always great to see others go through the same experimental process with telemetry and data collection systems! (Also, check out bps.space on youtube, who has done a lot with telemetry on model rockets.)

[joshvm]

I would have thought that the max SPI speed on the Pi is probably faster than your peripherals can deal with. Your bigger problem is system interrupts which might cause discontinuous capture. The speed is derived from the core clock which can go to well over 100MHz.

Control of the bus is probably handled by kernel drivers anyway, so you're not doing much overhead with Python.

Also make sure you use hardware video encoding. I don't know if the picamera library does, but worth checking. I normally use OpenCV with the omxh264 encoder. You could also run a gstreamer pipeline in parallel with your air code.

Edit: looking at you pipeline you use the lsm303 which is I2C not SPI? So that's 400khz max. And the Bosch BMP something which isn't designed for rapid measurements - even if it's over SPI they output of the chip will be your limit there.

[mch82]

Donkeycar might give you some ideas... it logs still images with a RaspberryPi camera module. The frames can be stitched into video afterward (though maybe lower frame rate/resolution than what you want). If it crashes, logged frames are on disk.

[johnjones4]

Thanks!