| Hi HN, I am building a Terminal User Interface (like Claude Code) for self-hosted AI agents on Jetsons. Works in air-gapped environments.
Unlike other solutions, this is optimised for unified memory machines, as to avoid OOM errors. The agent can do stuff like edit, read, create files - manage and interpret data locally. Currently, it gets ~17 tok/s on Jetson Orin Nano 8GB using Qwen3-4B-Instruct-4bit
In the future, adding TensorRT .engine support which will boost inference further. I am trying to get the memory footprint down, so if anyone has knowledge on kv cache optimisation, that would be great. I would love to get your feedback and people try running it on more capable devices and models - post your results here. Run
```
pip install open-jet
open-jet --setup
``` Webiste: https://www.openjet.dev/
Directly on Pypi: https://pypi.org/project/open-jet/
Repo: https://github.com/L-Forster/open-jet/ |
A big thing for us is bandwidth constraints on the inputs and outputs, S-Band radio in LEO is like a dialup connection that works for 10 minutes 2-3 times a day so pushing updates takes ages and even just live interactions during the access windows can benefit from brevity.
I'll definitely show this to my team, would be interesting to see what kinds of benefits we could get from this.