Wow, this is really cool! Is inference running on GPU or CPU? I wonder how well this would run if model storage and inference is done on a jetson nano!
Thanks! The inference is currently run on GPU (a g4dn.xlarge instance), but can be swapped for CPUs by simply changing the compute request in the configuration YAML.
In theory, a nano has more than enough memory and storage to run it, but with how tied into AWS the current implementation is, I don't know how well it would run on a Jetson Nano without some hackery. Admittedly, however, I've not done much with the Nano, so I don't want to come off as overly confident here.
In theory, a nano has more than enough memory and storage to run it, but with how tied into AWS the current implementation is, I don't know how well it would run on a Jetson Nano without some hackery. Admittedly, however, I've not done much with the Nano, so I don't want to come off as overly confident here.