[stymaar]

Is it a fully custom inference engine or are you reusing parts of an existing stack? (llama.CPP, vLLM, etc.)

[i386]

Our skippy library is a patch queue on top of llama that allows us to access internal information, such as activations, and filter tensors on model load.

[rao-v]

This really should be in the blogpost. It’s both useful info and basic courtesy to be explicit about which underlying inferencing engine you are using

[sig_kill]

We didnt post it, we use a library (iroh) who featured us - so we are here answering any Q’s instead :)