Hacker News new | ask | show | jobs
by jmvoodoo 347 days ago
You can't as far as I'm aware unless you control the entire batch during inference, or don't use batching which would require you to run your own inference.