Y
Hacker News
new
|
ask
|
show
|
jobs
by
jmvoodoo
347 days ago
You can't as far as I'm aware unless you control the entire batch during inference, or don't use batching which would require you to run your own inference.