|
|
|
|
|
|
by joshuamoyers
11 days ago
|
|
|
I think it'd be very hard to achieve viable tokens/s or get arithmetic intensity to be high enough in general, since many cases in existing training and inference are memory bandwidth limited. Definitely seems possible to conceptually have a slow pipeline that is distributed though. |
|
|