[rwz]

0.5t/s is still too slow even for email. For a moderately large inquiry (1MTok output, let's ignore the 4k context window limitation for now) it'll take the model around 23 days or uninterrupted execution to answer a single email.

Real world inquiries are gonna be much slower of course, but this setup is still too slow to do anything meaningfully useful I think.

[springtimesun]

I’m sure it’s possible, but I really struggle to think of an example that would result in a 1m token output.

[8note]

it would be 1M tokens worth of work, with some small report for the end

[0cf8612b2e1e]

Sure, you cannot do 1M, but there are plenty of useful questions you could ask that are far more modest. Simple Q+A, look at this function, how would you design X? All of those could have few paragraphs of outputs that would finish within a day.