Hacker News new | ask | show | jobs
by alexsubq 37 days ago
We are SOTA in some ways and not in others, continuously working to make it better! We need a little more time to scale, as we are working on things like disaggregated prefill, etc., the norms of large-scale model infra.

I am happy to answer any questions!
2 comments
supern0va 37 days ago
This seems super cool if as described, but I'm sure you can understand the skepticism.

Do you anticipate having any kind of public accessible chat interface for testing in the near future?

Also, what, if any, benefits are there for smaller context windows? Is there still a material improvement in cost to serve under say 256k? I'm curious about the broader implications for the space beyond improvements for very large context windows.

link
alexsubq 37 days ago
I do, for sure! Yes, we have a few product rollouts lined up. The differentials for latency are posted in our blog post, so that should provide an idea of where the scaling law differentials kick in.
link
dvfjsdhgfv 36 days ago
> I do, for sure! Yes, we have a few product rollouts lined up.

When, more or less?

link
alexsubq 35 days ago
We will have a few rollouts in the next two months.
link
dirtyalt 36 days ago
I have questions.

Can you back up your claims?

Why did you not release the white paper in parallel with the product?

Feels really fishy.

link
lelanthran 33 days ago
In this new knowledge economy, there is no benefit to publishing your secret sauce.

If I came up with a novel thing I'd monetise it first, because publishing it makes it part of the training that adds value to billion dollar corps with zero credit to me.

In the old knowledge economy I benefited from the credit assigned to me.

So, to me, nothing fishy at all.

link
alexsubq 35 days ago
What do you want in a whitepaper that was not in our blog post? There is time to add more before the whitepaper is released.
link
jazzypants 33 days ago
I'm not GP, but I would want a benchmark that actually tests the entire context window. A benchmark that only tests the first 128K tokens effectively tells us nothing about how well it works at its full capacity.
link
alexsubq 31 days ago
That makes sense! We are working on that.
link