[rozim]

Data moats? Data not on the web, both for the first pass of training [1], and for fine tuning / RLHF [2].

Maybe compute would be a moat...can you run it in inference cheaper? Conditional computation, very quantized models. Probably training compute is not a moat however.

Not sure what SOTA is on callouts to oracles (e.g. calling out to Mathematica or some database) during inference, but maybe if you could use.

I would look at BloombergGPT for inspiration on a SOTA vertical GPT [3].

Probably speed to deployment & integration & sales would be a moat too, as once you have customers using your stuff your competitors will have more trouble selling to the same folks.

[1] In the recent interview on Lex, Sam Altman referred to "data from partnerships". [2] Probably very reviewed by human raters. [3] https://arxiv.org/abs/2303.17564