[csande17]

Salesforce has created a computer program where you put in a small prompt, like "Wikipedia page about badgers" or "News article starting with the line, 'Donald Trump was impeached today'", or "French translation of 'I like pears'", and it tries to predict what the text will be. You can also run the program in reverse, where you put in a snippet of text and it predicts whether it came from Wikipedia or a mystery novel or the fitness subreddit.

Salesforce created the program by first writing some relatively simple linear algebra, then fiddling with the constants until the output happened to look right. Their program contains 1.6 billion constants, which is more than any other program of its kind.

This program is also special because Salesforce has released it publicly; other organizations, like OpenAI, have previously claimed that text-generation software is too dangerous to release to the general public.

[lixtra]

> writing some relatively simple linear algebra

Except, that it wouldn’t work if it was purely linear.

[csande17]

Right, yeah, it's linear algebra combined with a few non-linear functions. The point is that Salesforce didn't come up with an algorithm that generated English text by writing a grammar or thinking really hard about what sentences look like—all the functionality comes from the "training" process that set the constants.