[nutanc]

This maps a little to what we are doing research on what we are calling as shape of stories[1].

We can clearly see in 2D space itself how different "concepts" are explored.

Using the shape of stories for semantic chunking we can clearly see in multiple articles how we can chunk by "concepts". [2]

Now we are trying to see if we can just use these chunks and train a next "chunk" predictor instead of a next word predictor.

In the paper, they take a sentence to mean a concept. We believe that a "semantic chunk" is better suited for a concept instead of a sentence.

[1] https://gpt3experiments.substack.com/p/the-shape-of-stories-...

[2]https://gpt3experiments.substack.com/p/a-new-chunking-approa...

[Lerc]

Can you spot conceptually similar stories by their shape?

For instance what is the shape of the ugly duckling compared to Rudolf the red nosed reindeer. They are essentially the same story, so presumably on some dimension you should be able to spot them in a group of unrelated stories.

[nutanc]

Will check for these particular stories. But yes, when we tried this on some stories with a similar arc we saw that their path is similar in the semantic space.