[going_ham]

1. The trained model has 7B parameters or weights for each neuron.

2. It can handle upto 8k tokens. Tokens are usually some representation for a word. If your tokens are characters then, "h", "e", "y" represent 3 tokens for hey. Most of the algos use byte pair encoding. For example "hand-le" has two tokens "hand" and "le". This is a very crud example which is enough to give the gist but is not accurate. You can look into byte pair encoding for more details.

3. The token size 1.5T token means they have huge variations for input and output. Simply put, it was trained on large data corpus.

I hope this simplifies it. You can research further if you are interested! Hope it helps!

[thomasahle]

Don't just post ChatGPT answers as comments on hackernews.

This one doesn't even make any sense. Of course it doesn't have 7B parameters _per_ neuron.

[going_ham]

Sorry doc, I wrote that comment in a smartphone without putting any thought. What I wanted to say was: > there are 7B parameters. A parameter is a weight assigned to single neuron.

I hope this clarifies the answer now.

Now that is done I am quite curious on how you came up with the idea it was written by ChatGPT? I just wanted to simplify as best as I could. It’s funny you thought it that way.

What could I have done so that it didn’t sound like response from ChatGPT? I am asking it to prevent future misunderstandings. I thought my grammatical errors would be enough to show it wasn’t a ChatGPT response.

Looking forward to your reply!

[johanvts]

Don’t be too critical dr. Ahle. Maybe it’s a new single-neuron architecture.

[Sunhold]

Doesn't look like ChatGPT. Grammatical errors like "on large data corpus," the poor comma usage, misspelling crude, etc. are more of a human thing.

[thomasahle]

Maybe you are right. I was confused by sentences like "I hope this simplifies it. You can research further if you are interested! Hope it helps!" which seemed to be responding to a prompt other than just the previous comment.

[raincole]

This is a good example on why StackOverflow banned ChatGPT-generated answers.