[miven]

Any guesses as to why they bumped the parameter count up from 7B to 8B?

[matrix2596]

I also wondered the same and check the model configs. they are using bigger vocab size and the intermediate size of fully connected layer seems to be bigger.