[morpheos137]

ai can abitrarily closely fit the human corpus. why people expect it to magically achieve superhuman qualities is beyond me. we got a very good statistical interpolator. how do you go from there to superhuman when training is on the human corpus and alignment is by RHLF?

[orbital-decay]

This is a simplistic take. It's not a mere interpolator by any measure, there's a ton of research on that, starting with the basics https://arxiv.org/abs/2309.10668v2

[morpheos137]

again, try thinking critically it is not merely an interpolator means it can interpolate on many dimensions. it does not follow that greater than human capability results from doing so. explain to me how a statistical function approximator (which is what a transformer is) with human training input and human tuning (rhlf) exceeds the aggregate human cognitive envelope? What is the mechanism? Let's say an LLM makes an inference that no human could have possibly made (arguably impossible itself) how does the inference survive rhlf or become useful to humans if they can not judge its validity? how do you take the shape of the human corpus and all its gradients and some how arrive at something greater than human, where was the missing information hiding?

[Timwi]

> how do you take the shape of the human corpus and all its gradients and [somehow] arrive at something greater than human, where was the missing information hiding?

Well, how do humans do it? Scientists discover new stuff that isn't in any corpus. Even I as a lowly computer user occasionally figure something out about a software without reading a help screen. It's obviously possible to arrive at new information by interpolating existing information.

[morpheos137]

yes and it is imposible to verify and evaluate appropriately such information without empiricism. Any empricism LLMs show is stylistic mimicry not a hard coded operational constraint. You can prompt an LLM to test its claims but what it is really doing is still genrating plausible completions not following a proceedure. So of course new things can be discovered. The point is for them to be useful requires iterative real world grounded refinement and or subject matter expert judgment. The error is assuming scaling magically turns a prediction algorithmn into a cognitive agent that can exceed its masters. it doesn't. even if llms generate profound insights accidentally by definition if such insights are not in the corpus they are not retained given frozen weights and if beyond the human capability envelope the epistemically blind llm has no way to ensure retention if they arise during training.

[orbital-decay]

Sorry, I just noticed I posted a wrong link in the comment above. Here's the proper one: https://arxiv.org/abs/2110.09485

[morpheos137]

ok however i would say extrapolating the current data set is not a way to exceed the the human envelope. it is unclear to me the human evelope has been demonstrated as a convex hull or how transformers could find points outside it. in other words intelligence and knowledge does not exist as some abstract possibility space but only as a set of contextual contingences. LLMs have no context beyond the human envelope. weights are frozen. there is no selection mechanism for retaining suprahuman inferences made during training if that were even possible. thus i grant that llms. could theoretically make inferences outside the human corpus there is no way to distinguish the from errors or hallucinations during training (because by definition the are beyond human capacity) and no iterative learning from experience process after training (frozen weights). thus it seems impossible for today's models to exceed aggregate human capacity.