Rather than showing the capability for generalized logical inference, these chain-of-thought models are “a sophisticated form of structured pattern matching” that “degrades significantly” when pushed even slightly outside of its training distribution, the researchers write. Further, the ability of these models to generate “fluent nonsense” creates “a false aura of dependability” that does not stand up to a careful audit.
As such, the researchers warn heavily against “equating [chain-of-thought]-style output with human thinking” especially in “high-stakes domains like medicine, finance, or legal analysis.” Current tests and benchmarks should prioritize tasks that fall outside of any training set to probe for these kinds of errors, while future models will need to move beyond “surface-level pattern recognition to exhibit deeper inferential competence,” they write. __ Fluent Nonsense
Halandzsa is a secret language invented by Karinthy Frigyes in Budapest in the 1920s. Two people are chatting when a third, uninvited fellow comes near to take part in the conversation. The original participants continue as before, but introduce invented words and sentences, making the third person wonder what they are talking about and in what language. After a while, he gives up and leaves. They were speaking "Halandzsa". AI learned the art and speaks it fluently.
No comments:
Post a Comment