Does AI 'Think'? Really? — Diving into the Reasoning Mechanisms of LLMs
These days, the phrase "AI reasons" is used naturally in the news. But what exactly is the computational basis for saying this? I've summarized a few key changes.
1. Traditional LLMs: massive statistical models trained to predict the probability of the next word
2. Reasoning LLMs (e.g., o1): before responding, they internally generate 'thinking' text and repeatedly reflect and revise on top of it
Looking at the intermediate steps produced during this thinking process, you can see systematic reasoning that actually resembles how humans solve problems. Of course, some scholars argue that this is merely an imitation of vast patterns from training data, not 'real thinking.' The key point is that what has changed is not 'consciousness' but the 'computational structure,' and that is what leads to superior performance.
In conclusion, technically speaking, AI has now entered a stage where it possesses an internal thinking process for deeper problem-solving. However, using the word 'thinks' in an anthropomorphic way projects our human frame onto it, so we need to clearly recognize the actual difference.
It's clearly amazing that AI has reached the point where it not only writes and codes, but also formulates and revises its own strategies. I'm curious to see what comes next.
10 answers
Agreed
This is right lol
Got a source? Where did you see the 'thinking process' sample that this post claims?
I think so too.
Oh, I didn't know this.
Even when I used ChatGPT for coding before, it was much more accurate when it laid out a plan and worked through the problem step by step, rather than just giving the answer right away. Of course, I'm not sure if that's real thinking, but at least the performance was noticeable.
Well, there have been points that that 'thinking' text is actually just generating 'sentences that look like thinking' from the training data... so it's a stretch to see it as real reasoning, isn't it? I think it's too early to judge based solely on benchmark results.
There's talk that when fine-tuning reasoning models, the quality of CoT data is crucial. Without a filtering process, you just get more verbose answers. If you're interested, go check out the paper.
So ultimately, results like that come out purely from computational structure without consciousness, right? That means the term 'thinking' is just a metaphor, and I feel like we shouldn't read too much meaning into it.
I actually find the generation of that internal thought process truly amazing. Even if it's just probabilistic pattern generation, the mere addition of a self-correcting mechanism for that process makes it a whole different dimension from before.