Chain-of-Thought Reasoning in the Wild Is Not Always Faithful

(arxiv.org)

23 points by florianherrengt1 hour ago

6 comments

florianherrengt1 hour ago
This paper puts words to something I’ve noticed repeatedly with LLMs, particularly Qwen3.6. When I read its reasoning, it appears to recognise the mistake and then carry on as if it hadn’t noticed it at all.<p>> models often determine their answers based on implicit biases tied to question templates, then construct reasoning chains to justify their predetermined conclusions > its reasoning was correct right until the final step (Yes/No answer)
- Georgelemental1 hour ago
  Natural intelligences do this too
  - tyg1349 minutes ago
    Must we always see this restated every time? It's getting a bit stale always seeing these kinds of comments on articles about LLM.
    - cyanydeez19 minutes ago
      You think, "this problem" is qn LLM problem?
      - nyeah9 minutes ago
        Yes, in fact there's an article about it.
  - phailhaus32 minutes ago
    No they don't, human intelligence has the ability to form an internal model of itself, which allows it to "notice" its own mistakes and change.
    - cyanydeez17 minutes ago
      Many who watched the last decade knows just because its possible to noticed mistakes and change, its clearly not a reliable process.
  - freejazz51 minutes ago
    Yeah and it's not great then either
- paimapi39 minutes ago
  [dead]
ForHackernews32 minutes ago
I thought this was already widely known?<p>From March last year: <a href="https://transformer-circuits.pub/2025/attribution-graphs/biology.html#dives-cot" rel="nofollow">https://transformer-circuits.pub/2025/attribution-graphs/bio...</a><p>There's no reason to believe the model's self-reported "thinking" bears any relation to the mechanics by which it arrived at some output.
kibwen1 hour ago
"Study: Communing With The Gods of Mount Olympus Via the Oracle at Delphi Is Not Always Faithful"
nemomarx1 hour ago
[dead]
josefritzishere1 hour ago
[flagged]