The Illusion of Thinking in Large Reasoning Models
Failed to add items
Add to basket failed.
Add to wishlist failed.
Remove from wishlist failed.
Adding to library failed
Follow podcast failed
Unfollow podcast failed
The Illusion of Thinking in Large Reasoning Models
-
Narrated by:
-
By:
About this listen
This episode investigates the reasoning capabilities of Large Reasoning Models (LRMs), a new generation of language models designed for complex problem-solving. The authors evaluate LRMs using controllable puzzle environments to systematically analyze how performance changes with problem complexity, unlike traditional benchmarks that often suffer from data contamination. Key findings reveal three performance regimes: standard LLMs surprisingly excel at low complexity, LRMs gain an advantage at medium complexity, and both models experience complete collapse at high complexity, often exhibiting a counter-intuitive decline in reasoning effort despite having a sufficient token budget. The analysis also examines the internal reasoning traces, uncovering patterns like "overthinking" on simpler tasks and highlighting limitations in LRMs' ability to follow explicit algorithms or maintain consistent reasoning across different puzzle types.
Send us a text
Support the show
Podcast:
https://kabir.buzzsprout.com
YouTube:
https://www.youtube.com/@kabirtechdives
Please subscribe and share.