> Which on the one hand is a little disappointing to me in terms of the human performance but is kind of good news for LLMs

Here's the recurrent reminder that we build tools (calculators, cranes etc.) to outperform the strong, not the weak.