Opus 4.8 and gpt constantly hallucinate stuff as well. If you haven’t encountered or caught it that’s something different. Of course these days it’s mostly confidently asserting a wrong thing.
They said that they hadn't "suffered" from hallucinations in months due to effort they put into harness engineering. It's not quite the same thing as saying hallucinations are never happening for them.
I find hallucinations happen for us with those models, but we've worked on baking in guardrails and fact checking against sources of truth, so that it's less of a problem.
We're engineers. We just try to engineer out the failure modes.
I primarily use Opus for a Lisp-like DSL codebase (non public, closed source) and it genuinely has _never_ hallucinated. All it pulls from is BNF, language spec + examples. So I have no idea how people are getting it to hallucinate on _popular_ languages.
I think you forget that they really are stochastic (I'd wager it has hallucinated things to you that wasn't important so you missed it, or you've just been very lucky), and the people you're arguing with are forgetting that there is significant difference in when and how often even frontier LLMs hallucinate.
I catch claude every now and then, but isn't the measured hallucination rates down in low single digit percent for claude now?
And this is fine. Developing new software with a really smart intern is the same, you, as an expert, need to bring your experience/expertise on the table to have everything right. Because experience needs time.
Those models simply don't hallucinate if you use them properly in any form. The only way they _might_ hallucinate is if you use the web based chat interface and give them zero context.
