Yes, LLMs fundamentally operate as a lossy compression scheme for their training data. There's been countless examples of them reproducing their training data with very high accuracy
People claim that the data isn't stored, but clearly a representation of it is encoded and reproducible. I saw chatgpt word for word plagiarise a stack overflow comment just two days ago
Well yeah, if you're making the claim that it stores a representation of the data in some form.
Does your calculator app store a representation of the answer to 1+2/2*1.1 and all other combinations of inputs or does it determine the answer from a set of rules?
It's a different case when the input contains more information than the output.
If you put "1+2/2 x 1.1" into a calculator and it spit out a verbatim copy of a New York Times article, does it necessarily contain a representation, or does it just contain some really extensive rules? I'd argue those rules necessarily are a representation of that information, given that it contains far more information than provided by the input.
Ah but can it reply snarkily and close your ticket as a duplicate that is NOT A DUPLICATE? If not it will never recreate the real stack overflow experience.
People claim that the data isn't stored, but clearly a representation of it is encoded and reproducible. I saw chatgpt word for word plagiarise a stack overflow comment just two days ago