Gamers and the general consumer electronics customers have been told for the past two years that GPUs cannot be offered at a reasonable price anymore due to unprecedented AI data center demand. Gl…
…LLMs don’t “know” anything about their training. Beyond (sometimes) a stock greeting intentionally finetuned in.
I’ve brought this up dozens (hundreds) of times, for years, every time someone thinks they’ve identified a stealth model or identified some clue because they’ve gotten a model to “admit” it. It’s all just blind improvisation, it’s literally what they do.
If you want a more clear example, run Deepseek in Mikupad with the raw syntax, ask it your “meta” question, look at the logprobs for the critical decision token, then continue different answers from there, and it will confidently improvise any answer because it is roleplaying.
Or you could just consider what information the model itself has access to, and realize that kind of insight is physically impossible.
Do Chinese companies have a history of skirting IP? Sure. That’s literally history.
Did Deepseek (and other Chinese companies) harvest Claude answers for training data?
Oh yes. It’s obvious. They pulled Claude absolutely shamelessly.
If you want actual proof, see papers on it, or EQBench’s “slop family tree” graphs:
But other LLM companies around the world have no leg to stand on. They all train on each other’s output, they all train on the same legally questionable data anyway.
And I reiterate:
You can’t “investigate” an LLM by asking it for information. They just make stuff up. You can get any model to say it’s training data came from anywhere with the right initial seed, and many will ape Claude or ChatGPT unprompted because they trained on so much of it across the internet.
…LLMs don’t “know” anything about their training. Beyond (sometimes) a stock greeting intentionally finetuned in.
I’ve brought this up dozens (hundreds) of times, for years, every time someone thinks they’ve identified a stealth model or identified some clue because they’ve gotten a model to “admit” it. It’s all just blind improvisation, it’s literally what they do.
If you want a more clear example, run Deepseek in Mikupad with the raw syntax, ask it your “meta” question, look at the logprobs for the critical decision token, then continue different answers from there, and it will confidently improvise any answer because it is roleplaying.
Or you could just consider what information the model itself has access to, and realize that kind of insight is physically impossible.
Do Chinese companies have a history of skirting IP? Sure. That’s literally history.
Did Deepseek (and other Chinese companies) harvest Claude answers for training data?
Oh yes. It’s obvious. They pulled Claude absolutely shamelessly.
If you want actual proof, see papers on it, or EQBench’s “slop family tree” graphs:
https://eqbench.com/creative_writing_longform.html
But other LLM companies around the world have no leg to stand on. They all train on each other’s output, they all train on the same legally questionable data anyway.
And I reiterate:
You can’t “investigate” an LLM by asking it for information. They just make stuff up. You can get any model to say it’s training data came from anywhere with the right initial seed, and many will ape Claude or ChatGPT unprompted because they trained on so much of it across the internet.
Yeah I don’t know what the other guy is talking about, if he thinks he can “extract the data” from a LLM. Bonkers.
Wow, you think you know so much more than you do! It’s impressive!
…Okay. I don’t appreciate the mocking tone, but if you are skeptical, share your prompt, and I will try it with Deepseek locally.
You presented so many false ideas as facts that I have determined it is not worth my time to engage. Sorry!
I am not trying to overwhelm, but you challenged us to reproduce your assertion, and I can’t really do that without the prompt.
The whole point was to train it on an obscure thing, implement it, and recall to prove the retention. It’s not any specific prompt
Let’s see, who should I believe? The guy actually referencing research and data or the guy going “believe me bro, I did the research”.
Hmmmm…
I’m not asking you to believe anyone. Set up the experiment yourself if you’re capable (doubt). I owe you nothing.