• brucethemoose@lemmy.world
    link
    fedilink
    English
    arrow-up
    15
    ·
    edit-2
    5 hours ago

    …LLMs don’t “know” anything about their training. Beyond (sometimes) a stock greeting intentionally finetuned in.

    I’ve brought this up dozens (hundreds) of times, for years, every time someone thinks they’ve identified a stealth model or identified some clue because they’ve gotten a model to “admit” it. It’s all just blind improvisation, it’s literally what they do.

    If you want a more clear example, run Deepseek in Mikupad with the raw syntax, ask it your “meta” question, look at the logprobs for the critical decision token, then continue different answers from there, and it will confidently improvise any answer because it is roleplaying.

    Or you could just consider what information the model itself has access to, and realize that kind of insight is physically impossible.


    Do Chinese companies have a history of skirting IP? Sure. That’s literally history.

    Did Deepseek (and other Chinese companies) harvest Claude answers for training data?

    Oh yes. It’s obvious. They pulled Claude absolutely shamelessly.

    If you want actual proof, see papers on it, or EQBench’s “slop family tree” graphs:

    https://eqbench.com/creative_writing_longform.html

    But other LLM companies around the world have no leg to stand on. They all train on each other’s output, they all train on the same legally questionable data anyway.


    And I reiterate:

    You can’t “investigate” an LLM by asking it for information. They just make stuff up. You can get any model to say it’s training data came from anywhere with the right initial seed, and many will ape Claude or ChatGPT unprompted because they trained on so much of it across the internet.

    • archwaycurl@piefed.social
      link
      fedilink
      English
      arrow-up
      4
      ·
      3 hours ago

      Yeah I don’t know what the other guy is talking about, if he thinks he can “extract the data” from a LLM. Bonkers.

      • brucethemoose@lemmy.world
        link
        fedilink
        English
        arrow-up
        7
        ·
        3 hours ago

        …Okay. I don’t appreciate the mocking tone, but if you are skeptical, share your prompt, and I will try it with Deepseek locally.

        • LooseLemons@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          arrow-down
          9
          ·
          2 hours ago

          You presented so many false ideas as facts that I have determined it is not worth my time to engage. Sorry!

          • brucethemoose@lemmy.world
            link
            fedilink
            English
            arrow-up
            5
            ·
            2 hours ago

            I am not trying to overwhelm, but you challenged us to reproduce your assertion, and I can’t really do that without the prompt.

            • LooseLemons@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              arrow-down
              5
              ·
              2 hours ago

              The whole point was to train it on an obscure thing, implement it, and recall to prove the retention. It’s not any specific prompt

      • mushroommunk@lemmy.today
        link
        fedilink
        English
        arrow-up
        9
        ·
        4 hours ago

        Let’s see, who should I believe? The guy actually referencing research and data or the guy going “believe me bro, I did the research”.

        Hmmmm…

        • LooseLemons@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          arrow-down
          12
          ·
          4 hours ago

          I’m not asking you to believe anyone. Set up the experiment yourself if you’re capable (doubt). I owe you nothing.