• frezik@midwest.social
    link
    fedilink
    English
    arrow-up
    5
    arrow-down
    1
    ·
    1 year ago

    Real big if. There’s reason to believe that current models aren’t going to get much better. They’ve eaten all the training data they possibly can. Improving with further training takes exponentially more power to get a small improvement. We’re talking about new nuclear reactors because that’s what they need to get anywhere, but it’s still not going to improve by much.

    The field needs a new model that can get better results on less data and less training. Then we wouldn’t need those nukes. It doesn’t appear we’ll get much better any other way.