• XLE@piefed.social
    link
    fedilink
    English
    arrow-up
    2
    ·
    5 hours ago

    There’s a story about a guy who asked his LLM to remind him to do something in the morning, and it ended up burning quite a lot of money checking to see if daylight had broken once every 30 minutes with an unnecessary API call. Such is the supposed helpful assistant.

    • wonderingwanderer@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      3
      ·
      3 hours ago

      That story was about a guy paying to use an API for a flagship model (like 200 billion parameters).

      I think these people are talking about self-hosting a local model (probably like 12-32 billion parameters depending on your hardware), which means no API, no payments, and more personal control over settings and configuration.

      Thousands of open-source models are freely available on huggingface, and you can even make your own fine-tuned version based on an existing one using any datasets you choose.

      Still no point in using an AI agent to do what a basic alarm/reminder could do, but it allows people to innovate their own ways to integrate them into specific workflows. You can even configure them to play minecraft, just as an example