• 0 Posts
  • 56 Comments
Joined 3 years ago
cake
Cake day: June 30th, 2023

help-circle






  • Absolutely not true. Disclaimer, I do work for NVIDIA as a forward deployed AI Engineer/Solutions Architect—meaning I don’t build AI software internally for NVIDIA but I embed with their customers’ engineering teams to help them build their AI software and deploy and run their models on NVIDIA hardware and software. edit: any opinions stated are solely my own, N has a PR office to state any official company opinions.

    To state this as simply as possible: I wouldn’t have a job if our customers weren’t seeing tremendous benefit from AI technology. The companies I work with typically are very sensitive to CapX and OpX costs of AI—they self-serve in private clouds. If it doesn’t help them make money (revenue growth) or save money (efficiency), then it’s gone—and so am I. I’ve seen it happen; entire engineering teams laid off because a technology just couldn’t be implemented in a cost-effective way.

    LLMs are a small subset of AI and Accelerated-Compute workflows in general.


  • We’re looking at this from opposite sides of the same coin.

    The NN graph is written at a high-level in Python using frameworks (PyTorch, Tensorflow—man I really don’t miss TF after jumping to Torch :) ).

    But the calculations don’t execute on the Python kernel—sure you could write it to do so but it would be sloooow. The actual network of calculations happen within the framework internals; C++. Then depending on the hardware you want to run it on, you go down to BLAS or CUDA, etc. all of which are written in low-level languages like Fortran or C.

    Numpy fits into places all throughout this stack and its performant pieces are mostly implemented in C.

    Any way you slice it: the post I was responding to is to argue that AI IS CODE. No two ways about that. It’s also the weights and biases and activations of the models that have been trained.










  • The most annoying thing about a lot of these is that tutorials are “minimal viable setup” sorta things. Like “now you have it setup, make sure you tune it for production”

    Dude I’m already in pain from trying to serve these models and you just have to go rub salt into my eyes. “Simplify your stack with <Tech>” they said. “Share your resources effectively and easily with <Tech>” they said. “Here’s your fuckin’ ‘Hello, World’ now GRTFM and buzz off” they said.

    Working close to the metal do be like that.




  • Nah, I work with real big data all the time—I’m a ML engineer/DataSci depending on the day.

    It’s not crashing because I put a trivial couple hundred rows of data into a spreadsheet.

    It crashes because there’s some conflict between its Java core and the Linux kernel I’m running it on. It’s been like this across many versions; I keep everything updated, etc. Tried many versions of Java, and OpenJDK because FuckOracle. I’m no Java developer though, so Inwouldnt be able to contribute unless they want to refactor the entire core to Rust in which case I’d love to help.

    I send bug reports and it’s always just crickets—either they don’t know and don’t communicate that they don’t know, or don’t care, or more likely are just too busy with their realjobs to go on the hunt for a solution to a corner-case bug/crash scenario like mine probably is.

    I use office programs so infrequently that I just deal with it. But if I was like my directors and managers who live and die by office productivity apps then I’d have to abandon LibreOffice and go to the closed-source solution.