cross-posted from: https://feddit.org/post/36071801
This author trieds hard to do a pragmatic view. He does not mention ethical or ecological concerns (which should matter far more and subject to legal regulation IMO).
Instead, he focuses on the quality topic. His thinking is that companies are doing now a kind of quality arbitrage: Companies can make low-quality code at far cheaper costs, and sellthe result still at human-made prices. Which will only work for so long, he thinks: Customers will demand lower prices. (Easy to imagine a quality downward spiral.)
Which invokes the question: What has happened that people accept lower and lower quality? Is new software nowadays still really useful?
“The reason LLMs are successful in writing code is because we’ve made a feedback loop that feeds the errors back to the LLM and loops until most errors are solved or hidden. Remember that LLMs can and do cheat too.”
I’ve heard, and somewhat seen, that AI makes more code than necessary when fed large code bases. Then the context window isn’t big enough to encompass the whole thing anymore, so it starts to create the issues you’ll make it solve later. This feedback loop manifests in longer debug times, prompt adjustments, and greater lack of understanding overall.
Context collapse and hallucination in the face of anything novel has been my experience 90% of the time. I even tried context compression by trying to do the research, test spec, plan, code tests, product spec, plan, code product method, but then the issue was the plan satisfied the AI too quick (“actually we already test all of the code, you can see that by reading the “we test all of the code” spec”).
I’ve had let them help me with shell scripts sometimes. Not anymore; they use the more popular tool, even if it increases complexity x100.
I really like when LLMs invent new Bash Syntax.
Ah right. Or non-existing parameters.
Oh yeah, that’s also always nice. I got unholy amounts of non existing/working Parameter Expansions from Various LLMs and it also kept hallucinating ways to “pipe” parameter expansions into each other.
In case of tests, I’ve seen not simply more code than necessary, but hundreds of line of generated test cases focusing on implementation details based on a badly worded issue description, without anyone bothering to check if the generated integration works at all. These test cases are an immediate dead weight for maintainers that make refactoring and fixing issues more difficult.
Don’t need an LLM for that. I found human-made test cases successfully testing for wrong assumptions.