When 60-80% or so of your PR can be refactored away then it’s a crap PR and honestly never should have been one.
So their time savings in getting AI help to write their code means that you spend more of your presumably more expensive time doing reviews and educating them about slop removal instead of some higher-value activity. Sounds like you’re veering into negative ROI for the AI use if that’s true.
Luckily, I don’t review PRs very often, I have people to do that. But the general principle is that the content of a PR is the responsibility of the submitter, regardless of its source. Wrong algorithm? Their fault. No-good UTs? Their fault. Inappropriate or unsafe dependencies? Their fault. Slop? Their fault.
Luckily, with the work we do, there’s often nothing someone could train an LLM on, so we don’t see all that many PRs with AI-generated content, unless we’re using some well-known commodity library or framework in a common way. And that was always the easy stuff anyway.
A slop PR should be the submitter’s responsibility, but if they’re trying to push out unreviewed slop, you know they’re just going to ask an LLM to refactor it instead of looking at it themselves. The best response is just to reject it outright and have someone else work on a new solution.
I have had a fair amount of success getting AI to do those refactorings, reducing 2000 lines of code to 400, and generating 3000 lines of documentation (including flowcharts) explaining how the 400 lines work, adding 1200 lines of automated testing to prevent regressions, etc. etc.
I started by designing all those files around last September-October timeframe. By November I was realizing: the AI knows better how to make those files than I do, so instead of writing SKILL.md or whatever, I describe to the agent what I want it to configure and explain to me how it’s going to do it. It makes SKILL.md for me, I review it - correct the (usually few and far between) things that are misquoting my intent - and tell it to install it. In areas like softtware development that’s been working pretty well, and getting more and more invisibly automated by the tools since then.
Claude Opus 4.8 dropped a few weeks back “best graphical understanding ever” - well, I’m trying to use it to do some Blender work, it sucks - it’s worse at it than the code writers were in January 2025.
.
So their time savings in getting AI help to write their code means that you spend more of your presumably more expensive time doing reviews and educating them about slop removal instead of some higher-value activity. Sounds like you’re veering into negative ROI for the AI use if that’s true.
Luckily, I don’t review PRs very often, I have people to do that. But the general principle is that the content of a PR is the responsibility of the submitter, regardless of its source. Wrong algorithm? Their fault. No-good UTs? Their fault. Inappropriate or unsafe dependencies? Their fault. Slop? Their fault.
Luckily, with the work we do, there’s often nothing someone could train an LLM on, so we don’t see all that many PRs with AI-generated content, unless we’re using some well-known commodity library or framework in a common way. And that was always the easy stuff anyway.
A slop PR should be the submitter’s responsibility, but if they’re trying to push out unreviewed slop, you know they’re just going to ask an LLM to refactor it instead of looking at it themselves. The best response is just to reject it outright and have someone else work on a new solution.
I have had a fair amount of success getting AI to do those refactorings, reducing 2000 lines of code to 400, and generating 3000 lines of documentation (including flowcharts) explaining how the 400 lines work, adding 1200 lines of automated testing to prevent regressions, etc. etc.
deleted by creator
I started by designing all those files around last September-October timeframe. By November I was realizing: the AI knows better how to make those files than I do, so instead of writing SKILL.md or whatever, I describe to the agent what I want it to configure and explain to me how it’s going to do it. It makes SKILL.md for me, I review it - correct the (usually few and far between) things that are misquoting my intent - and tell it to install it. In areas like softtware development that’s been working pretty well, and getting more and more invisibly automated by the tools since then.
Claude Opus 4.8 dropped a few weeks back “best graphical understanding ever” - well, I’m trying to use it to do some Blender work, it sucks - it’s worse at it than the code writers were in January 2025.