Anthropic’s Mythos has flagged bugs faster than Microsoft can fix them. Documents reviewed by ProPublica reveal the tech giant's “mad dash” behind the scenes to patch holes before hackers can find and exploit them.
this type of thing really shows llm’s to me. its great at creating this busy work but not so much with fixing it. so it can find them and we know microsoft is all in on ai coding, but it does not seem to help fixing them quickly. they are spending so much money to muck things up more and make more expense.
That’s not what they’re saying. They’re saying llms are getting so good at revealing vulnerabilities that they have to put focus and effort on finding and fixing them before external malicious actors do.
Yeah it’s faster to find a bug than to refactor a whole codebase to fix it, they’re still using llms to speed up fixing the bugs. If you believe them, anyway.
I do. Their software stacks were so shit and bloated and had many owners with different visions and understanding of what code does what and why and they rely on being closed source to survive penetration, I do believe llms are finding vulnerabilities left and right, and focused refactoring to address legacy human slop can take time, even for a super competent llm that does it correctly in one shot.
its great at creating this busy work but not so much with fixing it
Actually, if you know what needs fixing, LLMs can accelerate the fixing process quite a bit too.
The problem is: LLMs don’t know what they don’t know, so they’ll declare “good enough” and it’s up to the LLM operator to call BS and give them additional context to ensure they keep going until ALL the applicable concerns are addressed.
this type of thing really shows llm’s to me. its great at creating this busy work but not so much with fixing it. so it can find them and we know microsoft is all in on ai coding, but it does not seem to help fixing them quickly. they are spending so much money to muck things up more and make more expense.
That’s not what they’re saying. They’re saying llms are getting so good at revealing vulnerabilities that they have to put focus and effort on finding and fixing them before external malicious actors do.
Yeah it’s faster to find a bug than to refactor a whole codebase to fix it, they’re still using llms to speed up fixing the bugs. If you believe them, anyway.
I do. Their software stacks were so shit and bloated and had many owners with different visions and understanding of what code does what and why and they rely on being closed source to survive penetration, I do believe llms are finding vulnerabilities left and right, and focused refactoring to address legacy human slop can take time, even for a super competent llm that does it correctly in one shot.
M$ always has.
But even open source relied a great deal on the volume and complexity of their code base to survive penetration… until now.
Actually, if you know what needs fixing, LLMs can accelerate the fixing process quite a bit too.
The problem is: LLMs don’t know what they don’t know, so they’ll declare “good enough” and it’s up to the LLM operator to call BS and give them additional context to ensure they keep going until ALL the applicable concerns are addressed.