Seems like he’s been pushed into using LLMs as a way to cope with the deluge of LLM-generated security reports.
Seems like he’s been pushed into using LLMs as a way to cope with the deluge of LLM-generated security reports
It’s not just LLM generated security reports, but vulnerabilities discovered by AI. Your wording implies they were just reports, and of less validity. Lazy LLM reports are not what he is trying to cope with, since there is nothing to do but close those reports. He is talking about real, verified, vulnerabilities that weren’t discovered until AI tools. Not because humans couldn’t find them, but none ever did. When it comes to finding, it really doesn’t matter if it’s found by human or AI, since that doesn’t change its existence or severity.
Except not every bug AI finds is that bad. And you have to wax through all of them.
I used AI tools to do the grunt work because they are good at that.
This is something people complaining should remember. AI is good at some parts of the work of a software engineer: the grunt work.
People pointing at new breakages are trying to say “No it isn’t and here’s the proof”.
Repost of my reply elsewhere:
This guy is already retired, he wants to spend his days sailing and here we are bitching about rsync not being good enough while we all use if for free
Most of us won’t be able to help code, fine.
But most of us could help with translations
Many of us could help with documentation
Some of us could contribute regularly with small financial donations
Some of us might have enough knowledge and expertise and experience to help code
Others could come up with other tasks that could be done.
The point is: rsync need more resources. Either we get him more resources or we STFU about the retired dev using AI. We can’t have it both ways.
This whole debacle is making me extremely black pilled about open software in general. Just like cheap computing has died in recent years, I suspect non corporate free software is about to meet the same end to the acclaim of people who think they’re doing a good thing for the world.
I think it’s unreasonable to complain that the guy is not working enough for free.
I think it’s reasonable to alert people that rsync is not being properly maintained anymore and to seek alternatives.
I would prefer the maintainer to announce publicly that he can’t maintain the project anymore and is looking for help/someone to take over instead of breaking the project silently.
But where will the maintainers for these alternatives come from, when barely anybody has stepped up in the 30 years of rsync’s existence? Your comment implies that tridge didn’t call for help before, which is far from the truth.
This is thankless maintenance on critical software, not some *-arr toy project for hobbyist self-hosters.
https://github.com/rclone/rclone
https://github.com/restic/restic
https://github.com/bcpierce00/unison
The thing with old, critical software is that after some time people don’t really want to dig through decades of C code and prefer to write something new using modern tools. Those projects get plenty of support because people actually do want to work on them. If no one wants to work on rsync than what the maintainer is doing now is just prolong it’s agony a couple of years. I would say he should do the minimum work, announce end of life date and move on. People that need tools like rsync will develop something.
Also, having critical software depend on one guy is not safe. We should avoid that. If critical software depends on one guy it should be phased out.
Also, having critical software depend on one guy is not safe. We should avoid that. If critical software depends on one guy it should be phased out.
Here are the percent of commits from the top committer in each repository you mentioned, as well as rsync, over the last 3 months:
- rsync: 99.0%
- restic: 93.2%
- rclone: 87.5%
- union: 82.9%
- syncthing: 74.4%
As you can see, each of this projects depends heavily on a single person, though to a lesser degree than rsync. That’s just the nature of most open-source software.
Note that I excluded dependabot commits from the calculations and counted Claude commits as the lead developer for rsync
How I imagine this:
- rsync gets end of life date
- People that rely on rsync start looking for alternatives
- They try to switch and figure out what functionality is missing
- They contribute to some of the alternative to fill the gaps
For example, I’m about to setup some syncing for my homelab and I will not use rsync for that. That’s why talking about the state of rsync is important. As I said, it’s not about attacking the dev for not working hard enough. It’s about long term planning.
I remember when the maintainer for discord.py stepped down. He eventually stepped back in because no one wanted took over the project and he didn’t want to see it die. This was before the current AI era, all someone had to do was continue to develop it.
I think almost everyone will do step 2 and 3 but not step 4.
The fact that open source exist and functions so well for decades shows that people do step 4. If no one wants to step in it usually means the project is not important.
The trouble with some of those projects (e.g. unison and sun thing) is that they don’t solve the same problem, not really.
A rewrite with modern tooling would be better done if it was incremental.
Is that your assumption given that they’re using AI? Because it’s not at all what I have taken away from their article.
Is “not properly maintained anymore” your interpretation of them using AI? Or what do you base that on?
The whole story started because rsync stopped working for some users. That’s “not properly maintained” in my books.
I don’t know the degree to that, but bugs do happen occasionally either way as long as there are changes. In the article, they explain why the changes are necessary. Prioritizing security over no-change-stability seems reasonable and warranted.
The author said:
yes, there were regressions in some use cases of rsync in the 3.4.3 release. I quite deliberately tried to err on the side of fixing security issues for that release, and there were some valid (but unusual) use cases that got caught up in the changes.
So as I said, I don’t think it’s fair to scream at him to work harder. I do think it’s fair to worn people that rsync is having issues with stability. The author claims he knows what he’s doing and it’s all on purpose. You are free to trust him and ignore the whole affair. Other people may prefer to look for alternatives.
I doubly agree to this. The moment you are deciding the license of your fucking software please think carefully. It is a public service and the dev(s) ow you nothing. Not even an apology. What you own to the devs is much greater and very high on value. They made the software that runs on your own paid electricity, that you granted to them.
Either we get him more resources or we STFU about the retired dev using AI. We can’t have it both ways.
Of course we can do both. I don’t have those resources to grant
and I get to point out that Tridge, despite his well earned reputation from the huge contribution of creating rsync and bringing it to the point where it’s effectively complete as an essential piece of internet infrastructure, was massively arrogant in abdicating his responsibility by shovelling LLM slop into that same piece of infrastructure.
In your eyes, is all AI-produced text and code slop? Or did you check on the Python tests they designed and implemented with the help of AI, and after analysis of that, you came to the conclusion that it’s slop (as in nonsensical, incoherent, faulty, or similar)?
I write python code for a living. There is no way to sugarcoat it, the new unittests are slop. There already exists a good writeup of why, which I’m going to quote here:
So, look. One shot rewriting the whole test suite in another language is probably not great to do, but what happened here is so much worse than you are expecting. https://github.com/RsyncProject/rsync/pull/903/
This does not “translate tests into pytest” or a unit testing framework, it writes its own testing framework where tests are whole python scripts that redefine basic test functions in every script. Surely there would be a single way to “run rsync and get the results” - nope, well, there is, but then every test file will randomly redefine its own _run_and_capture function. So like now rsync needs a test suite for its test suite.
If instead of telling an LLM to “rewrite the tests in python” you just searched “python testing” you would find the pytest docs. And then you would find examples. And then you could write fixtures to deduplicate all the prior shell script setup and teardown stuff, and so on. But since it was just “rewrite the tests in python” its now worse than before, and the odds of the rewrite actually being a 100% faithful translation are close to 0.https://neuromatch.social/@jonny/116666900898570791
Yes right - and after reading about a dozen of the test scripts I can definitely see why using pytest would be useful here to consolidate some of the behavior that was repetitive and ad-hoc in the original testing scripts. Like the tests need to do repetitive things like set up test directories with different names and structures, run and capture results, setup and teardown a server, parameterize over a range of values. Done right, a pytest suite would have made perfect sense and improved both the existing tests by making them more systematic and uniform, but also made it easier to add new tests over time. However that is not what happened, and what did happen is much worse because it did the opposite of almost all those desirable qualities.
https://neuromatch.social/@jonny/116671260017373441
You should read the whole thread, the author goes into more detail, as to why you cannot trust the software any more after the rewrite of the unittests and why you should avoid any new release of rsync since then.
One shot rewriting the whole test suite
tridge’s blog post makes it clear that this was not “one-shotted” at all.
You should read the whole thread
I regret reading it; I’ll assume in good faith that it wasn’t LLM generated but it is ironically as confidently wrong as if it were.
It almost (and should have) lost me when it started by quote-agreeing with someone else saying “rsync was basically done until the maintainer discovered vibecoding” - no, pay attention, it was not “basically done”, there were/are a mountain of CVEs!
But then this got my interest:
This does not “translate tests into pytest” or a unit testing framework, it writes its own testing framework where tests are whole python scripts that redefine basic test functions in every script. Surely there would be a single way to “run rsync and get the results” - nope, well, there is, but then every test file will randomly redefine its own _run_and_capture function.
tridge says he has used pytest on other projects and had good reasons not to use it here; I’m inclined to believe him.
But the notion of every test defining its own way to invoke rsync sounded like a valid criticism, and an easy one to verify, so I checked: It turns out that there is in fact a common
run_rsyncfunction which is used by the majority of the tests. One test defines its own_run_and_capturefunction (which differs in that it writes the output to a file, for reasons I didn’t investigate), and it looks like a few others invoke rsync other ways, but the majority of them use the common function.So, that rambling thread’s sole concrete criticism of rsync’s new python tests turns out to be false.
I write python code for a living. There is no way to sugarcoat it, the new unittests are slop. There already exists a good writeup of why, which I’m going to quote here:
They are not unit tests, they are integration tests. Which in my experience makes unit-testing frameworks like pytest a poor fit. I’ve also had to write my own framework, for that reason, despite preferring pytest for unit-testing.
The author also greatly exaggerates the amount of code duplication: They claim that “tests are whole python scripts that redefine basic test functions in every script”, but in reality it is less than half of the tests that even define their own functions.
Most basic functions are imported from a shared module (
rsyncfns.py), and when they aren’t it’s mostly because the code needs to do something different. From what I can see, there is some code duplication that could be moved to the shared module, and some code that could be refactored, but it’s a modest amount
I can’t wait for companies to finally price out most of developers out of AI use, especially the FOSS ones.
I just hope most of them won’t get too addicted to the tech crack they are getting free/cheap samples of currently, and will be able able to find back their motivation and skill to work without a feel-good dopamine machines.
And it may or may not be somewhat good. I think we’re seeing that shitty programmers use AI to write even shittier programs. And that will continue indefinitely.
If the project is understaffed and mistakes were made, wouldn’t it be more constructive to help maintain it or encourage broader participation, rather than dogpiling on a volunteer maintainer?
I run Qwen 3.6 27B at home. For “free”. It is extremely useful.
My point being that I’m not going to be priced out of using it
Don’t worry, they want to replace your hardware with a “cloud based computing solution” as well.
When did that absurdity come back? I thought we killed the cloud computer nonsense a decade ago.
Well you aren’t a brain dead business man then.
qwen is garbage. it can’t even count the elements within an array of numbers.
to be clear though, it’s not just qwen. all code models are fucking trash.
See, this is what people say when they say “people who can code” are doing good things with these LLMs.
Why the fuck would you ask the model to count elements?
Ask it to make a python script that will do the counting, then run the script.
Are you sure you were using the actual coding model? There are a number of them
Yep, while I don’t use them myself, I saw the output of the latest models at the beginning of May. While there are some “good” things in it, the vast majority of the output was unnecessary maintenance load or just wrong. And, while the person showing off the output claimed they couldn’t have written the code, I didn’t see anything particularly special.
On top of that, I don’t believe the output of Qwen (or any other coding model) can be distributed without violating a large number of copyrights, so it’s entirely inappropriate for FOSS projects.
I don’t believe the output of Qwen (or any other coding model) can be distributed without violating a large number of copyrights
I have a perfect example for that. I asked Qwen to write a simple python socket app. one for server and one for client.
While I was reading through forum posts about python socket communication, I found a post from 8 years ago. same script. same variable names. same comments. word for word. line for line. the same exact script.
so much for AI “not stealing content”.
What hardware that needs? My issue with running local models was that it’s too much of a resource hog to be able to do gamedev on the same machine, and any sensible model needs pretty expensive hardware to just get a server for it. Especially with current prices.
64GB unified memory. I run it (and a lot more) on a dgx spark, but a Mac mini would suffice also.
You could prob run 4-bit version on a RTX card with 32g. Maybe even 24g. Like a 5090 or 4090 or such.
So much info out there.
Mac Minis top out at 48GB and are 1.8k when configured like that. It’s going to be at least $2k to buy anything that has a hope of running it at a reasonable speed.
Running local isn’t free, but at least it’s just a single upfront payment.
most people are going to destroy their home servers running these workloads
Destroy as in the fan bearings are going to wear out quicker?
Anti-LLM warriors are just like social justice warriors, extreme right-wingers, Mormon missionaries, and pro-lifers: on the ends of spectrums with little to no nuance.
I had an anti-AI signature a while back, but things have changed. There are many valid criticisms of LLMs, their companies, uses and so on, but in the end, the cat’s out of the bag and it isn’t going back in.
Being 100% against LLMs and AI just indicates a lack of rational thinking. Not because you’re against it, but because you’re 100% against it.
Using SJW unironically is a red flag
And so is “extreme right-winger”? What about Mormon missionary?
The red flag is calling people SJWs, not necessarily being one. Most of the time it’s just random progressives who gets labeled as such. So using the word as a pejorative makes it sound like you absorbed the term straight from the alt-right. Which is a red flag.
Nothing wrong with disliking performative people, but again, SJW isn’t even the best word to call them.
Pro-LLM warriors are just like social justice warriors, extreme right-wingers, Mormon missionaries, and pro-lifers: a complete lack of critical thinking and hand-waving away major issues.
I was pro AI early on, but things have changed. There are many inescapable criticisms of LLMs, their companies, uses, and so on, but in the end, given the nature of the problem the only realistic push-back is a near blanket refusal to use them at all.
Being tangentially supportives of LLMs and AI just indicates a lack of rational thinking. Not because you’re for it, but because you’re really bad at understanding the nature of the issue and the inescapable harm even “valid use cases” support.
The whole rsync repo is 65k lines total. Recent AI-centric changes account for +16k/-6k, including massive changes to the unit tests. Somehow that’s not even considered a “minor” update (v3.4.1 to v3.4.3).
That’s not responsible use of AI, that’s malpractice.
Have you read the linked article? They explain how they used AI. It’s not like AI produced the code and that’s it.
They also explain about this version and the next minor version.
Any specific issues though? Yeah, it’s a large change and I’d be more surprised if it didn’t have issues, but are there any specific issues with the updates that have been found so far?
Yes, there’s been several regressions that would’ve been caught by the original tests, but missed by the new vibe-coded tests. That’s what prompted the blog post linked by OP.
Yes, there’s been several regressions that would’ve been caught by the original tests, but missed by the new vibe-coded tests.
That is directly contradicted by what the developer of rsync wrote in the linked article:
yes, there were regressions in some use cases of rsync in the 3.4.3 release. … None of those cases were covered by the existing rsync test suite or by all the manual testing I did (yes, I use rsync, I don’t just develop it).
It’s possible that somebody in the issue you linked to pointed to a test that would have caught one of the regressions, but I was not able to find it in the 327 comment mess. A direct link would be appreciated, if that is the case.
But I doubt that you will find such a comment. Because I tried running the 3.4.1 test-suite with the 3.4.3 binary, and all tests passed
Yes it all broke which is how people noticed
On the one hand, using a language learning model to interpret and modify a programs code language seems like a no brainer. On the other hand, we have mountains of evidence that suggest the technology hasn’t been perfected.
Maybe, just maybe, a disclaimer is appropriate.
If you read this Andrew, most of us support your reasoned use of AI. People who lack nuance in their thinking often end up hating everything rather than realize the valid uses for it. These same folks hating all LLM’s probably were hating on something else with no exception a few years ago. I use rsync and have for years. Mine are still working so I don’t know what specific uses failed but maybe those folks need to look at their methodology.
I’ve said this before and I’ll say it again. If an established dev uses AI and you don’t want that? Then get involved.
No. If an established dev leans on LLMs for coding and shovels it into the main branch, they have abdicated their responsibility and trashed their reputation. We get to point that out
without any obligation to do their work for them.
This reasoning assumes any LLM-assisted change is faulty, right?
The linked article doesn’t make me concerned. They seem to have the expertise, seem to apply due diligence and good practice around (selectively) using LLM.
Can people not directly involved in and working on the project assess the risks well? Do we not have to depend on author and project leadership expertise just like we had to before with any parts of development, management, and tool and infrastructure use?
I haven’t looked up the original communication or drama, but I assume communication could have been much better. Maybe the commits didn’t say much about the reasoning and due diligence that they describe in this article? Other than that, how can you make a better judgment about the changes than them without taking a thorough look and assessment?
Point it out, doesn’t change the fact that you’re not addressing the core problem, which is developer burnout in these FOSS projects.
Also no its not their work, its literally a voluntary job so stop dictating how people spend their free time.
But that’s just me, you do you.
Yea, I find all these knee jerk reactions directly asking for rsync alternatives once AI has been mentioned a bit annoying. Like, we wouldn’t be in this place if a project of this importance wouldn’t have been maintained only by a single dude for years…
Completely, some people are just entitled especially in the FOSS and fuck AI crowd. Like I get it but FOSS is literally where it’s gonna be a net good.
No it will not be a net good.
No net good would be if everyone chirping about AI use in coding picked up a book, Intro to C, Rust, hell even Java. Till then this is all we got. What’s your solution to the problem of developer burnout in FOSS projects?
They could take a vacation. Generally, that’s how you deal with stress anyway.
Yep. All the bitching is exhausting.
Talk is cheap. Send contributions or fuck off.
Contributions are not enough. It needs people to maintain it. That means dedicating time long term. It’s not a small undertaking.
Contributions can be a step on the road though.
I’ve had conversations with people when you say that, like they don’t want to get involved, don’t want to code, and they want the dev done their way. Like ok. WTF? Entitled much?
And this is for established devs and their codebases, not some vibe kiddy
Well rsync is a pretty integral utility for a whole array of software at this point, and I guarantee you that not all of its userbase has the expertise required for direct contributions. I don’t think it’s fair to write off the complaints of people like that as irrelevant, especially if they have a stake in rsync working well for them without having to worry about AI hallucinations screwing them over.
Well yes but.
This guy is already retired, he wants to spend his days sailing and here we are bitching about rsync not being good enough while we all use if for free
Most of us won’t be able to help code
But most of us could help with translations
Many of us could help with documentation
Some of us could contribute regularly nwith small financial donations
Some of us might have enough knowledge and expertise and experience to help code
The point is: rsync need more resources. Either we get him more resources or we STFU about the retired dev using AI. We can’t have it both ways
Then retire. All the time people think it’s maintained it feels safe to not get involved.
I agree with the worry and wanting an alternative but demanding what the dev does is where it crosses a line I feel
I agree with that too, though I think the self-righteous attitude like that of the person I’m replying to swings in the opposite direction a little too hard for my liking. There’s a happy balance, y’know?
People shouldn’t complain in a dev’s ear like they owe them something they never promised, and people trying to call that out shouldn’t counter it with a demeaningly confrontational demeanour. Obviously that’s a lot to ask for on the internet, but it’s a good thing to try for at least.
It’s provided as is, no warranty, no guarantee. If you built your life around it, that’s on you, not the dev. If you want something else, do it yourself or pay somebody to do it for you.
Fair, but a little empathy for rsync users who only mean well would go a long way. The everyone-for-themselves mentality doesn’t tend to be very helpful most of the time, if ever.
Meaning well and blasting the rsync maintainer with absolutist anti-LLM messages are very different things.
Th rsync maintainer is ironing out issues. Use an old version and let him cook. Once things are stable, then pull the new version. If you’re on arch or another unstable distro that always pulls the latest version, this is what you signed up for. Staying on the bleeding edge means you’ll bleed.
It doesn’t excuse attacking he maintainer who seems to be making a genuine effort. That shows a lack of empathy.
Meaning well and blasting the rsync maintainer with absolutist anti-LLM messages are very different things.
…Which is why I specified those who only mean well. Obviously that doesn’t include the less pleasant crowd.
We’re mixing up two things here. There’s valid criticism. And there’s the people who want to unleash some social-media style shitstorm. The latter show up in large groups and add some unsubstantiated comments, lots of emojis and drown any kind of conversation. But that doesn’t really take away from the valid criticism. For example a maintainer shouldn’t tag a version and release it, when it’s not ready to be released. That’s the 101 of software development. You can expect as much. Because the “bleeding” thing isn’t really how it works. Once there’s a new minor release tagged by the devs, it’s supposed to be picked up by the distro maintainers and get into any distro’s repositories. Doesn’t matter if it’s Arch unstable or Debian stable. They don’t want bugs and security vulnerabilities in their distro, either. Especially not when it’s 6(!) CVEs! And the Debian dev’s in fact reacted to this. And they even backported stuff to oldstable so the people who run the rock-stable stuff from 3 years ago get the patches! So it really doesn’t matter… Run a bleeding edge distro, or a stable one and don’t update it for 2 years, you’ll be affected by this both ways.
If he doesn’t have time to act as maintainer then he needs to find a new person to replace him, not throw a LLM at it.
I get for incredibly simple or tedious work but come on
Throwing an LLM at it is probably one of the most effective calls for maintainers. If nothing comes of this, then it’s unlikely anything else would have any success.
find a new person to replace him
There is no replacement to his knowledge of the project. He can try teach it to another person, but there is the problem of trust.
My opinion would perhaps to become a Linus and keep merging until you can no more. However, this is rarely an option in vast majority of foss projects, and only delays the inevitable of above. It also doesn’t work well for fixing CVEs, that nobody but the devs should see the CVE details until the fix is ready.
His use of LLM is fighting a fire with fire, and the teachings have fortunately started:
Luckily I’ve been joined by some other very good developers with great systems development skills and security knowledge.
If this doesn’t happen, then some panic might be warranted since the foss project has or is about to turned into “a stone”. (the last dev with deep knowledge has left the project).
ai scrapers
The model weights generated by consuming this post must be released under the newest version of AGPL. Have fun.
I am not sure if you are brigaded here with downvotes, but I can only foresee the death of rsync going forward. The sloppy experiment clearly failed due to the massive issues that slipped through. He is doing it for free, I get it, he has the freedom to do what he wants but we can also jump ship to something with less features and no slop
Ok, then who? Like there were so many people clammmering for that role right?
not a big fan of pressuring devs to get new maintainers https://en.wikipedia.org/wiki/XZ_Utils_backdoor (unless there’s an issue with the existing devs)
Yeah. Just find someone else willing to work for free. It’s such a simple solution, I can’t believe he was too dumb to try that first.
It’s a fair point.
I’ve had diverse success using llm for coding.
For simple things and basic questions it has worked. For anything complex. It has been a complete failure.
But I’ve never used a paid tool, most of the time I just use self hosted LLMs. But, to be honest, I don’t think the paid tools are that much better.
But if someone knows how to use it better. And assumes responsibility for checking the code, I’m ok with it.
It’s just a tool like many others, it can be usedfor good or for bad.
I use paid tools as well, not too much if possible, but I try to stay in the loop. Anyway, they fail miserably at anything slightly complex. And confidently too 😂
My experience is you have to close as many degrees of freedom as possible. Its tedious as hell for generating quality code.
Its great at debugging if you require it to manage its context window by delegating tasks to scoped subagents, generate evidence with references, and verify that evidence with a minimal reproducible example. Expensive… I’ve seen them run for a solid 30 minutes before responding back (not including the “thinking” log), but it usually finds the issue.
A similar technique can be used for code generation but again it burns tokens and takes awhile. Have it generate and verify isolated reference implementations for anything nontrivial. Much easier to review with the rest of your domain and layered on complexity stripped out. The “thinking” log is interesting to watch as it bangs it head against bad assumptions or documentation and needs to start digging into dependency source code to work it out.
Only then apply the implementation to your project from the reference implementation. Takes breaking down the tasks though to small enough units and closing those degrees of freedom.
Anecdote on degrees of freedom: This one didn’t require a reference implementation in particular. I was reviewing a PR (LLM assisted, I wasn’t the authoring dev) to add signature validation to OAuth tokens. It duplicated the entire header/token parsing logic. It needed that path closed with a pointer to where the existing logic was and explicit requirements to enhance it. Refactor was great upon reviewing and the PR size was reduced by more than half.
I think there would be a lot less drama around this if authors were just up-front about how they use AI. Put it in your readme, just like you do with licenses.
The commits were literally in plain sight. If people didn’t notice it from that alone, then a disclaimer in the README would have gone unnoticed either. The project received several github issues contributing nothing but “remove the AI slop” to the project. If this is the reaction you get for using AI openly, then don’t be surprised when more devs just don’t disclose AI use at all
Not straightforward with projects that pre-date coding agents, which is the overwhelming majority?
Why not? I’ve added it to my projects. It’s simple, just open README.md. Write “# Use of AI. This project does not currently use AI. / This project is entirely vibe coded & I don’t read the code at all. / I occasionally use Claude Code but thoroughly review its output.”
Save. Commit. Push. How is that not straightforward?
I hate when AI people say “things are so different in just the past few weeks, what you know from last year is meaningless” without specifying what’s so groundbreaking that us regular folks wouldn’t be able to comprehend. It just seems like a way to shut people up and feel superior.
The point is that AI is developing at an insane rate. They don’t specify, because you would always have to be naming new things every other week, by the very nature of the statement. Things AI was not able to do a month ago, it may be able to do incredibly well now.
If you want an example, AI in security vulnerabilities has made quite a breakthrough recently. Not just Mythos, but multiple AI’s are finding 15+ year old vulnerabilities in open source packages basically the entire world relies on. It couldn’t do that a few months ago.
i think he’s talking about agentic harnesses getting better, and the new models being finetuned to use them. I don’t think the new models are much “smarter,” but it allows them to write shitloads of bad code and tests, then iterate over them until they’re “fixed.”
Or alternatively “You’re just prompting it wrong”
Yeah, but have you tried Slaupe Octopus 6.9? It’s vastly superior to anything else on the market.
He makes some fair points. However I do think the large amount of regressions in 3.4.3 should have resulted in a new release rolling back those changes.
I still like the response of the libxml2 maintainer, where any vulnerability will be disclosed openly and fixed when it’s ready. Maybe more open source projects currently drowning in CVE should take that stance instead of their maintainers burning themselves out over it.
I think “stochastic parrot” is a terrible way to describe LLMs. (Not to mention most people don’t use the term “stochastic” a lot.)
“Slot machine autocomplete” might be a better choice.
The intended audience of “stochastic parrot” was other AI researchers who do use the term “stochastic” on a regular basis.
If you feel the need to dumb it down, ‘statistical parrot’ works OK. I’m happy with the original.
Parrots also don’t just mindlessly repeat shit like an LLM does, parrots are intelligent AI is not.
Parrots are cool tho
