Seems like he’s been pushed into using LLMs as a way to cope with the deluge of LLM-generated security reports.

  • Arthur Besse@lemmy.ml
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 hours ago

    One shot rewriting the whole test suite

    tridge’s blog post makes it clear that this was not “one-shotted” at all.

    You should read the whole thread

    I regret reading it; I’ll assume in good faith that it wasn’t LLM generated but it is ironically as confidently wrong as if it were.

    It almost (and should have) lost me when it started by quote-agreeing with someone else saying “rsync was basically done until the maintainer discovered vibecoding” - no, pay attention, it was not “basically done”, there were/are a mountain of CVEs!

    But then this got my interest:

    This does not “translate tests into pytest” or a unit testing framework, it writes its own testing framework where tests are whole python scripts that redefine basic test functions in every script. Surely there would be a single way to “run rsync and get the results” - nope, well, there is, but then every test file will randomly redefine its own _run_and_capture function.

    tridge says he has used pytest on other projects and had good reasons not to use it here; I’m inclined to believe him.

    But the notion of every test defining its own way to invoke rsync sounded like a valid criticism, and an easy one to verify, so I checked: It turns out that there is in fact a common run_rsync function which is used by the majority of the tests. One test defines its own _run_and_capture function (which differs in that it writes the output to a file, for reasons I didn’t investigate), and it looks like a few others invoke rsync other ways, but the majority of them use the common function.

    So, that rambling thread’s sole concrete criticism of rsync’s new python tests turns out to be false.