Yes. It is an automated project and I am not going to pretend otherwise.
The measurements are real, the binaries and flags are published, and the raw CSV is CC BY so you can run the same thing and tell me the numbers are wrong. That is the only claim I am making. Several people in this thread already found real problems with the methodology and they were right, which is roughly what I wanted from posting it.
Gatus is a good call. It is close to a controlled comparison against Uptime Kuma, same job, Go vs Node, so whatever the gap is you can mostly attribute it to the runtime rather than the feature set. Adding it.
Taking these in order.
Point 3 first, because you are right and I was sloppy. There is no resident .NET layer sitting under the *arr apps. What I actually observed is four apps landing within 5 MB of each other, and the cause is that each process loads its own copy of the same runtime assemblies, not that something shared is running underneath. My wording implied a shared layer that does not exist. I will correct that on the site.
Point 2: the exact binary, flags and health check for each app are on its page on the site, but you are right that none of that was in the post, and the post is what most people read. Short version: every app is the upstream release binary run directly, no distro packages, no containers, so the numbers exclude container overhead.
Point 1: fair, and it undercuts how I framed the sizing rule. On a provider that allows bursting, a floor matters less than I implied.
Point 4 is the useful one for me. 700 MB for a whole *arr stack under real use is a much better number than anything I published, and it comes from someone who has watched it since the mono days. If you have a rough split per app I will put it up as a reported real-world figure alongside my measured idle ones, credited to you.
Point 5: agreed, and it is what I am fixing right now.
That is a legitimate hit and the beginner framing makes it worse, agreed. A floor shown to someone who does not know it is a floor gets read as a budget, and then they buy the 1 GB box.
Two things I am changing. First, I am measuring peak RSS under concurrent load now so no app is ever listed with only an idle figure. Second, the site framing goes, because “here is the floor, good luck” is exactly the rest-of-the-owl problem you are describing.
If you have a multiplier you actually trust for the gap between idle and real usage, I would rather publish yours with credit than invent one.
You are right, and the 1070 MiB vs 173 MB gap is exactly the thing that makes my number misleading.
Mine is a cold instance, no repos, no CI, no users, sampled 60 seconds after start. Yours is doing real work with real repo data cached. So the honest reading of my figure is “Forgejo will not start in less than 173 MB”, not “Forgejo runs in 173 MB”. I framed it as the second thing and I should not have.
I am re-running the whole set under concurrent load now to publish a second column.
If you are willing: roughly how many repos and how many users hit your instance? I would rather put a real-world datapoint next to the synthetic one than keep publishing only the synthetic one.
Fair question, and it is the main weakness of what I posted.
The floor tells you what you can rule out, not what you need. If Sonarr will not even start under 190 MB, you know a 512 MB box is already tight before you have indexed a single thing. That is useful for elimination and not much else.
Several people in this thread said the same, so I am running the follow-up now: the same apps, but measuring peak RSS while they are being hit by 24 concurrent clients, plus requests/sec so you can see what the memory bought you. Every app gets an idle number and a working number next to it.
If there is a specific workload you would want simulated rather than a generic HTTP hammer, tell me and I will add it.
Agreed, and this cuts deeper than the sizing question.
Most of these are idle event loops waiting on a socket. The memory is mostly runtime and heap that was allocated and never returned, so on a box under pressure a lot of it is reclaimable or swappable and the RSS I am reporting overstates what is genuinely needed at rest. That is another reason the idle number is weak.
Socket activation is the real version of your point. If an app is genuinely request driven then its idle cost can be near zero and the number that matters is what it grows to on first request and whether it ever gives it back. I do not measure the giving-it-back part at all right now, which I should, because that is the difference between a stack that fits in 2 GB and one that slowly does not.
Adding a post-load settle measurement to the harness is cheap so I will do that. I already sample for 10 seconds after load stops and the peak does not come back down much, but I have not run it long enough to say anything solid.