I tried this in my homelab, and I wanted to convert my Proxmox with it’s LXC’s over to it…but the problem I ran into was that it was a little too obtuse for what I wanted to use it for.

Long story:

I wanted to get away from Proxmox as a platform and LXCs in general over to another hypervisor and using standard containers. I was tired of having a full fat web GUI to manage things with. Yes you can use pct and other CLIs tools in Proxmox to get some speed, but you are still dealing with an opinionated, dare I say proprietary stack.

Incus is really cool in that it’s CLI driven first, and you can add a GUI later if you want. I thought that was nice, and I was able to get some VMs started in my lab cluster and get clustering kinda working. I thought I was in a good place to switch.

But then, my main file server died (thankfully just the mobo, not the ZFS disks). And I was scrambling to figure out how to get all of my functionality back without having to go back to adding the storage back to Proxmox. So I took it as an opportunity to go to incus.

Well it turns out, it’s not so turn key depending on your method of deployment. For one, the actual “IncusOS” deployments are really particular about having a newer version of TPM (not something the old boxes I purchase typically have). So you are still likely to install something like Debian + incus-cli tools.

Once you have your distro + incus package setup though, there are a few things you have to tackle. For one, if you actually want separate IP addresses per VM (or LXC) you’ll likely need to solve the chicken and egg problem with macvlan, otherwise you’ll have to figure out something for a reverse proxy with the network bridge.

So because I was pressed against the wall (my family was irritated about no jellfyin or immich), and I knew that I didn’t want to do LXCs again and simply wanted to do most things I can in containers, I went the podman route instead.

Every app gets it’s own local user. You can install podman-compose, and use the same commands as docker compose basically to get your app up. All rootless, and you can also setup podman compose apps with daemons so that they start on boot: https://www.it-hure.de/2024/02/podman-compose-and-systemd/. If you don’t care about rootless isolation, I think Docker is still fine. Everyone seems to make documentation and build their apps with it in mind still.

I still had a need for VMs, so now I just use the standard Qemu+KVM+libvirtd combo for the one or two apps that need that need a full on separation from other apps.

All of it is in Ansible playbooks now, and I’m pretty happy with the setup I think.

TL; DR

Incus is really cool. I probably wouldn’t bother with IncusOS on old hardware, I’d just install a standard Linux distro + incus packages. Spend your time evaluating it (and making it infrastructure code for easy deployment maintenance). Maybe try out the other “standard” ways to do it instead to compare (Docker, Podman, Qemu+KVM, etc).