I recently decided to rebuild my homelab after a nasty double hard drive failure (no important files were lost, thanks to ddrescue). The new setup uses one SSD as the PVE root drive, and two Ironwolf HDDs in a RAID 1 MD array (which I’ll probably expand to RAID 5 in the near future).
Previously the storage array had a simple ext4 filesystem mounted to /mnt/storage, which was then bind-mounted to LXC containers running my services. It worked well enough, but figuring out permissions between the host, the container, and potentially nested containers was a bit of a challenge. Now I’m using brand new hard drives and I want to do the first steps right.
The host is an old PC living a new life: i3-4160 with 8 GB DDR3 non-ECC memory.
-
Option 1 would be to do what I did before: format the array as an ext4 volume, mount on the host, and bind mount to the containers. I don’t use VMs much because the system is memory constrained, but if I did, I’d probably have to use NFS or something similar to give the VMs access to the disk.
-
Option 2 is to create an LVM volume group on the RAID array, then use Proxmox to manage LVs. This would be my preferred option from an administration perspective since privileges would become a non-issue and I could mount the LVs directly to VMs, but I have some concerns:
- If the host were to break irrecoverably, is it possible to open LVs created by Proxmox on a different system? If I need to back up some LVM config files to make that happen, which files are those? I’ve tried following several guides to mount the LVs, but never been successful.
- I’m planning to put things on the server that will grow over time, like game installers, media files, and Git LFS storage. Is it better to use thinpools or should I just allocate some appropriately huge LVs to those services?
-
Option 3 is to forget
mdadmand use Proxmox’s ZFS to set up redundancy. My main concern here, in addition to everything in option 2, is that ZFS needs a lot of memory for caching. Right now I can dedicate 4 GB to it, which is less than the recommendation – is it responsible to run a ZFS pool with that?
My primary objective is data resilience above all. Obviously nothing can replace a good backup solution, but that’s not something I can afford at the moment. I want to be able to reassemble and mount the array on a different system if the server falls to pieces. Option 1 seems the most conducive for that (I’ve had to do it once), but if LVM on RAID or ZFS can offer the same resilience without any major drawbacks (like difficulty mounting LVs or other issues I might encounter)… I’d like to know what others use or recommend.
You must log in or # to comment.
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
Fewer Letters More Letters LVM (Linux) Logical Volume Manager for filesystem mapping SBC Single-Board Computer ZFS Solaris/Linux filesystem focusing on data integrity
[Thread #234 for this comm, first seen 12th Apr 2026, 07:10] [FAQ] [Full list] [Contact] [Source code]
Zfs for some reasons is always loved by the self hosting experts.
Personally I don’t like it because it’s over complicated and not officially part of the Linux kernel.
20y of self hosting for me means Linux software raid (raid1 then moved to raid5) and mostly ext4, recently (=last 2 years) upgraded to btrfs on my data raid. Btrfs at least is integrated in the Linux kernel and while has some drawbacks (don’t do raid with btrfs, put btrfs on MD raid instead) its been super rock solid.
I would have kept ext4, but thought why not try btrfs, and it’s been so smooth that I had no reasons to go back.
That whole “1 GB per TB of capacity” is some generic rule someone made up once that doesn’t really have anything backing it up. It depends completely on your use case. If it’s mostly media storage that is rarely accessed, I’m sure that 4 GB is plenty.
I run a beefy TrueNAS server for a friends video production company with a 170 GB ZFS array, right now ARC is using 40 GB of RAM with 34 GB free that it’s not even bothering to touch, I’m sure most of the ARC space is just wasted as well. That’s just one example of how 1 TB = 1 GB makes no sense.
If you end up with a second proxmox server with its own storage for redundancy in a cluster then ZFS is great. You can mirror each VM. The only issue is the ZFS storage needs to be named the same. But it can be a different size or media.
My main concern here, in addition to everything in option 2, is that ZFS needs a lot of memory for caching. Right now I can dedicate 4 GB to it, which is less than the recommendation – is it responsible to run a ZFS pool with that?
Yes, you’re going to be OK.
In your typical “self hosting” scenarios you can safely stay well below the RAM recommendation - assuming you don’t have many simultaneous users and tasks.
ZFS does not actually need that much RAM for basic operation, but it can make very good use of it in medium/heavy load scenarios.
ZFS is slightly more portable than md and comes with some additional benefits and performance tunables you don’t get with md, so that’s my 0.02.
Note that ZFS can use memory for arc, but it is happy to run on much less than what you have. My nas is a rock 5 SBC with 4GB memory and it’s fine, performance-wise, but I’ve run ZFS on much less and it adapts well to more or less ram.
Just curious – why is the 4GB cache needed?
ZFS uses the RAM intensively for caching operations. Way more than traditional filesystems. The recommended cache size is 2 GB plus 1 GB per terabyte of capacity. For my server, that would be three quarters of the RAM dedicated entirely to the filesystem.
If you plan to expand further, or add another node, I would recommend starting with zfs and see how it performs, because it’s integrated with proxmox and is required for native replication. But you can’t safely convert a zfs mirror to raidz1, you have to take one of the mirrors offline, create a degraded raidz1 with only two disks, copy the data, then wipe the third disk and add it. It’s sketchy and should not be performed with data you can’t afford to lose. But it works, I’ve done it.
Second to that, I’d do traditional software raid, though you’d probably end up doing the same process.
I personally don’t like lvm. I find it frustrating to work with. I’m not sure if you could do the same operations as above.
I’m not sure if you can do data recovery on proxmox virtual disks, but even if the host is completely dead, you can reinstall it and import the VMs and disks. I’ve done that too. But now I run pbs and rclone the backup files to the cloud.
