Xen storage for NetBSD guests: performance vs. consistent backups (sanity check)

●To: NetBSD <netbsd-users%NetBSD.org@localhost> ●Subject: Xen storage for NetBSD guests: performance vs. consistent backups (sanity check) ●From: Matthias Petermann <matthias%d2ux.net@localhost> ●Date: Mon, 19 Jan 2026 21:15:17 +0100

Hello,

I am currently in the process of rebuilding my home server from scratch using NetBSD and would appreciate a bit of sanity-checking from people with more experience in this area.

The setup is a fairly typical Xen environment: one NetBSD Dom0 and a small number of NetBSD guest VMs running various workloads. Some of these are relatively data-intensive (e.g. a Samba file server and a database), so three requirements are absolutely critical for me:


- Reliable, consistent online backups from running guests
- Good read/write performance, ideally close to raw disk access
- Flexibility in terms of free space allocation (on demand)

=== Background: backup workflow ===

For backups, I strongly prefer doing them from inside the guest, using dump with FFS snapshots (dump -X / fssconfig), streaming the dump over SSH to the host and writing it directly to an external USB disk. This approach has worked very well for me over the years - when the storage stack underneath behaves nicely.

Unfortunately, once LVM enters the picture, I have repeatedly run into situations where triggering an FSS snapshot causes severe stalls or complete lockups. ZFS zvols, while very attractive feature-wise, showed significantly lower performance in my setup compared to raw partitions, and also compared poorly to simpler approaches such as CCD or even vnd-backed storage.


=== Performance analysis ===

Towards the end of last year I ran a series of simple but reproducible performance tests to compare different Xen guest storage backends under NetBSD. The results are described in more detail here:

https://www.petermann-digital.de/blog/netbsd-xen-storage/

In short, the observations were not too surprising:

- Raw partitions deliver the best and most predictable performance (asexpected).


- vnd-backed images introduce some overhead but are still reasonably close.

- ZFS zvols were significantly slower in my environment, especially for write-heavy workloads.

- CCD-backed volumes, built from GPT partitions, came very close to raw performance while remaining simple and transparent.

Given these results, CCD appeared to hit a sweet spot between performance and flexibility, without introducing an additional snapshotting or metadata layer that could interfere with FFS/FSS.


=== CCD-based approach and ccdtool ===

Based on this, I ended up using CCD volumes composed of multiple GPT partitions, exposed to guests as block devices. To make this manageable, I wrote a small shell tool (ccdtool, https://forge.ext.d2ux.net/Polaris/toolbox/src/branch/main/ccdtool) that provides very lightweight, LVM-like convenience:


- human-friendly aliases instead of raw ccdN names

- simple create/extend/destroy operations

- no hidden metadata, only GPT labels and /etc/ccd.conf

- no attempt to add snapshot logic — snapshots are still handled purely at the FFS/FSS level


This setup has been running quietly and reliably for a few months now.

Why I’m posting? Now that I am about to rebuild everything cleanly and migrate the data, I would like to pause for a moment and ask:


- Does this overall approach sound reasonable to you?

- Is there an important detail in my performance analysis that I might have overlooked?

- Is there a simpler or more idiomatic NetBSD/Xen solution today that would achieve the same goals (consistent guest-side backups + near-raw performance + flexibility) without the complexity or issues I observed with LVM and zvols?


Thanks a lot for your time and for any feedback you might have.

Best regards,
Matthias


--
Für alle, die digitale Systeme verstehen und gestalten wollen:
jede Woche neue Beiträge zu Architektur, Souveränität und Systemdesign.
👉 https://www.petermann-digital.de/blog

●Follow-Ups: ●Re: Xen storage for NetBSD guests: performance vs. consistent backups (sanity check) ●From: Sad Clouds ●Re: Xen storage for NetBSD guests: performance vs. consistent backups (sanity check) ●From: Hauke Fath ●Re: Xen storage for NetBSD guests: performance vs. consistent backups (sanity check) ●From: Niels Dettenbach

●Prev by Date: Re: Internet services and US vs EU jurisdiction ●Next by Date: Re: Xen storage for NetBSD guests: performance vs. consistent backups (sanity check) ●Previous by Thread: Name resolution and underscores? www.engadget.com ●Next by Thread: Re: Xen storage for NetBSD guests: performance vs. consistent backups (sanity check) ●Indexes: ● reverse Date ● reverse Thread ● Old Index Home | Main Index | Thread Index | Old Index