Naïve VM image holding FS benchmarking

other computers

Postby lvd » 10 Nov 2021, 19:39

Preface
Recently I've noted that among all my VMs, debian based ones were taking especially long time upgrading with `apt upgrade`. Specifically, time to depack/configure each packet seemed to be too long. This observation came after some time since I had started using BTRFS for my /home (where also all VM images are situated).

I also heard from the talks on official #btrfs irc channel that BTRFS is poor for storing VM images. So I decided to make some benchmarking, that would include BTRFS, classic EXT4 and widely-advertized ZFS. Also as a base level, a measurement was done with VM image written directly on the bare disk.

Conditions
Host machine: AMD FX 8320 cpu (8 cores), 16 Gb memory, dedicated (used only for this benchmarking) sata HDD ??????
Host OS: Ubuntu 20.04 running 5.11.??? kernel with ZFS 0.8.6

Hypervisor: qemu-kvm used through virt-manager.

Guest machine: 4 threads, 4Gb RAM, virtio disk and virtio network.
Guest OS: 64bit devuan ascii installed on 5Gb single-partition image (no swap), controlled through SSH.

Other conditions: during the tests, host machine did not run any cpu-, memory- or IO-intensive unrelated tasks.

Methodology
Basically, I was measuring time required to do `apt dist-upgrade` first from ascii to beowulf, then to chimaera. That time did not include download time, as the download was performed preliminarily with `apt dist-upgrade -d`

That time was measured while the VM image was situated on different kinds of storage, specifically:
    - Plain SATA disk
    - file on EXT4 on that disk
    - Nodatacow (+C) file on BTRFS on the same disk
    - Ordinary CoW file on BTRFS
    - Ordinary file on ZFS on the same disk
    - zvol device on ZFS


Each FS was created with full default options (generally no parameters to mkfs/zpool create), with the following exceptions:
- Each FS was mounted with 'noatime' (incl. ZFS)
- Also for ZFS, there were also 'mountpoint=none' for pool and 'mountpoint=/some/path' for single dataset. Otherwise everything was the default, including 'ashift', 'recordsize' and 'volblocksize'.

For each storage type, test suite was run twice, first with 'cache=writeback, aio=threads' disk setup in qemu, then with 'cache=none, aio=native'.

For each test, only single time measurement was done.

First I prepared a golden image of installed guest os: first install base system from ???-dvd.iso, then do `apt update/upgrade`, then install some extra packets (vim subversion git build-essential), then do `apt clean` and then shutdown: the golden image is ready.

For each test in question, the golden image was written anew to the specific destination, after what the VM was run off that destination.
For plain disk tests and for ZFS zvol tests, it was done simply as `dd if=golden.img of=/dev/device`, for non-CoW files that was `dd if=golden.img of=working.img conv=notrunc` (so that FS doesn't re-allocate file storage space while the contents is renewed). For a CoW file, the FS was additionally re-created anew for each run, while the image file was then simply copied. The FS recreation also took place for ZFS zvol tests.

The test itself included the following steps:

1. boot the VM, change /etc/apt/sources.list to the beowulf
2. `apt update`, then `apt dist-upgrade -d`.
3. First measurement: execute
Code: Select all
time bash -c '????'

4. after that, `apt autoremove --purge`, then `apt remove --purge linux-image ????` (removing older kernel), then `apt clean`, then reboot.
5. Again change /etc/apt/sources.list to chimaera, then same as in p.2
6. Second measurement with exactly the same command as in p.3
7. Reboot to check whether upgrade was successful.

The reason for second measurement was to compare (on a CoW FSes) how badly the image file could fragmented after the first measurement and how that would influence the IO speed.
F̞͖̭̿̔ͯu̐̅cͬ̑ͩk̨̤̳͇̮̭̪̠̽̿̓̆ͭͩ ̷̩̰͎̩͓̘̾̀ͬ̊ͭ͛ͅda̝̺͙̬͎̝̾͟ ̰̜̝̯͉̯̖̓̎́ͨ̽ͫ͟f̟͇̭̀ͬͨͭ̐̚u̹̼̹̗̞͑̔͂͐̚cͭ̅̊̆̒̆ǩ̝̩̯́ͥ̔̍̑ḭ͓͍̳̬ͦ̽͂n͍͎͈̈̅ͩͬ ̊ͫ̂̾̑̈́f̲͚͉͓͗̋́ͧͦ̅ȗ͇̲̻͈̲̅̎͗͒ͭ͡c̬̟̠̹̯̈́ͩ͘ͅk̫̠̻̋͜a̲͒̾̇!͙͕̺͉̗̩̲̂̏̄̀
User avatar
lvd
 
Posts: 7262
Joined: 07 Apr 2007, 21:28
Group: Registered users

Return to Другие компьютеры

Who is online

Users browsing this forum: No registered users and 1 guest

cron