Host-side guided disk resize via UI when using Apple Virtualisation

[lloeki]

Note: this issue is specific to Apple Virtualization, not QEMU, which has the feature. It is also not about growing the partitions/filesystem within the guest.

What did you see?

There is no resize option when using Apple Virtualization:

What did you expect?

A nice UI similar to the one when using QEMU:

• at a minimum: to grow the disk size.
• bonus: to shrink the disk size (which would be just as dangerous as it its for QEMU if the underlying filesystem/partition has not been shrunk).
• bonus: to compress disk usage while maintaining disk size.

Additional Information

Situation

Finder detects it as being a disk image, mountable via DiskImageUtil:

file detecting a MBR boot sector + size seems to indicate it's presumably a raw image, entirely preallocated:

$ file ~/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/Data/EF8386E1-7CB8-4307-A844-DB6EC293D22E.img
DOS/MBR boot sector; partition 1 : ID=0xee, start-CHS (0x3ff,254,63), end-CHS (0x3ff,254,63), startsector 1, 268435455 sectors, extended partition table (last)

$ ls -lh ~/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/Data/EF8386E1-7CB8-4307-A844-DB6EC293D22E.img | awk '{ print $5 }'
128G

$ ls -l ~/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/Data/EF8386E1-7CB8-4307-A844-DB6EC293D22E.img | awk '{ print $5 }' 
137438953472

Math checks out down to the byte for a 128 GiB (not GB) image:

$ bc
>>> 137438953472 / 1024
134217728
>>> 134217728 / 1024
131072
>>> 134217728 /1024
131072
>>> 131072 / 1024
128

BUT it's a sparse file, which is visible when using du (real usage) vs du -A (apparent file size); oh, and using -k to fix on KiB instead of blocks (default) or variable-scale humanised size (-h):

$ du -k ~/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/Data/EF8386E1-7CB8-4307-A844-DB6EC293D22E.img | awk '{ print $1 }'
100892952

$ bc
>>> 100892952 * 1024
103314382848

$ du -k -A ~/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/Data/EF8386E1-7CB8-4307-A844-DB6EC293D22E.img | awk '{ print $1 }'
134217728

$ bc
>>> 134217728 * 1024
137438953472

There seems to be no indication of disk size being stored in configuration, so presumably it is inferred from the apparent file size:

$ cat ~/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/config.plist | grep -A12 '<key>Drive</key>'
	<key>Drive</key>
	<array>
		<dict>
			<key>Identifier</key>
			<string>EF8386E1-7CB8-4307-A844-DB6EC293D22E</string>
			<key>ImageName</key>
			<string>EF8386E1-7CB8-4307-A844-DB6EC293D22E.img</string>
			<key>Nvme</key>
			<false/>
			<key>ReadOnly</key>
			<false/>
		</dict>
	</array>

Apple Developer documentation seems to point towards it being indeed a raw image:

Create a RAW disk image in the file system of the host computer, and use that disk image to initialize your VZDiskImageStorageDeviceAttachment object.

So, on the host side, presumably simply growing the file to a bigger size would make the disk bigger.

Manually growing the image

Test run

Just for making sure of the units used, let's create a 1 GiB file using macOS dd (which is BSD-based and may differ from GNU dd):

$ dd if=/dev/zero of=./test.img bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes transferred in 0.168656 secs (6366460867 bytes/sec)

$ ls -l ./test.img | awk '{ print $5 }'
1073741824

Checks out as 1GiB (not GB), down to the byte.

Let's grow the file by, say, 512MiB by appending - very simply via a shell >> redirection - to it:

$ dd if=/dev/zero bs=1M count=512 >> test.img  
512+0 records in
512+0 records out
536870912 bytes transferred in 0.225418 secs (2381668332 bytes/sec)

$ ls -l ./test.img | awk '{ print $5 }'
1610612736

Math checks out:

$ bc
>>> 1073741824 + 536870912
1610612736

That said, this wrote zeros to disk, so it is not a sparse file:

$ du -k -A test.img | awk '{ print $1 }' 
1572864

$ du -k test.img | awk '{ print $1 }' 
1572864

Let's do this correctly and create a file full of virtual zeros by doing no actual writes to the file's contents:

$ dd if=/dev/zero of=./test.img bs=1M count=0 seek=1024
0+0 records in
0+0 records out
0 bytes transferred in 0.000013 secs (0 bytes/sec)

$ ls -l ./test.img | awk '{ print $5 }'
1073741824

$ du -k -A test.img | awk '{ print $1 }'
1048576

$ du -k test.img | awk '{ print $1 }'
0

And we can grow the file in the same way:

$ bc                                                    
>>> 1024+512
1536

$ dd if=/dev/zero of=./test.img bs=1M count=0 seek=1536
0+0 records in
0+0 records out
0 bytes transferred in 0.000016 secs (0 bytes/sec)
$ du -k -A test.img | awk '{ print $1 }'               
1572864
$ du -k test.img | awk '{ print $1 }'                  
0

And there we go, we have extended a sparse file.

Extending the real disk

So in my case, the disk is using 100892952 KiB but its apparent size is 134217728 KiB (128 GiB) and I want it to be twice that (256 GiB) in apparent size:

$ du -k ~/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/Data/EF8386E1-7CB8-4307-A844-DB6EC293D22E.img | awk '{ print $1 }' 
100892952

$ du -k -A ~/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/Data/EF8386E1-7CB8-4307-A844-DB6EC293D22E.img | awk '{ print $1 }'
134217728

Converting 134217728 KiB to MiB because that's what I use in dd as block size (bs), then doubling it:

$ bc
>>> 134217728 / 1024
131072
>>> 131072 + 131072
262144

So, for growing to a 256GiB disk, 262144 is the value to plug into count with a bs of 1M (a.k.a MiB).

$ : dd if=/dev/zero of="${HOME}"/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/Data/EF8386E1-7CB8-4307-A844-DB6EC293D22E.img bs=1M count=0 seek=262144

Reminder to quit UTM (or at least stop the VM) before running such a command. Also why I added a : (shell NOOP) before it and prevent a dramatic fat-finger accident.

$ dd if=/dev/zero of="${HOME}"/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/Data/EF8386E1-7CB8-4307-A844-DB6EC293D22E.img bs=1M count=0 seek=262144
0+0 records in
0+0 records out
0 bytes transferred in 0.000020 secs (0 bytes/sec)

The actual disk usage is untouched, and the apparent file size is correct:

$ du -k ~/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/Data/EF8386E1-7CB8-4307-A844-DB6EC293D22E.img | awk '{ print $1 }'
100892952

$ du -k -A ~/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/Data/EF8386E1-7CB8-4307-A844-DB6EC293D22E.img 
268435456	/Users/loic.nageleisen/Library/Containers/com.utmapp.UTM/Data/Documents/Personal.utm/Data/EF8386E1-7CB8-4307-A844-DB6EC293D22E.img

$ bc
>>> 262144 * 1024
268435456

Starting UTM, the correct size is shown, in both disk usage and as 256 GB Drive:

And in the VM settings:

Of course as usual, data within the disk did not change, so within the guest OS both partitions and filesystems (being data relatively to the raw disk) are untouched (here, a macOS guest, showing sizes in GB, not GiB):

To see the actual disk, toggle Show All Devices and witness the VirtIO disk having grown.

Left as an exercise to the reader:

• Resizing the guest filesystem (out of scope for this feature request).
• Shrinking a disk size.
• Compressing a disk i.e reducing real disk usage while maintaining apparent size, i.e making it as sparse as possible.

[mig8447]

This post has instructions for macOS guests as well #4186 (comment)