virtio-blk Support for Custom VMM

Introduction

Continuing gokvm development ^{1 2 3 4 5}.

Following the previous virtio-net support, I added support for virtio-blk. The data structure and behavior of virt queue can be reused as is. I’m impressed that Virtio is well designed in this regard.

7389ff59 Adjust Guest Kernel Compile Options

To achieve block IO through the filesystem from the guest kernel, I enabled the following options:

• CONFIG_VIRTIO_BLK=y
• CONFIG_XFS_FS=y
• CONFIG_EXT3_FS=y
• CONFIG_EXT4_FS=y

4f4bbb78 virtio-blk Implementation

Now, let’s move on to the main topic of virtio-blk implementation. The behavior of virtio-blk is almost the same as virtio-net, so if you haven’t read the previous blog, I recommend reading it first. The only differences are the number of queues and the data structure pointed to by entries in the descriptor table. While virtio-net required two queues for sending and receiving, virtio-blk achieves reading and writing with one queue. This is because both disk reads and writes are issued from the OS side, so there’s no need to receive external interrupts. The data structure pointed to by descriptor table entries is that three entries are connected in a Linked List manner as follows ⁶.

The data structure pointed to by the first entry is blkReq, where if the type field is 1 it means write, and 0 means read. The sector field means the offset from the beginning of the disk. Since one sector is 512 bytes, it means reading or writing from sector x 512 bytes from the beginning of the virtual disk file.

type blkReq struct {
	typ    uint32
	_      uint32
	sector uint64
}

The second entry points to the actual data. The data you actually want to read or write is stored here in binary format. The third entry is the status. If an error occurs, write a non-zero number.

Other aspects such as Virt Queue initialization method and usage of Avail Ring and Used Ring are exactly the same as virtio-net.

0819b1ed Add Scenario Test

I want to test that the block device provided by virtio-blk is working correctly using Go’s standard testing framework $ go test. There are probably several ways to do this, but this time I created a test disk file called vda.img and formatted it with ext2. Furthermore, I mounted vda.img as a filesystem and placed index.html inside it.

In the Go test, I boot a VM and the guest starts an http server. Then, by reading the previously created index.html from outside with the curl command, I was able to verify that virtio-blk is working correctly inside the guest. I successfully implemented a scenario test according to the standard testing framework.

Conclusion

I was able to implement block IO and networking using virtio. It’s interesting that I’m gradually developing a feeling of being able to operate device virtualization in my own hands. Now, what should I aim for next? If I can get my hands on hardware, I’d like to challenge different architectures like ARM or RISC-V. Or advanced implementations like live migration would also be interesting.

During this implementation, I also received some pull requests on github.com. It’s a niche area, but I’m happy there are people interested in it.