I read Learning eBPF, so I’d like to leave some reading notes. This book was released last year and I’ve been wanting to read it for a while. lizrice/learning-ebpf - github.com has abundant sample code for reference. I’ll just note things that caught my attention personally, without paying much attention to context. BCC It starts with an example using BCC. Using bpf_trace_printk() allows you to output text to the pseudo-file /sys/kernel/debug/tracing/trace_pipe....
I conducted an experiment to run SRv6 L3VPN in Mininet 1. The script is here. I was able to run it with a small configuration as shown in the diagram, so I’d like to introduce it here. Two routers (r1 and r2) are responsible for Encap/Decap with SRv6, and they exchange L3VPN information between r1 and r2 via eBGP. r1 and r2 each have two VRFs (vrf10 and vrf20), and tenants are separated by VRF (Tenant10 and Tenant20)....
First, let me introduce TRex. TRex 1 is a software-implemented traffic generator that supports two modes: Stateful/Stateless. Stateless is a mode for generating packet sequences to a stateless target DUT (Device Under Test), and can be used as a performance measurement tool for switching and routing. Although TRex has many features, personally I often need to generate simple TCP/IP packet sequences while changing their size, so I created autotrex 2 as a wrapper for TRex....
I bought an Intel NUC 12 Pro 1 as a development machine and installed Proxmox VE 2. Until now, I was using an old Thinkpad as a development machine, so I think it’s become more comfortable. The specs are like this. I wanted to prioritize memory and disk, so I went with more. On the other hand, I didn’t care much about core count, so I chose the i3 model. I considered a vPro-equipped model, but I passed on it because of limited availability and the assumption that virtualization would reduce the frequency of remote management operations....
It seems I broke something under /lib/modules while playing with a privileged container. I’m not sure why, but when going out from this machine to the outside, it sometimes connects and sometimes doesn’t. I’m using a BUFFALO LUA4-U3-AGTE-NBK 1 adapter that receives Ethernet as USB Type-A, and the driver for it seems suspicious. The log looks like this. ubuntu2004thinkpad:~$ dmesg IPv6: ADDRCONF(NETDEV_CHANGE): enx9096f349a025: link becomes ready usb 3-1: USB disconnect, device number 45 ax88179_178a 3-1:1....
Introduction When adapting gokvm to vhost-user, I investigated the initialization part, so I’m leaving it as a memo. The QEMU documentation Vhost-user Protocol 1 summarizes it in detail, but there are parts that can’t be understood without actually running it (exception handling, request order, log output, etc.), so I tried it. After various trial and error, I was able to test it easily with just QEMU and DPDK. Here, I ran QEMU in server mode and DPDK in client mode....
Introduction Continuation of gokvm development 1 2 3 4 5 6. As introduced up to last time, by supporting virtio-blk and virtio-net, the virtual machine can now communicate with the outside through IO. This time, I changed the initrd from busybox-based to u-root-based, so I’ll talk about that. 0d89a47f Introduction of u-root-based initrd I received a Pull Request suggesting that a VMM written in Go language should have an initrd also written in Go....
Introduction Continuing gokvm development 1 2 3 4 5. Following the previous virtio-net support, I added support for virtio-blk. The data structure and behavior of virt queue can be reused as is. I’m impressed that Virtio is well designed in this regard. 7389ff59 Adjust Guest Kernel Compile Options To achieve block IO through the filesystem from the guest kernel, I enabled the following options: CONFIG_VIRTIO_BLK=y CONFIG_XFS_FS=y CONFIG_EXT3_FS=y CONFIG_EXT4_FS=y 4f4bbb78 virtio-blk Implementation Now, let’s move on to the main topic of virtio-blk implementation....
Introduction Continuing gokvm development 1 2 3 4. Through recent development, I was able to provide a virtual NIC to VMs on gokvm via virtio-net. Networking support was one of the initial goals, so I feel a sense of accomplishment. With this support, VMs on gokvm can now communicate with the host (or the outside via a software switch). It broadens what you can do, such as providing a web server or logging in via SSH, which I think is a major change....
Introduction Continuing gokvm development 1 2 3. I’ve been developing to enable VMs on gokvm to handle PCI devices. The road will be long, but ultimately I want to establish IP connectivity between the VM and the outside via virtio-net. At the moment, I was able to get the guest kernel to recognize the virtio-net device as a network interface, so I’ll leave a log up to that point for now....
Introduction A progress report on gokvm development 1 2. Until now, I only supported a single virtual CPU. I wanted to support SMP (Symmetric Multiprocessing) for multi-CPU, and after about 2-3 weeks of trial and error, I was able to implement it successfully. As far as I know, while searching for efforts to create a VMM with KVM, I couldn’t find many resources that explain what SMP support specifically looks like in implementation....
This blog has covered relatively new networking technologies, but it’s good to revisit the fundamentals. The Linux kernel will be used for a long time to come, and even without perfect understanding, having a grasp of the basics is meaningful. This book is over 1,000 pages and consists of parts 1-7, so maintaining motivation to read it all at once is difficult. In this article, I’ll summarize what I’ve read so far....
As many people have already undertaken similar efforts and compiled them nicely into blog posts 1 2 3, I think being able to quickly create such an environment at hand is very meaningful. Here I’ll record the rough mechanism. I’ve compiled the results into scripts and uploaded them to Github. Features Supports building kernels for major distributions like CentOS6, CentOS7, Ubuntu20.04, enabling practical applications Uses Busybox to deploy userland in memory, creating a pure and minimal environment on each boot SSH login and external network connectivity are possible, making it easy to verify operations involving integration with other systems Can reference internal kernel data through debugging with GDB Currently only supports x86/64 Building the Kernel Building the kernel basically corresponds to writing the build configuration in a ....
Introducing https://github.com/bobuhiro11/mininetlab. Using Mininet, you can run several switches and hosts on a single machine. I’ll use this to virtually launch two hosts and connect them via BGP (Unnumbered) included in the FRR package. In Mininet, you can describe the topology and command execution on each host in Python as follows. Although FRR may seem complex at first glance, if you properly place the three files daemons, vtysh.conf, and frr.conf, you can easily start it with frrinit....
2021/2/24 WSL2 Support 4f6b785 When running gokvm on Ubuntu 20.04 on WSL2 (Windows Subsystem for Linux 2), output to IO port 0x64 was repeated infinitely and didn’t reach the Init process startup. It seems the behavior around the PS/2 keyboard was the cause. In kvmtool, it returns 0x20 for in (0x61) 1, so I followed that approach. IO port 0x61 appears to be used as NMI (Non-Maskable Interrupt) status and control register 2....
Introduction I created a naive and experimental VMM using KVM. It creates virtual machines by calling /dev/kvm through ioctl, and can boot the Linux Kernel and user processes on them. I also implemented a very simple serial console emulation that can be recognized by the kernel’s device driver, allowing operation from the login shell. On the other hand, networking and disks are not yet supported at this time. Recently, KVM has been used not only as a traditional virtual machine, but also to strengthen isolation levels in multi-tenant cloud environments, such as Google gVisor 1, Kata Containers 2, and Amazon Firecracker 3, for use in containers and micro VMs....
What is SRv6 SRv6 is an extension of IPv6 that implements Source Routing. Source Routing means that the data sender specifies not only the destination but also the route. Nodes to pass through are identified by SIDs (Segment Identifiers), and the route can be freely controlled by including the list in the packet header. In SRv6, an IPv6 address corresponds to a SID. The specification of SRv6 is being developed mainly by IETF (Internet Engineering Task Force) 1....
I was able to run virtual machines on a WSL2 guest on Windows 10 via /dev/kvm in a nested configuration. Environment Windows 10 Pro Insider Program (Dev Channel, OS Build 20246.1) 1 Guest on WSL2 Ubuntu 20.04.1 LTS (Focal Fossa) Linux 4.19.128-microsoft-standard Kernel parameters initrd=\initrd.img panic=-1 nr_cpus=4 swiotlb=force pty.legacy_count=0 QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.7) Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz Procedure By adding the following settings to the WSL2 (Windows Subsystem for Linux 2) global configuration C:\Users\username\....
A method to achieve high-performance (NIC wirespeed) and flexible I/O in virtual machine and container environments. Not much Japanese information is available yet. I haven’t actually tried it, so there may be misunderstandings. vDPA Kernel Framework In March 2020, the vDPA kernel framework was merged into Linux 5.7. A vDPA device handled by the vDPA kernel framework refers to a device where the data plane follows the virtio specification and the control plane is vendor-specific....
Introduction I read “The eXpress data path: fast programmable packet processing in the operating system kernel” 1. This article is mostly based on this paper, with some references to news articles. The popularity of eBPF/XDP can be felt from the GitHub star counts of projects using eBPF/XDP, such as BCC, bpftrace, Facebook Katran, and Cloudflare Gatebot. eBPF/XDP has a powerful advantage: it can achieve high-speed packet processing as a kernel mechanism without depending on special hardware or software....