I conducted an experiment to run SRv6 L3VPN in Mininet 1. The script is here.
I was able to run it with a small configuration as shown in the diagram, so I’d like to introduce it here. Two routers (r1 and r2) are responsible for Encap/Decap with SRv6, and they exchange L3VPN information between r1 and r2 via eBGP. r1 and r2 each have two VRFs (vrf10 and vrf20), and tenants are separated by VRF (Tenant10 and Tenant20). Of course, there is no L3 connectivity between tenants and overlapping IP ranges are allowed, so the same prefix is assigned here.
On r1 and r2, FRR is used as the BGP daemon. Here’s an excerpt from the configuration. The complete FRR configuration file (frr.conf) is described in mininetlab 2. The FRR test code 3 is very concise and well organized, which I used as a reference. FRR’s SRv6 L3VPN development is active, so I recommend using a newer version if possible. FRR 8.5 was used here.
mininet> r1 vtysh -c "show running-config"
# Excerpt
router bgp 65001
bgp router-id 203.0.113.1
bgp default ipv4-vpn
bgp default ipv6-unicast
bgp bestpath as-path multipath-relax
no bgp network import-check
neighbor r1-eth0 interface remote-as external
!
segment-routing srv6
locator default
exit
exit
!
router bgp 65001 vrf vrf10
bgp router-id 203.0.113.1
!
address-family ipv4 unicast
redistribute connected
sid vpn export 16
rd vpn export 65001:10
rt vpn both 0:10
export vpn
import vpn
exit-address-family
exit
!
segment-routing
srv6
locators
locator default
prefix 2001:db8:1:1::/64 block-len 40 node-len 24 func-bits 16
!
end
In SRv6, the destination is expressed as a list of SIDs (Segment IDs) during Encap.
SIDs can be written in IPv6 format and consist of three fields: Locator, Function, and Argument.
Locators are usually given individual values for each router, so r1 is assigned 2001:db8:1:1::/64.
The router ASN and router id are common to all vrfs, which are 65001 and 203.0.113.1 for r1.
To enable VPN in the Default VRF,
you need to configure bgp default ipv4-vpn to enable it for all neighbors,
or configure it individually for each neighbor in address-family ipv4 vpn.
sid vpn export corresponds to the function of the SID, so configure individual values for each VRF.
The IP prefix to advertise in the VPN is configured in address-family ipv4 unicast in the VRF configuration.
Mininet has a convenient CLI mechanism that allows you to enter each host and execute commands, so let’s add it as shown below and take a look.
$ git diff
diff --git a/mininetlab/srv6_l3vpn.py b/mininetlab/srv6_l3vpn.py
index 4a7c4a9a3f94..d3185398e252 100644
--- a/mininetlab/srv6_l3vpn.py
+++ b/mininetlab/srv6_l3vpn.py
@@ -2,6 +2,7 @@
from mininet.net import Mininet
from mininet.log import setLogLevel
+from mininet.cli import CLI
import time
frr_conf = '''
@@ -185,6 +186,8 @@ def run():
loss_rate = net.ping(hosts=[c11, c21]) + net.ping(hosts=[c12, c22])
+ CLI(net)
+
for h in [r1, r2]:
h.cmd("/usr/lib/frr/frrinit.sh stop")
First, let’s check the communication within the VPN roughly. Since c11 and c12 belong to different tenants, even if the destination IP address is the same, they communicate to different hosts.
mininet> c11 curl 192.168.2.1
c21
mininet> c12 curl 192.168.2.1
c22
Next, let’s check the routing table for both the Default VRF and Tenant10 on r1.
Entries with proto bgp are set by BGP.
In the Default VRF, Decap is performed for communication destined for the SID,
and the End.DT4 action is applied to vrf10 and vrf20.
In the VRF corresponding to Tenant10, the route to 192.168.2.0/24
applies an Encap action with r2’s SID 2001:db8:2:2:10::.
In this way, BGP can be used to configure Encap/Decap routing.
# Default VRF.
mininet> r1 ip -6 route show
2001:db8:1:1::1 dev lo proto kernel metric 256 pref medium
2001:db8:1:1:10:: nhid 11 encap seg6local action End.DT4 vrftable 10 dev vrf10 proto bgp metric 20 pref medium
2001:db8:1:1:20:: nhid 12 encap seg6local action End.DT4 vrftable 20 dev vrf20 proto bgp metric 20 pref medium
2001:db8:2:2::/64 nhid 22 via fe80::98de:31ff:fe2d:6903 dev r1-eth0 proto bgp metric 20 pref medium
fe80::/64 dev r1-eth0 proto kernel metric 256 pref medium
# VRF for Tenant10.
mininet> r1 ip -4 route show vrf vrf10
192.168.1.0/24 dev r1-eth1 proto kernel scope link src 192.168.1.254
192.168.2.0/24 nhid 23 encap seg6 mode encap segs 1 [ 2001:db8:2:2:10:: ] via inet6 fe80::98de:31ff:fe2d:6903 dev r1-eth0 proto bgp metric 20
Let’s also look at the SRv6 packets.
This can be confirmed by pinging from c11 while capturing packets on r1.
As intended, SRv6 packets are sent to SID 2001:db8:2:2:10::.
mininet> c11 ping 192.168.2.1 &
mininet> r1 tshark -i r1-eth0 -V host 2001:db8:2:2:10::
Frame 2: 162 bytes on wire (1296 bits), 162 bytes captured (1296 bits) on interface r1-eth0, id 0
Ethernet II, Src: 1a:4c:3e:9d:f7:09 (1a:4c:3e:9d:f7:09), Dst: 9a:de:31:2d:69:03 (9a:de:31:2d:69:03)
Internet Protocol Version 6, Src: 2001:db8:1:1::1, Dst: 2001:db8:2:2:10::
Next Header: Routing Header for IPv6 (43)
Source Address: 2001:db8:1:1::1
Destination Address: 2001:db8:2:2:10::
Routing Header for IPv6 (Segment Routing)
Next Header: IPIP (4)
Length: 2
[Length: 24 bytes]
Type: Segment Routing (4)
Segments Left: 0
Last Entry: 0
Flags: 0x00
Tag: 0000
Address[0]: 2001:db8:2:2:10::
Internet Protocol Version 4, Src: 192.168.1.1, Dst: 192.168.2.1
0100 .... = Version: 4
Protocol: ICMP (1)
Source Address: 192.168.1.1
Destination Address: 192.168.2.1
Internet Control Message Protocol
Type: 8 (Echo (ping) request)
Let’s also look at the BGP UPDATE message. Let’s add 192.168.5.0/24 to r1 vrf10.
RFC 9252 - BGP Overlay Services Based on Segment Routing over IPv6 (SRv6)
The TLV format matches the one described in.
mininet> r1 bash -c "sleep 10; ip addr add 192.168.5.254/24 dev r1-eth1" &
mininet> r1 tshark -i r1-eth0 -V port 179
Frame 6: 250 bytes on wire (2000 bits), 250 bytes captured (2000 bits) on interface r1-eth0, id 0
Ethernet II, Src: 9a:de:31:2d:69:03 (9a:de:31:2d:69:03), Dst: 1a:4c:3e:9d:f7:09 (1a:4c:3e:9d:f7:09)
Internet Protocol Version 6, Src: fe80::98de:31ff:fe2d:6903, Dst: fe80::184c:3eff:fe9d:f709
Transmission Control Protocol, Src Port: 179, Dst Port: 34248, Seq: 20, Ack: 163, Len: 164
Border Gateway Protocol - UPDATE Message
Path attributes
...
Path Attribute - MP_REACH_NLRI
Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
Type Code: MP_REACH_NLRI (14)
Address family identifier (AFI): IPv4 (1)
Subsequent address family identifier (SAFI): Labeled VPN Unicast (128)
Next hop: RD=0:0 IPv6=fe80::98de:31ff:fe2d:6903 RD=0:0 Link-local=fe80::98de:31ff:fe2d:6903
Route Distinguisher: 0:0
IPv6 Address: fe80::98de:31ff:fe2d:6903
Route Distinguisher: 0:0
Link-local Address: fe80::98de:31ff:fe2d:6903
Number of Subnetwork points of attachment (SNPA): 0
Network Layer Reachability Information (NLRI)
BGP Prefix
Prefix Length: 112
Label Stack: 256 (bottom)
# From "rd vpn export 65001:10"
Route Distinguisher: 65001:10
# Prefix 192.168.5.0/24 added to VPN this time
MP Reach NLRI IPv4 prefix: 192.168.5.0
...
Path Attribute - BGP Prefix-SID
Type Code: BGP Prefix-SID (40)
SRv6 L3 Service
# 5 for L3, 6 for L2. From "2. SRv6 Services TLVs"
Type: SRv6 L3 Service (5)
SRv6 Service Sub-TLVs
SRv6 Service Sub-TLV - SRv6 SID Information
# From "3.1. SRv6 SID Information Sub-TLV"
Type: SRv6 SID Information (1)
# SID for r1 vrf10
SRv6 SID Value: 2001:db8:1:1::
SRv6 SID Flags: 0x00
# From https://www.iana.org/assignments/segment-routing/segment-routing.xhtml
SRv6 Endpoint Behavior: End.DT4 (0x0013)
Reserved: 00
SRv6 Service Data Sub-Sub-TLVs
SRv6 Service Data Sub-Sub-TLV - SRv6 SID Structure
# From "3.2.1. SRv6 SID Structure Sub-Sub-TLV"
Type: SRv6 SID Structure (1)
# Locator bit count
Locator Block Length: 40
Locator Node Length: 24
Function Length: 16
Argument Length: 0
Transposition Length: 16
Transposition Offset: 64
I was able to understand the rough behavior. In the future, I’d like to make the SR Domain more complex and add Network Functions for each tenant. That’s all.