Ein weiteres Beispiel für eine Leaf-Spine Konfiguration. Heute aus dem Hause Cumulus Networks.
Folgende Topologie wird verwendet:
Im ersten Schritt werden die Management Interfaces, NTP und Hostnamen eingerichtet.
leaf01
1
2
3
4
5
6
7
8
9
10
11
net add interface eth0 alias MGMT
net add interface eth0 ip address 192.168.100.101/24
net add interface eth0 vrf mgmt
net add time zone Europe/Berlin
net add time ntp server 192.168.100.1 iburst
net add time ntp source eth0
net add dns nameserver ipv4 192.168.100.1
net add hostname leaf01
leaf02
1
2
3
4
5
6
7
8
9
10
11
net add time zone Europe/Berlin
net add time ntp server 192.168.100.1 iburst
net add time ntp source eth0
net add interface eth0 alias MGMT
net add interface eth0 ip address 192.168.100.102/24
net add interface eth0 vrf mgmt
net add dns nameserver ipv4 192.168.100.1
net add hostname leaf02
spine01
1
2
3
4
5
6
7
8
9
10
11
net add time zone Europe/Berlin
net add time ntp server 192.168.100.1 iburst
net add time ntp source eth0
net add interface eth0 alias MGMT
net add interface eth0 ip address 192.168.100.103/24
net add interface eth0 vrf mgmt
net add dns nameserver ipv4 192.168.100.1
net add hostname spine01
spine02
1
2
3
4
5
6
7
8
9
10
11
net add time zone Europe/Berlin
net add time ntp server 192.168.100.1 iburst
net add time ntp source eth0
net add interface eth0 alias MGMT
net add interface eth0 ip address 192.168.100.104/24
net add interface eth0 vrf mgmt
net add dns nameserver ipv4 192.168.100.1
net add hostname spine02
Im zweiten Schritt werden die Interfaces zu den Spines bzw. Leafes eingerichtet.
leaf01
1
2
3
4
5
6
net add interface swp1 alias Uplink--Spine01
net add interface swp1 ip address 10.23.0.2/30
net add interface swp1 mtu 9216
net add interface swp2 alias Uplink--Spine02
net add interface swp2 ip address 10.23.0.10/30
net add interface swp2 mtu 9216
leaf02
1
2
3
4
5
6
7
net add interface swp1 alias Uplink--Spine01
net add interface swp1 ip address 10.23.0.6/30
net add interface swp1 mtu 9216
net add interface swp2 alias Uplink--Spine02
net add interface swp2 ip address 10.23.0.14/30
net add interface swp2 mtu 9216
spine01
1
2
3
4
5
6
7
net add interface swp1 alias Uplink--Leaf01
net add interface swp1 ip address 10.23.0.1/30
net add interface swp1 mtu 9216
net add interface swp2 alias Uplink--Leaf02
net add interface swp2 ip address 10.23.0.5/30
net add interface swp2 mtu 9216
spine02
1
2
3
4
5
6
7
net add interface swp1 alias Uplink--Leaf01
net add interface swp1 ip address 10.23.0.9/30
net add interface swp2 mtu 9216
net add interface swp2 alias Uplink--Leaf02
net add interface swp2 ip address 10.23.0.13/30
net add interface swp2 mtu 9216
Jetzt wird BGP konfiguriert. Die Spines besitzen die gleiche AS-Nummer, hier wird ECMP verwendet, die Leafes haben jeweils eine eigene AS Nummer. Mittels sudo vtysh kann der Zebra-Daemon ebenfalls konfiguriert werden, hier beziehe ich mich jedoch auf die NCLU.
leaf01
1
2
3
4
5
net add bgp autonomous-system 65001
net add bgp router-id 172.16.0.1
net add loopback lo ip address 172.16.0.1/32
net add bgp neighbor 10.23.0.1 remote-as 65100
net add bgp neighbor 10.23.0.9 remote-as 65100
leaf02
1
2
3
4
5
net add bgp autonomous-system 65002
net add bgp router-id 172.16.0.2
net add loopback lo ip address 172.16.0.2/32
net add bgp neighbor 10.23.0.5 remote-as 65100
net add bgp neighbor 10.23.0.13 remote-as 65100
spine01
1
2
3
4
5
net add bgp autonomous-system 65100
net add bgp router-id 172.16.1.1
net add loopback lo ip address 172.16.1.1/32
net add bgp neighbor 10.23.0.2 remote-as 65001
net add bgp neighbor 10.23.0.6 remote-as 65002
spine02
1
2
3
4
5
net add bgp autonomous-system 65100
net add bgp router-id 172.16.1.2
net add loopback lo ip address 172.16.1.2/32
net add bgp neighbor 10.23.0.10 remote-as 65001
net add bgp neighbor 10.23.0.14 remote-as 65002
Sollten sich beim konfigurieren keine Fehler eingeschlichen haben, der Befehl
net commit
ausgeführt wurde, dann sollten jetzt BGP-Sessions gebildet werden.
1
2
3
4
5
6
7
8
9
10
11
12
13
cumulus@spine01:mgmt-vrf:~$ net show bgp summary
show bgp ipv4 unicast summary
=============================
BGP router identifier 172.16.1.1, local AS number 65100 vrf-id 0
BGP table version 43
RIB entries 15, using 2280 bytes of memory
Peers 2, using 39 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
leaf01(10.23.0.2) 4 65001 6205 6204 0 0 0 03:19:40 4
leaf02(10.23.0.6) 4 65002 6198 6195 0 0 0 03:19:29 4
Total number of neighbors 2
Auf den Leaf-Switches wird ein Netz (VLAN 10) für die beiden Clients eingerichtet. Beide Clients sind am Port “swp3” angeschlossen.
leaf01
1
2
3
4
5
6
7
8
net add bridge bridge vids 10
net add bridge bridge vlan-aware
net add bridge bridge ports swp3
net add interface swp3 bridge access 10
net add vlan 10 ip address 10.10.10.1/24
net add vlan 10 vlan-id 10
net add vlan 10 vlan-raw-device bridge
leaf02
1
2
3
4
5
6
7
8
net add bridge bridge vids 10
net add bridge bridge vlan-aware
net add bridge bridge ports swp3
net add interface swp3 bridge access 10
net add vlan 10 ip address 10.10.10.1/24
net add vlan 10 vlan-id 10
net add vlan 10 vlan-raw-device bridge
Damit sollte der Ping auf den jeweils angeschlossene Client funktionieren.
1
2
3
4
5
6
7
8
cumulus@leaf01:mgmt-vrf:~$ ping 10.10.10.2 -c 2
PING 10.10.10.2 (10.10.10.2) 56(84) bytes of data.
64 bytes from 10.10.10.2: icmp_seq=1 ttl=64 time=0.272 ms
64 bytes from 10.10.10.2: icmp_seq=2 ttl=64 time=0.297 ms
--- 10.10.10.2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.272/0.284/0.297/0.021 ms
Im nächsten Schritt wird auf den Leaf-Switches VxLAN eingerichtet. VxLAN benötigt ebenfalls eine ID, der Einfachheit wird eine 10 vorangestellt. Hier also 1010. Das VxLAN nennt sich “vni-1010”. Die lokale Tunnel-IP ist die jeweilige Loopback-IP des Switches.
leaf01
1
2
3
4
5
6
7
8
9
10
net add vxlan vni-1010 vxlan id 1010
net add bridge bridge ports vni-1010
net add vxlan vni-1010 bridge access 10
net add vxlan vni-1010 bridge learning off
net add vxlan vni-1010 mtu 9152
net add vxlan vni-1010 stp bpduguard
net add vxlan vni-1010 stp portbpdufilter
net add vxlan vni-1010 vxlan local-tunnelip 172.16.0.1
leaf02
1
2
3
4
5
6
7
8
9
10
net add vxlan vni-1010 vxlan id 1010
net add bridge bridge ports vni-1010
net add vxlan vni-1010 bridge access 10
net add vxlan vni-1010 bridge learning off
net add vxlan vni-1010 mtu 9152
net add vxlan vni-1010 stp bpduguard
net add vxlan vni-1010 stp portbpdufilter
net add vxlan vni-1010 vxlan local-tunnelip 172.16.0.2
Damit das Netz übertragen wird, muss der BGP-Prozess erweitert werden. Es wird jetzt die BGP-Erweiterung “EVPN” konfiguriert. Zeitgleich werden alle verbundenen Netze redistributiert, sowie die Loopback-IP annonciert. Der Aufbau des RD setzt sich aus Loopback-IP:VxLAN-ID zusammen, beim Leaf01 folglich als 172.16.0.1:1010.
leaf01
1
2
3
4
5
6
7
8
9
net add bgp ipv4 unicast network 172.16.0.1/32
net add bgp ipv4 unicast redistribute connected
net add bgp evpn vni 1010 route-target import 65001:1010
net add bgp l2vpn evpn neighbor 10.23.0.1 activate
net add bgp l2vpn evpn neighbor 10.23.0.9 activate
net add bgp l2vpn evpn advertise-all-vni
net add bgp l2vpn evpn vni 1010 rd 172.16.0.1:1010
net commit
leaf02
1
2
3
4
5
6
7
8
9
10
net add bgp ipv4 unicast network 172.16.0.0/30
net add bgp ipv4 unicast redistribute connected
net add bgp evpn vni 1010 route-target import 65002:1010
net add bgp l2vpn evpn neighbor 10.23.0.5 activate
net add bgp l2vpn evpn neighbor 10.23.0.13 activate
net add bgp l2vpn evpn advertise-all-vni
net add bgp l2vpn evpn vni 1010 rd 172.16.0.2:1010
net commit
spine01
1
2
3
4
net add bgp ipv4 unicast network 172.16.1.1/32
net add bgp l2vpn evpn neighbor 10.23.0.2 activate
net add bgp l2vpn evpn neighbor 10.23.0.6 activate
net commit
spine02
1
2
3
4
net add bgp ipv4 unicast network 172.16.1.2/32
net add bgp l2vpn evpn neighbor 10.23.0.10 activate
net add bgp l2vpn evpn neighbor 10.23.0.14 activate
net commit
Wenn die Konfiguration richtig ist, sollten jetzt unter BGP L2VPN ebenfalls BGP-Sessions aufgebaut werden.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
cumulus@spine01:mgmt-vrf:~$ net show bgp summary
show bgp l2vpn evpn summary
===========================
BGP router identifier 172.16.1.1, local AS number 65100 vrf-id 0
BGP table version 0
RIB entries 3, using 456 bytes of memory
Peers 2, using 39 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
leaf01(10.23.0.2) 4 65001 6697 6695 0 0 0 03:44:15 4
leaf02(10.23.0.6) 4 65002 6690 6687 0 0 0 03:44:04 4
Total number of neighbors 2
Die EVPN Zusammenfassung sollte in etwa so aussehen:
1
2
3
4
5
6
7
8
9
10
cumulus@leaf01:mgmt-vrf:~$ net show bgp evpn vni
Advertise Gateway Macip: Disabled
Advertise SVI Macip: Disabled
Advertise All VNI flag: Enabled
BUM flooding: Head-end replication
Number of L2 VNIs: 1
Number of L3 VNIs: 0
Flags: * - Kernel
VNI Type RD Import RT Export RT Tenant VRF
* 1010 L2 172.16.0.1:1010 65001:1010 65001:1010 default
Ein ping auf Client-2 sollte jetzt funktionieren:
1
2
3
4
5
6
7
8
9
server01:~# ping 10.10.10.3 -c 2
PING 10.10.10.3 (10.10.10.3): 56 data bytes
64 bytes from 10.10.10.3: seq=0 ttl=64 time=1.545 ms
64 bytes from 10.10.10.3: seq=1 ttl=64 time=1.415 ms
--- 10.10.10.3 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1.415/1.480/1.545 ms
server01:~#
Da es sich um ein Layer2 Tunnel handelt sollten auch MAC-Adressen auftauchen:
1
2
3
4
5
6
7
8
9
10
11
12
13
cumulus@leaf01:mgmt-vrf:~$ net show evpn mac vni 1010
Number of MACs (local and remote) known for this VNI: 3
MAC Type Intf/Remote VTEP VLAN
00:0c:29:1a:1f:79 local swp3 10
00:0c:29:8b:2b:2d local vlan10 10
00:0c:29:b1:e8:4b remote 172.16.0.2
cumulus@leaf02:mgmt-vrf:~$ net show evpn mac vni all
VNI 1010 #MACs (local and remote) 3
MAC Type Intf/Remote VTEP VLAN
00:0c:29:1a:1f:79 remote 172.16.0.1
00:0c:29:cf:6b:a4 local vlan10 10
00:0c:29:b1:e8:4b local swp3 10
Auf Leaf01 sollte die Routing-Tabelle so aussehen:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
cumulus@leaf01:mgmt-vrf:~$ net show route bgp
RIB entry for bgp
=================
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
B>* 10.23.0.4/30 [20/0] via 10.23.0.1, swp1, 04:00:04
* via 10.23.0.9, swp2, 04:00:04
B>* 10.23.0.12/30 [20/0] via 10.23.0.1, swp1, 04:00:04
* via 10.23.0.9, swp2, 04:00:04
B>* 172.16.0.2/32 [20/0] via 10.23.0.1, swp1, 04:00:04
* via 10.23.0.9, swp2, 04:00:04
B>* 172.16.1.1/32 [20/0] via 10.23.0.1, swp1, 04:00:15
B>* 172.16.1.2/32 [20/0] via 10.23.0.9, swp2, 04:00:15
Auf Spine01 sollte die Routingtabelle so aussehen:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
cumulus@spine01:mgmt-vrf:~$ net show route bgp
RIB entry for bgp
=================
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
B>* 10.10.10.0/24 [20/0] via 10.23.0.2, swp1, 04:02:14
B 10.23.0.0/30 [20/0] via 10.23.0.2 inactive, 04:02:14
B 10.23.0.4/30 [20/0] via 10.23.0.6 inactive, 04:02:03
B>* 10.23.0.8/30 [20/0] via 10.23.0.2, swp1, 04:02:14
B>* 10.23.0.12/30 [20/0] via 10.23.0.6, swp2, 04:02:03
B>* 172.16.0.1/32 [20/0] via 10.23.0.2, swp1, 04:02:14
B>* 172.16.0.2/32 [20/0] via 10.23.0.6, swp2, 04:02:03
Die BGP-Konfiguration kann ebenfalls über vtysh eingerichtet werden. Die Konfiguration sieht in Cisco-Syntax folgendermaßen aus:
leaf01
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
router bgp 65001
bgp router-id 172.16.0.1
neighbor 10.23.0.1 remote-as 65100
neighbor 10.23.0.9 remote-as 65100
!
address-family ipv4 unicast
network 172.16.0.1/32
redistribute connected
exit-address-family
!
address-family l2vpn evpn
neighbor 10.23.0.1 activate
neighbor 10.23.0.9 activate
vni 1010
rd 172.16.0.1:1010
exit-vni
advertise-all-vni
exit-address-family
leaf02
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
router bgp 65002
bgp router-id 172.16.0.2
neighbor 10.23.0.5 remote-as 65100
neighbor 10.23.0.13 remote-as 65100
!
address-family ipv4 unicast
network 172.16.0.0/30
redistribute connected
exit-address-family
!
address-family l2vpn evpn
neighbor 10.23.0.5 activate
neighbor 10.23.0.13 activate
vni 1010
rd 172.16.0.2:1010
exit-vni
advertise-all-vni
exit-address-family
spine01
1
2
3
4
5
6
7
8
9
10
11
12
13
router bgp 65100
bgp router-id 172.16.1.1
neighbor 10.23.0.2 remote-as 65001
neighbor 10.23.0.6 remote-as 65002
!
address-family ipv4 unicast
network 172.16.1.1/32
exit-address-family
!
address-family l2vpn evpn
neighbor 10.23.0.2 activate
neighbor 10.23.0.6 activate
exit-address-family
spine02
1
2
3
4
5
6
7
8
9
10
11
12
13
router bgp 65100
bgp router-id 172.16.1.2
neighbor 10.23.0.10 remote-as 65001
neighbor 10.23.0.14 remote-as 65002
!
address-family ipv4 unicast
network 172.16.1.2/32
exit-address-family
!
address-family l2vpn evpn
neighbor 10.23.0.10 activate
neighbor 10.23.0.14 activate
exit-address-family
FRR ist erst ab Version 5.0.2 in der Lage EVPNs zu routen.