I will give an example of balancing only incoming traffic with two channels using Quagga.
On the test, I will use Ubuntu 16.04.4 LTS and Quagga 0.99.24.1, the network interface ens1f0 for the second provider with one neighbors and ens2f0 for the first provider with two neighbors, the local network will be connected to ens2f1. Both providers announce “default”.
3.3.3.0/23 this will be my network with white IP addresses.
Network adapters used HP 560SFP+ (TX and RX buffers 4Mb, 10 threads) and HP 562SFP+ (TX and RX buffers 4Mb, more threads), 4Gb/s traffic.
Assigned IP addresses to network interfaces in /etc/network/interfaces:
auto ens1f0 iface ens1f0 inet static address 22.22.22.82 netmask 255.255.255.252 auto ens2f0 iface ens2f0 inet static address 1.1.1.218 netmask 255.255.255.252 auto ens2f0:0 iface ens2f0:0 inet static address 1.1.1.74 netmask 255.255.255.252 auto ens2f1 iface ens2f1 inet static address 10.0.0.1 netmask 255.255.255.0
In /etc/quagga/zebra.conf I got the following configuration:
hostname test password test enable password test log file /var/log/zebra.log ! interface eno1 ipv6 nd suppress-ra ! interface eno2 ipv6 nd suppress-ra ! interface eno3 ipv6 nd suppress-ra ! interface eno4 ipv6 nd suppress-ra ! interface lo ! interface ens1f0 ipv6 nd suppress-ra ! interface ens1f1 ipv6 nd suppress-ra ! interface ens2f0 ipv6 nd suppress-ra ! interface ens2f1 ipv6 nd suppress-ra ! ip forwarding ! ! line vty !
In /etc/quagga/daemons:
zebra=yes bgpd=yes ospfd=no ospf6d=no ripd=no ripngd=no isisd=no babeld=no
And here is my main configuration file /etc/quagga/bgpd.conf (where 12345 is my AS, 11111 and 22222 are AS providers, bgp router-id 1.1.1.218 is one of any IP addresses in the direction of providers):
password test enable password test log file /var/log/quagga/bgpd.log service advanced-vty ! bgp multiple-instance bgp config-type cisco ! router bgp 12345 no synchronization bgp router-id 1.1.1.218 bgp log-neighbor-changes network 3.3.3.0 mask 255.255.254.0 aggregate-address 3.3.3.0 255.255.254.0 summary-only redistribute connected timers bgp 20 60 neighbor 2.2.2.81 remote-as 22222 neighbor 2.2.2.81 description PROVIDER2 neighbor 2.2.2.81 update-source 2.2.2.82 neighbor 2.2.2.81 soft-reconfiguration inbound neighbor 2.2.2.81 route-map PROVIDER2-OUT out neighbor 1.1.1.73 remote-as 11111 neighbor 1.1.1.73 description PROVIDER1-2 neighbor 1.1.1.73 update-source 1.1.1.74 neighbor 1.1.1.73 soft-reconfiguration inbound neighbor 1.1.1.73 route-map PROVIDER1-2-OUT out neighbor 1.1.1.217 remote-as 11111 neighbor 1.1.1.217 description PROVIDER1-1 neighbor 1.1.1.217 update-source 1.1.1.218 neighbor 1.1.1.217 soft-reconfiguration inbound neighbor 1.1.1.217 route-map PROVIDER1-1-OUT out no auto-summary ! access-list 10 permit 127.0.0.1 access-list 10 deny any access-list all permit any ! ip prefix-list DEFAULT-ONLY seq 10 deny 0.0.0.0/0 ge 1 le 31 ip prefix-list DEFAULT-ONLY seq 20 permit 0.0.0.0/0 ip prefix-list DEFAULT-STRIP seq 10 deny 0.0.0.0/0 ip prefix-list DEFAULT-STRIP seq 20 permit 0.0.0.0/0 ge 1 le 31 ip prefix-list TO-PROVIDER1-1 seq 10 permit 3.3.3.0/23 ip prefix-list TO-PROVIDER1-1 seq 100 deny 0.0.0.0/0 le 32 ip prefix-list TO-PROVIDER1-2 seq 10 permit 3.3.3.0/23 ip prefix-list TO-PROVIDER1-2 seq 100 deny 0.0.0.0/0 le 32 ip prefix-list TO-PROVIDER2 seq 10 permit 3.3.3.0/23 ip prefix-list TO-PROVIDER2 seq 100 deny 0.0.0.0/0 le 32 ! route-map PROVIDER1-1-IN permit 10 match ip address prefix-list all ! route-map PROVIDER1-2-IN permit 10 match ip address prefix-list all ! route-map PROVIDER2-IN permit 10 match ip address prefix-list all ! route-map PROVIDER1-1-OUT permit 10 description MY <-> PROVIDER1-1 match ip address prefix-list TO-PROVIDER1-1 ! route-map PROVIDER1-2-OUT permit 10 description MY <-> PROVIDER1-2 match ip address prefix-list TO-PROVIDER1-2 set local-preference 50 ! route-map PROVIDER2-OUT permit 10 description MY <-> PROVIDER2 match ip address prefix-list TO-PROVIDER2 set as-path prepend 12345 12345 ! line vty access-class 10 no login !
Since I prefer to send outgoing traffic through PROVIDER1-1-OUT, I indicated “set local-preference 50” for PROVIDER1-2-OUT, and for PROVIDER2-OUT and PROVIDER1-1-OUT did not specify, unless I specify “local-preference” defaults to 100 (100 is better and higher priority than 50), well, outgoing traffic will still go through only one provider.
To balance incoming traffic, I specified “set as-path prepend 12345 12345” for PROVIDER2, thereby increasing the path to it, since in my case most of the traffic went through it. You can increase the path even more, for example:
set as-path prepend 12345 12345 12345 12345 12345 12345
The more often we specify our AS, the longer the path and less traffic will go, in general, you need to play around with the “as-path prepend”, I note that after changing it, you have to wait, as I sometimes had the traffic finally aligned after a couple of hours.
You can change the “as-path prepend” on the fly without rebooting quagga, to do this, connect to it and switch to configuration mode:
telnet localhost 2605 configure terminal
Изменим “as-path prepend”:
route-map PROVIDER2-OUT permit 10 set as-path prepend 12345 12345 12345
Exit configuration mode and apply the changes:
exit exit clear ip bgp 2.2.2.81 soft out
When finished, save the configuration and exit the console:
write exit
This completes the setup, but once I ran into a problem, as one of the administrators assigned IP for NAT on the network interface ens2f0, which looks in the direction of the first provider and enabled rp_filter, in this case traffic balancing did not work.
To fix the problem, I looked at the current values of rp_filter (was 1):
cat /proc/sys/net/ipv4/conf/all/rp_filter cat /proc/sys/net/ipv4/conf/default/rp_filter cat /proc/sys/net/ipv4/conf/ens1f0/rp_filter cat /proc/sys/net/ipv4/conf/ens1f1/rp_filter cat /proc/sys/net/ipv4/conf/ens2f0/rp_filter cat /proc/sys/net/ipv4/conf/ens2f1/rp_filter
And turned it off:
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter echo 0 > /proc/sys/net/ipv4/conf/default/rp_filter echo 0 > /proc/sys/net/ipv4/conf/ens1f0/rp_filter echo 0 > /proc/sys/net/ipv4/conf/ens1f1/rp_filter echo 0 > /proc/sys/net/ipv4/conf/ens2f0/rp_filter echo 0 > /proc/sys/net/ipv4/conf/ens2f1/rp_filter
After that, balancing started working, it will also work when rp_filter=2.
So that after restarting the system, the value of rp_filter remains the way we need, add to /etc/sysctl.conf:
net.ipv4.conf.default.rp_filter=0 net.ipv4.conf.all.rp_filter=0
After setting up, let’s see the list of neighbors and their status:
sh ip bgp neighbors
Let’s see which routes are accepted and announced to neighbors:
sh ip bgp sh ip bgp neighbors 2.2.2.81 advertised-routes sh ip bgp neighbors 2.2.2.81 received-routes sh ip bgp neighbors 1.1.1.73 advertised-routes sh ip bgp neighbors 1.1.1.73 received-routes sh ip bgp neighbors 1.1.1.217 advertised-routes sh ip bgp neighbors 1.1.1.217 received-routes
To test redundancy, you can turn off one of the interfaces:
sudo ifdown ens1f0 sudo ifup ens1f0
If NAT is also configured on the server, we will wrap the network for NAT (in this case, it is not necessary to assign IP on the interfaces):
telnet localhost 2601 enable configure terminal ip route 3.3.3.128/25 Null0 254
You can restart quagga like this:
sudo /etc/init.d/quagga restart
See also my article:
Setting up BGP in Quagga