In this article I will give an example of changing gc_thresh on Linux, this parameter usually needs to be increased on highly loaded access servers.
For example, in “gc_thresh1” the minimum number of stored ARP records is indicated which is not cleared, in “gc_thresh2” the amount after which the records begin to be cleaned after 5 seconds, in “gc_thresh3” the amount upon reaching which the records begin to be cleared immediately. If, for example, the server is used for NAT and with a large number of network nodes, and the “gc_thresh” values that you have specified are not enough, then in the /var/log/kern.log logs you will see the error “arp_cache: neighbor table overflow!”.
First, let’s see the current values:
cat /proc/sys/net/ipv4/neigh/default/gc_thresh1 cat /proc/sys/net/ipv4/neigh/default/gc_thresh2 cat /proc/sys/net/ipv4/neigh/default/gc_thresh3
Or so:
sysctl -a |grep "neigh.default.gc_thresh"
Let’s see the number of arp entries and the dynamics of their changes:
arp -an | wc -l
arp -an | grep incom | wc -l
arp -an | less
for i in {1..10}; do cat /proc/net/arp | wc -l && sleep 5; done
ip n | wc -l
ip n | grep STALE | wc -l
ip n | grep REACHABLE
ip n | less
ip -s neighbor show
For example, for a server that served more than 6,000 clients, I indicated:
sysctl -w net.ipv4.neigh.default.gc_thresh3=24456 sysctl -w net.ipv4.neigh.default.gc_thresh2=12228 sysctl -w net.ipv4.neigh.default.gc_thresh1=8192
To prevent the values from resetting after restarting the operating system, specify them in the /etc/sysctl.conf file:
net.ipv4.neigh.default.gc_thresh1=8192 net.ipv4.neigh.default.gc_thresh2=12228 net.ipv4.neigh.default.gc_thresh3=24456
Similarly, we specify for IPv6:
net.ipv6.neigh.default.gc_thresh1=8192 net.ipv6.neigh.default.gc_thresh2=12228 net.ipv6.neigh.default.gc_thresh3=24456
If you have not changed them before, then we apply what is in the /etc/sysctl.conf file with the command below:
sysctl -p
I note that it is not correct to specify large values for gc_thresh1, you need to consider how many clients your server serves, since if you specify a large value for gc_thresh1, this may mean that the number of arp entries above this value will never be deleted from the cache, and when If the client supposes the mac address or ip address changes, there may be failures with its availability, as the cache will have old entries.
Obsolete arp records are marked from REACHABLE in STALE and later cleared, non-existing records are marked as INCOMPLETE and also cleared.
Let’s look at the timers: checking stale arp entries (gc_stale_time), cleaning them (gc_interval) and marking them as stale (base_reachable_time_ms):
cat /proc/sys/net/ipv4/neigh/all/gc_stale_time cat /proc/sys/net/ipv4/neigh/default/gc_interval cat /proc/sys/net/ipv4/neigh/default/base_reachable_time_ms
If desired, you can increase:
echo 120 > /proc/sys/net/ipv4/neigh/default/gc_stale_time echo 60 > /proc/sys/net/ipv4/neigh/default/gc_interval
If we changed them, then we also specify in /etc/sysctl.conf (I show the default values for Ubuntu Server 16.04):
net.ipv4.neigh.default.gc_stale_time = 60 net.ipv4.neigh.default.gc_interval = 30 net.ipv4.neigh.default.base_reachable_time_ms=30000
See also my articles:
Tuning nf_conntrack
How to fix the error “nf_conntrack: table full, dropping package”