Configuring RPS (Receive Packet Steering)

Receive Packet Steering (RPS) is a software implementation of RSS. Useful for example when there are fewer interrupts on the network interface than the processor cores.

I highly recommend setting up RSS as I described in the article:
Distribution of network card interrupts across processor cores

As a last resort, you can configure RPS, for example, if the traffic of the network interface only hits the first interrupt, and the rest are not used (empty) or simply has only one interrupt and, accordingly, loads only one CPU core.
For example, in order for the queue to be processed on any of 8 cores, you can specify ff (by default 0):

echo "ff" > /sys/class/net/ens1f0/queues/rx-0/rps_cpus
echo "ff" > /sys/class/net/ens1f0/queues/rx-1/rps_cpus
echo "ff" > /sys/class/net/ens1f0/queues/rx-2/rps_cpus
echo "ff" > /sys/class/net/ens1f0/queues/rx-3/rps_cpus
echo "ff" > /sys/class/net/ens1f0/queues/rx-4/rps_cpus
echo "ff" > /sys/class/net/ens1f0/queues/rx-5/rps_cpus
echo "ff" > /sys/class/net/ens1f0/queues/rx-6/rps_cpus
echo "ff" > /sys/class/net/ens1f0/queues/rx-7/rps_cpus
 
echo "ff" > /sys/class/net/ens1f1/queues/rx-0/rps_cpus
echo "ff" > /sys/class/net/ens1f1/queues/rx-1/rps_cpus
echo "ff" > /sys/class/net/ens1f1/queues/rx-2/rps_cpus
echo "ff" > /sys/class/net/ens1f1/queues/rx-3/rps_cpus
echo "ff" > /sys/class/net/ens1f1/queues/rx-4/rps_cpus
echo "ff" > /sys/class/net/ens1f1/queues/rx-5/rps_cpus
echo "ff" > /sys/class/net/ens1f1/queues/rx-6/rps_cpus
echo "ff" > /sys/class/net/ens1f1/queues/rx-7/rps_cpus

Let’s check:

cat /sys/class/net/ens1f0/queues/*/rps_cpus
cat /sys/class/net/ens1f1/queues/*/rps_cpus

To specify f on any of the 0-3 cores, for example:

echo "f" | tee /sys/class/net/eth0/queues/rx-[0-3]/rps_cpus

Another example, once I came across a network adapter HP 544+QSFP (764284-B21) Mellanox ConnectX-3 Pro EN, the number of interruptions of which can be specified in the power of two, that is, 1,2,4,8,16,32,64, etc., and in my case there were 14 processor cores, in this case you can specify 8 interrupts on the network interface and configure RSS for them (from core 0 to 7), and configure RPS for the rest of the cores (core 8 to 13), that is, the number 00000000111111 is converted into hexadecimal and we get 3F00.

For example, F is a hexadecimal value that equals 1111. If you need to distribute interrupts to processor cores not from 0 to 3, but from 1 to 3, then it will be 0111 in binary, but in hex 7.

I will give you some more examples to understand:

2 cores (11) = 3
3 cores (111) = 7
4 cores (1111) = F
5 cores (11111) = 1F
6 cores (111111) = 3F
7 cores (1111111) = 7F
8 cores (11111111) = FF
9 cores (111111111) = 1FF
10 cores (1111111111) = 3FF
11 cores (11111111111) = 7FF
12 cores (111111111111) = FFF
13 cores (1111111111111) = 1FFF
14 cores (11111111111111) = 3FFF

15 cores (111111111111111) = 7FFF
16 cores (1111111111111111) = FFFF
17 cores (11111111111111111) = 1FFFF
...

To prevent changes from being reset after a system restart, add commands to /etc/rc.local

You can also use online converters to calculate the mask, for example:
bitsum.com/tools/cpu-affinity-calculator/
rapidtables.com/convert/number/binary-to-hex.html

See also my articles:
Solution: No /etc/rc.local file on Ubuntu 18
How to find out on which NUMA node network interfaces
Configuring the Network in Linux

Leave a comment

Leave a Reply