Tuning The Linux Connection Tracking System

.

Introduction

trackerThe connection tracking entry (conntrack entry) looks like this:

udp 17 170 src=192.168.1.2 dst=192.168.1.5 sport=137 dport=1025 src=192.168.1.5 dst=192.168.1.2 sport=1025 dport=137 [ASSURED] use=1

It contains two elements the original direction (the red) and the reply direction (the blue).

To display all table’s entries, read  “/proc/net/nf_conntrack”.

The conntrack entry is stored into two separate nodes (one for each direction) in different linked lists. Each linked list is called  bucket. The bucket is an element in a hash table. The hash value is calculated based on the received packet and used as index in the hash table. Iteration is done over the linked list of nodes to find the wanted node.

Long list is not recommended (iteration cost): The cost depends on the length of the list and the position of the wanted conntrack node.

Long hash table is recommended (constant time): The cost is the hash calculation.

Linked List Size (Bucket Size)= Maximum Number of nodes / Hash Table Size (Number of Buckets)

The hash table is stored in the kernel memory. We can tune the size of the bucket and the maximum number of nodes. The required memory = conntrack node’s memory size * 2* simultaneous connections your system aim to handle. Example: 304 bytes per conntrack and 1M connections requires 304*2 MB.

It is not recommended to set so big values if you have less than 1G RAM.

Tuning the Values

If your server has a lot of connections to be handled and the conntrack table is full, you will get this error “nf_conntrack: table full, dropping packet“. This will limit the number of simultaneous connections your system can handle.

To get the maximum number of nodes:

# /sbin/sysctl -a|grep -i nf_conntrack_max
net.nf_conntrack_max = 65536

To get the hash table size (number of buckets):

# /sbin/sysctl -a|grep -i nf_conntrack_buckets
net.netfilter.nf_conntrack_buckets = 16384

The bucket size (linked list length)= 4 (65536/16384).

To temporarily change the value of hash table size to 2*16384=32768:

# echo 32768 > /sys/module/nf_conntrack/parameters/hashsize

To permanently change the value:

# echo “net.netfilter.nf_conntrack_buckets = 32768” >> /etc/sysctl.conf
# /sbin/sysct -p

The same way for “nf_conntrack_max”:

Temporarily: # echo 131072 > /proc/sys/net/nf_conntrack_max

To permanently:

# echo “net.netfilter.nf_conntrack_max = 131072” >> /etc/sysctl.conf
# /sbin/sysct -p

This requires 38 MB memory.

“nf_conntrack” Other Values

You can change these values also in the same way:

# /sbin/sysctl -a|grep -i nf_conntrack
net.netfilter.nf_conntrack_acct = 0
net.netfilter.nf_conntrack_buckets = 16384
net.netfilter.nf_conntrack_checksum = 1
net.netfilter.nf_conntrack_count = 817
net.netfilter.nf_conntrack_events = 1
net.netfilter.nf_conntrack_expect_max = 256
net.netfilter.nf_conntrack_generic_timeout = 600
net.netfilter.nf_conntrack_helper = 1
net.netfilter.nf_conntrack_icmp_timeout = 30
net.netfilter.nf_conntrack_log_invalid = 0
net.netfilter.nf_conntrack_max = 65536
net.netfilter.nf_conntrack_tcp_be_liberal = 0
net.netfilter.nf_conntrack_tcp_loose = 1
net.netfilter.nf_conntrack_tcp_max_retrans = 3
net.netfilter.nf_conntrack_tcp_timeout_close = 10
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60
net.netfilter.nf_conntrack_tcp_timeout_established = 432000
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_last_ack = 30
net.netfilter.nf_conntrack_tcp_timeout_max_retrans = 300
net.netfilter.nf_conntrack_tcp_timeout_syn_recv = 60
net.netfilter.nf_conntrack_tcp_timeout_syn_sent = 120
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_unacknowledged = 300
net.netfilter.nf_conntrack_timestamp = 0
net.netfilter.nf_conntrack_udp_timeout = 30
net.netfilter.nf_conntrack_udp_timeout_stream = 180
net.nf_conntrack_max = 65536

Last word: Disable “nf_conntrack” if it is not necessary.


 

Leave a comment