This is Part 3 of “Linux Tuning For SIP Routers” topic. In Part 1 i have talked about Interrupts and IRQ Tuning. In Part 2 i have talked about File System Tuning, Journaling File System, and Swappiness . In this part i will talk about OOM killer, Private Momory, and Shared Memory Tuning.
Out-Of-Memory (OOM) Killer
# echo -15 /proc/PID/oom_adj where the PID is the process ID and oom_adj represents how likely is the process to be killed. oom_adj takes values between -16 to +15. Setting oom_adj to -17 exempts the process entirely from the OOM killer.
To know how likely is a process to be killed, view the content of the file “oom_score”:
# cat /proc/PID/oom_score
Private Memory (Per Process Memory) Settings
Check if there is any memory limitation for a user process. The command “ulimit -a” display the limitations on all resources not only the memory. Use the option -m to display the max memory size:
# ulimit -m
The output: unlimited
The command “ulimit” is built into the bash shell and used for read and write the limitations on the resources available to the shell and the processes started by it. The system must allow such control on the resources.
What remained is configuring the private memory of SIP router’s process. See the example below:
Example: Tuning the Private Memory of OpenSIPS Process
OpenSIPS private memory (file: mem/mem.h) is a memory dedicated to each process and used for process own business like building buffer for sending SIP message. The default size of private memory used by each OpenSIPS process is 1 MB. Common cases that need large private memory are loading user location contacts and doing NAT pinging. You can set the size of private memory (in megabytes) at startup by using the command parameter -M:
# ./opensips -M 10
Shared Memory Settings
The shared memory is memory shared between processes. The data that are stored in the shared memory can be accessed by many processes. You need to care about shared memory if your SIP router is multi process and the processes/threads from different processes share some data. To see the shared memory settings in your system, type: # ipcs -lm
—— Shared Memory Limits ——–
max number of segments = 4096
max seg size (kbytes) = 32768
max total shared memory (kbytes) = 8388608
min seg size (bytes) = 1
Usually the SIP router has its own memory manager which allocates and frees the memory offered by the operating system between its processes. So tuning the memory involves tuning the memory at the operating system level and at the router level. Linux as an operating system has some parameters that you can configure:
It is the maximum size (in bytes) of a single shared segment that a Linux process can allocate in its virtual address space. The value of this parameter depends on whether you are using 32 bit system or 64 bit system (size of the available address space).
What to do
- Check your system (# uname -a): I am using Fedora Red Hat SMP kernel on 64 bit platform (X86-64) so the maximum size of a shared memory segment theoretically is 2^64bytes. This is correspond to all physical RAM that you have.
- As a test case, I will set the value of the
SHMMAX parameter to “1/4 of physical RAM”. I have 4 GB of RAM so I will set SHMMAX value to 1*1024*1024*1024=1,073,741,824 (bytes):
- Temporarily: echo 1073741824> /proc/sys/kernel/shmmax
- Permanently: Edit the file “/etc/sysctl.conf” and set the entry: kernel.shmmax = 1073741824. This change can be reflected without restart by executing: “/sbin/sysctl -p“.
It is the total amount of shared memory pages that can be used system wide. Hence, SHMALL should always be at least SHMMAX / PAGE_SIZE. Execute “getconf PAGE_SIZE” to get the page size. In my case the page size is 4096 So the SHMALL value will be 1073741824/ 4096 = 262,144.
After configuring what the operating system allocate to your SIP router application, we need to configure the application itself. The following is an example:
Example: Tuning the Shared Memory of OpenSIPS SIP Router
# ./opensips -m 1024 (1GB * 1024= 1024 MB)
- Test the shared memory: You can get some statistics about the shared memory used and managed by OpenSIPS by sending FIFO commands to OpenSIPS. Have a look here on the SHMEM class. Use also system commands like “free” (MemShared or Shmem value) to get information about the memory.
- The size of the recommended shared memory depends on the complexity of the routing script. So do the test many times for different values and choose the best for your system. Don’t forget to tune the value of SHMMAX kernel parameter.
Note: Configure both the private memory (-M parameter) and the shared memory (-m parameter) at OpenSIPS startup. Test for different values and choose the one which suitable to your routing script.
Semaphores are like counters that are used for controlling access by multiple processes to shared resources like shared memories. To see all semaphores settings: # ipcs -ls
—— Semaphore Limits ——–
max number of arrays = 128 # SEMMNI Parameter
max semaphores per array = 250 # SEMMSL Parameter
max semaphores system wide = 32000 # SEMMNS Parameter
max ops per semop call = 32 # SEMOPM Parameter
semaphore max value = 32767
or you can do this: # cat /proc/sys/kernel/sem
The output will be in this order (SEMMSL, SEMMNS, SEMOPM, SEMMNI):
250 32000 32 128
Tuning these values also depends on the complexity of the routing script. More complexity –> More loaded modules (shared libraries) –> more sharing –> more need for tuning the synchronization between processes.
Keep these values AS IS if you don’t have problems with performance. To change these values temporarily:
# echo 250 32000 100 128 > /proc/sys/kernel/sem
# sysctl -w kernel.sem=”250 32000 100 128″
To permanently change these values, Edit the file “/etc/sysctl.conf” and override the settings: kernel.sem=250 32000 100 128
Check The System Limits using “ulimit” System Property
Follow these steps to monitor the resources offered by the system to specific process:
- Get the ID of the proess using # ps -A | grep …
- Execute this: # cat /proc/$ProcessID/limits. Replace the ProcessID with the ID you get using the “ps” above.
The output will look like this:
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 511830 511830 processes
Max open files 1024 1024 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 511830 511830 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
After tuning the memory of the operating system, you need to care about the memory of the SIP application itself. Here is an example (OpenSIPS Memory Debugging) .In the Next Part, I will talk about network tuning to optimize SIP Routers.