Building a Custom Linux kernel for your VOIP System – Part1

.

Introduction

Is your VOIP system based on userspace applications ? What about building kernel based VOIP system ?. This is not new but we are going to go through it here. I am assuming Linux as an operating system. The Linux kernel is about core and modules (micro kernel + loadable modules). You might not touch the core but you might need to build a module that support your VOIP system in the kernel for performance purposes. The new module supposed to do VOIP jobs (filtering, QoS, protection, media relay). There are already some useful kernel modules that can support your system like NATing, IPv6ing, and so on . Use them!

In this article i will just introduce what you need to know to follow this series of articles: how to compile and install a new, fresh and clean Linux kernel and what you need to know to compile and install a new kernel module.

We will not touch the current running kernel but building a new one, install it, boot from it, write a new module, add it to the kernel and test it. You need to be very careful when you work in the kernel.

Compilation and Installation Steps

This is a traditional common way to all Linux distributions to compile and install a Linux kernel:

Download the source:

I will work on the kernel 3.19.5 as an example. For other versions, go to https://kernel.org/pub/linux/kernel and see the list there.

# wget -c https://kernel.org/pub/linux/kernel/v3.x/linux-3.19.5.tar.gz

  Decompress the gz file and go to the directory “linux-3.19.5”:

# tar -xzvf linux-3.19.5.tar.gz

# cd linux-3.19.5

Configure the kernel:

Fresh configuration: traditional way: # make menuconfig or the modern way: # make nconfig

Note: To detect memory leaks in the kernel modules, you can use the tool ‘kmemleak’. To use it, you need to compile the kernel with ‘CONFIG_DEBUG_KMEMLEAK’ option enabled. Enable it as following: make menuconfig –> ‘kernel Hacking’ –> ‘Memory Debugging’ –> ‘Kernel Memory Leak Detector’. Enable this option and do save. You can check this file “/sys/kernel/debug/kmemleak” while testing.

You can use the config file of the current running kernel. In Fedora this file exists as /boot/config-$(KernelName). For example: /boot/config-3.19.5-100.fc20.x86_64. Then update it with ‘make menuconfig’ and save.

Compilation:

Prepare for the compilation: # make mrproper

Compile the kernel: # make

Install the compiled modules:

# make modules_install

The modules will be installed in /lib/modules/3.19.5/

Install the kernel: Now we want to copy and rename the image bzImage (arch/x86_64/boot/bzImage) to /boot + creating the initial RAM disk + creating the map/kernel symbols , and update the grub config file). This done by: # make install

After compiling and installing the kernel, we can boot from it (appear in the GRUB startup List).

Header Files needed to compile a new module:

If you want to build your own kernel module, you need to have the header files.  Check the folder /lib/modules/3.19.5/build/ where 3.19.5 is the new kernel version.

The module is header and source files with Makefile file. When it is built we got .ko file. In the Makefile you need to specify the kernel source and the obj-m.

After building the module, it will be ready to be installed. You need to boot from the new kernel and do ‘modprob $ModuleName’

Again this article is just an introduction to this series. More information is coming next.


OpenSIPS Module Interface – Part 2 (C Development)

.

Introduction

This is Part 2 of the topic “OpenSIPS Module Interface”. In Part 1,I have talked about the definition of the module interface, some fields of its structure, and how to export function from the module to the routing script. So you can call this function in the routing script. Also i have talked about how to export a module parameter. I have talked about the OpenSIPS management interface in the article “OpenSIPS Management Interface (C Development)“. In this article, i will explain a little bit in the module interface and i will show you how to compile a new module.

The definition of the module interface is defined in the header file: “sr_module.h” as following:

struct module_exports {.

……… /* Fields that are explained in the Previous Article are missed here*/

……..  /* Some other fields are missed to be explained in the Next Article */

init_function   init_f;                              /* Initialization function */

destroy_function destroy_f;                /* Function called when the module

.                                                                       should be “destroyed” */
.child_init_function init_child_f;            /* Function called by all processes after the fork */

}

init_f function which will be called at OpenSIPS startup time. In OpenSIPS 1.x which is multi process application, this function will be called when OpenSIPS is just one process (attendant process). For example checking for the database connection can be done here or allocating a memory that will be as shared memory for the forked processes (after forking). You can define your init_f function in the “exports” which is the actual module interface. Lets say “mod_init” is the initialization function in the module interface:

static int mod_init(void){

……

}

The initialization of the exported variables in the routing script will be done before calling the function mod_init. The full OpenSIPS configuration file will be parsed and the own module parameters will be included. For example the parameter “db_url” which will be configured in the routing script:  modparam(“module_name”,”db_url”,”mysql://user:passwd@host/database”) and the “db_url” will be evaluated and can be used in the function “mod_init”. So a connection to the database can be opened in the body of the “mod_init” function.

Also the helper API like shared memory, locking, timer, ..etc. will be available and can be used. Note some resources like new timer processes for the module can only be initialized in the “mod_init” function.

After forking, each process will get a copy of what the OpenSIPS attendant process had before forking (structures/connections).

init_child_f function will be called after OpenSIPS has been forked (It will be called by each process individually). For example if each process will have its separate DB connection, the creation of the DB connection will be in this function.

static int child_init(int rank){

……..

}

The rank parameter allows you to know which process is calling the function. For example it can be called from SIP worker or MI worker (e.g. fifo worker).

destroy_f function will be called when OpenSIPS will shutdown. Here you can do some cleaning of resources(memory, DB connections, and so on). For example you can free the memory for some module-created variables (e.g. pkg_free(Pointer to the Variable)). OpenSIPS 1.X has its own internal memory manager and the function pkg_free is like the function free(pointer to a variable) of glibc standard library. The pointer must not be null otherwise it will crash. Also here in this function, you can save a state that the module have into persistent storage so they can be loaded again at the next OpenSIPS startup. For example in this function, the “dialog” module saves the dialog states in the database.

OpenSIPS 2.X is multi thread application. I will back to it in one or two upcoming articles.

Add Include Directives

You need to add the include statements to the new module code to be able to compile it. For example: the “string.h”, “sr_module.h” which includes the definitions of the structures, “dprint.h” which is used to print stuffs to the syslog or standard error, “mem.h” which is used to allocate/free a memory, and the header file which contains the structures of the new module (e.g. new_module.h).

How to Compile a New OpenSIPS Module

Each OpenSIPS module has its own regular Makefile. It will be run by the master Makefile. The file name is “Makefile” and its content looks like this:

include ../../Makefile.defs
auto_gen=
NAME=new_module.so
LIBS=

include ../../Makefile.modules

The “Makefile.defs” must be included from the OpenSIPS source folder in addition to the name of the shared object “new_module.so” which will be generated and the file “Makefile.modules”. If the module has external libraries dependencies, they should be linked in the module’s Makefile.  Here we have two Makefile variables:

  • DEFS: This is used if the module has additional definitions that you want to pass it to the compiler. Add this line to the module’s Makefile if the module has additional definitions:

DEFS+=-I$(LOCALBASE)/include

OR DEFS+=-I../../../exdef/include

+= is a Makefile operator.

  • LIBS: This is if the  module requires external libraries. So these options:  -llibrary and the -Lpath_to_library will be set to LIBS variable. For example for “cachedb_memcashed module”, add this line to the module’s Makefile:

LIBS=-L$(LOCALBASE)/lib -lmemcashed

To compile the module:

  • The module source files (.c and .h files) and the Makefile should be in the folder “/usr/local/src/opensips_1_11/modules/new_module”
  • From the OpenSIPS ‘s src  folder (“/usr/local/src/opensips_1_11″), execute:

# make include_modules=”new_module” modules OR # make modules modules=modules/new_module

# make install

  • To see if the module is compiled and installed , check having the file “new_module.so” installed in the subdirectory “/usr/local/opensips_1_11/lib64/opensips/modules/“.

Testing

Write log statements (e.g. LM_INFO(….)) in the C code of the new module. Compile it , and restart opensips. Open the log file and search for the name of the module (for example search for “new_module”) to see what is happening to the module “new_module”.


Notes

1- If you’ve got a compilation error (e.g “….. No rule to make target ….”) and you want to clean everything (delete the generated object files and dependencies files (files with extensions “.d”), execute “make proper” and then “make”.

2- If the module depends on external libraries, the module must NOT be compiled by default. To disable this, we edit the file “Makefile.conf.template” which specifies the modules that are not compiles by default.

  • Add a line for your module in “Makefile.conf.template” file in this format:

modulename= Module-Description | module-dependency

For example for the “xcap” module:

xcap= XCAP utility functions for OpenSIPS. | libxml-dev

  • Add the module name to “exclude_modules” list in “Makefile.conf.template” file.
  • After this compile the module as explained in this article.

3- Each OpenSIPS ‘s module has a section in its documentation called “Dependencies -External Libraries” which contains the external libraries that must be installed for this module.


Next

I will continue explaining the module interface.

More Information


OpenSIPS Compilation and Installation Troubleshooting On Fedora

.

OpenSIPS Installation

Here i will install OpenSIPS 1.11. To get more control over the installation process, I will compile the OpenSIPS from the source.

# cd /usr/local/src/

Get the latest release of OpenSIPS source from OpenSIPS’s GitHub repository:

Go to the place where “GitHub.com” is remotely storing OpenSIPS and get the OpenSIPS ‘s repo HTTP-address https://github.com/OpenSIPS/opensips.git“. Then clone the repo.

# git clone https://github.com/OpenSIPS/opensips.git -b 1.11 opensips_1_11

# cd opensips_1_11

Note: The option -b is used to choose the branch “-b 1.11”

Install the prerequisites needed to configure the compilation:

  • Flex and Bison needed for parsing of configuration file.
  • ncurses-libs, ncurses-devel, and ncurses-static for grafical display (text-based user interfaces).

Some prerequisites must be installed for GCC like Flex, Bison, TCl and others. If you could not compile, check what is missed in your system. Have a look here.

Open the configuration interface:

# make menuconfig

If you couldn’t open the interface and have an error like “Make error *255*“, this means  needed prerequisites are not all installed. You could also need  to try these:

  • Remove GCC and install it again:

# yum remove gcc

# yum install gcc

If you are patient, try to do the same thing with some broken installed prerequisites.

  • You could need to change the Makefile (In my case: “/usr/local/src/opensips_1_11/menuconfig/Makefile“). For example adding include (-I) and library (-L) paths.

all: $(MENUCONFIG_FILES)
$(CC) -o $(EXEC_NAME) $(CFLAGS) $^ -I/usr/include -L/usr/lib -lncurses

  • Still have a problem, do system restart.

The text-based user interface to configure and compile will look like this:

Here we can do the following

  • Configure some compile options used at OpenSIPS compilation time.
  • Compile and install OpenSIPS.
  • Cleanup OpenSIPS sources.
  • Generate OpenSIPS script
  • Exit and save all changes.

To get the features you want, go to the first menu and configure the compilation. You can select the modules you want to compile (“Configure Compile Options” –> “Configure Excluded Modules”). The selected modules here (marked with star *) will be included in the compilation of the OpenSIPS. Also you can configure the installation directories (“Configure Compile Options” –> “Configure Install Prefix”). It can be for example “/usr/local/opensips_1_11/”. Note: press q letter or left-arrow to go back to the previous screen.

After the configuration press on “Save Changes” to save the changes you did. After this you will get a message of the required libraries which must be installed according to the modules included in the configuration of the compilation (module dependencies). Go ahead and download them before compilation. Put the graphical interface in the background (press control-Z) or open a new terminal so you can see the names of the required libraries from the graphical interface and install them in the new terminal.

For example the module “db_mysql” needs mysql-devel (mysql client development library) to be installed.

Back to the interface and select “Compile and Install OpenSIPS” , then the compilation will start. It takes few minutes.

After the installation is completed, we get OpenSIPS installed as it is configured. In my case, it will be installed in “/usr/local/opensips_1_11/”. This directory contains these:

In “etc” we have the OpenSIPS configuration files.

In “lib” we have OpenSIPS modules

In “sbin” we have the executable files.

In “share” we have the documentation.

Create OpenSIPS Mysql Database

Install your database server. As an example install MySQL database server.

Now we are in “/usr/local/opensips_1_11/”

  • Configure the database parameters: Edit the configuration file “etc/opensips/opensipsctlrc”. This file contains the configuration of the database connection. # vim opensipsctlrc. Adjust the parameters like the name of the database, the read/write user that the opensips will be using, password, the database root user, and others.
  • Create the OpenSIPS database.Now back to “/usr/local/opensips_1_11/”. In the sub-directory /sbin we have opensipsdbctl tool. Use it to create the database. Here is the command and its output:

# ./opensipsdbctl create  

        MySQL password for root:

INFO: test server charset

INFO: creating database opensips …

INFO: Core OpenSIPS tables succesfully created.

Install presence related tables? (y/n): y

INFO: creating presence tables into opensips …

INFO: Presence tables succesfully created.

Install tables for imc cpl siptrace domainpolicy carrierroute userblacklist b2b registrant call_center? (y/n): y

INFO: creating extra tables into opensips …

INFO: Extra tables succesfully created.

opensipsdbctl tool takes its configuration from the file “etc/opensipsctlrc”

Notes

  • Install and configure mysqld as a system service:

# yum -y install community-mysql-server

       # systemctl start mysqld.service

       # systemctl enable mysqld.service

  • Check if its running: # ps aux | grep mysql
  • Default password is empty so only press “Enter” when asking for password. For security, set root password as following:

  mysql> set password for root@localhost=password(‘fedoracore2’);

Generate OpenSIPS Configuration File

The prerequisite is to install m4 package which comes by default with most Linux distribution. M4 is GNU macro processor and it is useful for writing text files which can be logically parsed.

Now we are in “/usr/local/opensips_1_11/”

Go to sbin and execute ./osipsconfig. You will get the text interface to generate the configuration file.

snapshot3

Three kinds of configuration files/scripts can be generated:

  • Residential scripts: If we have customers and we want to offer them certain services.
  • Trunking scripts: where we have PBXs and SIP trunks and we want to route the traffic.
  • Load-balancing scripts: Simple load balancer that can load the traffic between chosen destinations.

Example: Generating residential script (“Generate OpenSIPS Script” –> “Residential Script” –> “Configure Residential Script”) so we come to this page:

snapshot4

Here we can configure how OpenSIPs will work. After this configuration press “q” to go to the previous page then click on “Generate Residential Script”.

snapshot5

The generated script will be placed in the OpenSIPs etc folder (/usr/local/opensips_1_11/etc/opensips) marked with the date of its generation. Example: “opensips_residential_2014-9-25_12:30:52.cfg”. After the generation of the file we need to customize it little bit. For example fix the IP addresses and ports that the OpenSIPs will be working on. For example:

listen=udp:10.0.0.4:5060 # CUSTOMIZE ME
Also customization of the places where DB connection is required. Example:

modparam(“siptrace”, “db_url”, “mysql://user:passwd@host/dbname”)
modparam(“usrloc”, “db_url”,”mysql://opensips:fedoracore2@localhost/opensips”) # CUSTOMIZE ME

Don’t forget to check the path of the installed modules: mpath=”/usr/local/opensips_proxy/lib/opensips/modules”

OpenSIPS Init Script

To make OpenSIPS works as system service, we need Init script. Follow these steps:

  • Go to the src folder : “/usr/local/src/opensips_1_11/packaging”
  • Here you have to select your system. For example if you have Fedora go to fedora sub-directory.
  • In that folder you will see the script opensips.init which is the actual init script. Copy this script to /etc/init.d/opensips and give it execution right.

# cp opensips.init /etc/init.d/opensips
# chmod +x /etc/init.d/opensips

  • Edit the script
    # vim  /etc/init.d/opensips

Here is a part of the script where there will be changes. The red line replace the line above it. Please see the corresponding to this in your system.

prog=opensips

opensips=/usr/sbin/$prog

opensips=/usr/local/opensips_1_11/sbin/$prog

cfgdir=”/usr/local/opensips_1_11/etc/$prog”

pidfile=”/var/run/$prog.pid”

lockfile=”/var/lock/subsys/$prog”

configfile=”/etc/$prog/$prog.cfg”

configfile=”$cfgdir/opensips_residential_2014-9-25_12:30:52.cfg”

Include New Syslog Entry for OpenSIPS

  • Open the configuration file. In the Global Configuration section you will see the following :

          debug=3

          log_stderror=no

          log_facility=LOG_LOCAL0

          Change 0 to 1

          log_facility=LOG_LOCAL1

  • Edit rsyslog.conf: #vim  /etc/rsyslog.conf
    Add this entry:   local1.*                        /var/log/opensips.log
  • Restart rsyslog daemon : #systemctl restart rsyslog OR  #service rsyslog restart
    All OpenSIPs log messages will be stored in this file /var/log/opensips.log

   By adding ““, the logging will be in Asynchronous mode so opensips will not wait until the messages are propagated.

Add Linux User and Group For OpenSIPS

Execute “# system-config-users”. Add user=opensips and create group=opensips for that user

Run OpenSIPS

The database server must be running before starting OpenSIPS. For example if the database server is mysql, check its status (# systemctl status mysqld.service) and run it if it is not running (# systemctl start mysqld.service).

To enable and start opensips as a system service.

# systemctl start opensips.service

# systemctl enable opensips.service

Check the file “/var/log/opensips.log” to see if everything is ok and no error messages (# tail -f  /var/log/opensips.log).

Example of an error message in the opensips.log:

ERROR:uri:db_checks_fixup1: configuration error – no database URL is configured!

uri” is the name of the module which has a problem.

Note: Problems which arise before the loading of the configuration file are still going to “/var/log/messages”.

Now the opensips should be running and we can check this by executing some commands:

♠ Check the status of the OpenSIPS’s service (i.e. Loaded and Running): #systemctl status opensips.service

♠ Check network’s state of the OpenSIPS server: # netstat -lpn |grep opensips

tcp 0 0 127.0.0.1:5060 0.0.0.0:* LISTEN 32634/opensips

udp 0 0 127.0.0.1:5060 0.0.0.0:* 32634/opensips

Note: OpenSIPS will not start on database init failure and it will exit immediately (return -1) on module initialization failure (calling the function “mod_init”).

Start/Restart OpenSIPS

To restart OpenSIPS, do the following:

# systemctl restart opensips.service

or

#  cd /etc/init.d/

# ./opensips restart

Start Working

Now after OpenSIPS has been installed successfully, you can start using its modules in the routing script. What i recommend when using any module:

  • Reading in the module documentation (Last Version). Note the section “Exported MI Functions” in the documentation of each module lists the module’s exported MI functions that can be called from outside of OpenSIPS (External applications). Example:

# opensipsctl fifo debug 1 

OpenSIPS project is growing quickly so please always read in the latest documentation.

  • Sometimes you need to look at the Database Schema. Some fields are pushed to database only when a specific module parameter is set. Example: In multi-domain scenarios, you might need to save the domain part of the user so the domain part and the username part are used to identify the user. So in this case you need to set the “use_domain” parameter of “usrloc” module to non zero value which means true.

modparam(“usrloc”, “use_domain”, 1)


More Information