Installation of MySQL Database Server

.

These simple steps install and start MySQL database server on Fedora 20:

  • Install the package “community-mysql-server”: # yum -y install community-mysql-server
  • Start the daemon service: # systemctl start mysqld.service
  • Enable the daemon service: # systemctl enable mysqld.service
  • Connect to MySQL: # mysql -u root. Type Enter
  • Set the root password: mysql> set password for root@localhost=password(‘password’);

You can replace the localhost with your domain or IP address.

  • Delete anonymous users: mysql> delete from mysql.user where user=”;
  • mysql> exit

If you forgot the root password, do these:

  • # mysqld_safe –skip-grant-tables &
  • mysql> use mysql;
    mysql> update user set password=PASSWORD(“New-Password”) where User=’root’;
    mysql> flush privileges;
    mysql> quit
  • # systemctl stop mysqld
  • # systemctl start mysqld
  • # mysql -u root -p

Advertisements

Distributed Hash Table (DHT) In Structured Peer-to-Peer SIP Overlay

.

peersDistributed Hash Table (DHT) is used in structured Peer-to-Peer overlay for resource location and discovery. In SIP world, the resource can be the contact list (physical addresses) associated with an AOR (SIP/SIPS URI that points to a user on domain -virtual address of the user). The mappings between the AORs and the contact lists will be distributed amongst the peers in the SIP overlay. In this way we get a distributed registrar functionality for SIP.

There should be an interface for the DHT so it can be used without caring about the implementation and it can easily integrate with the SIP server so the server can work in P2P mode. Basically the functions that need to be implemented are: Join (join the overlay), Leave (leave the overlay), Lookup (search for a resource using the AOR as a key), and Store (store the resource). For example the lookup function works as following:

  • Using the AOR as a key to find the appropriate node/peer. The hash value of the key is used to find the node/peer ID.
  • Then the selected node/peer will do normal hash table (HT) lookup to get the value/resource/contact-list

The mappings {AOR, Contact List} can be stored persistently on disks in a key-value databases like Berkeley DB.

The IETF standards body is working on P2P-SIP. There is a working group called p2psip where you can find the internet drafts (working documents) related to P2P-SIP. For example the RELOAD (RFC [6940]) is a signaling protocol for resource location and discovery. It specifies “chord-reload” as a mandatory DHT algorithm to be implemented. The purpose of this work is distributed SIP registrar. The RELOAD works with SIP to enable the distributed SIP solution. The RELOAD can be used by other protocols and not only SIP (e.g. A Constrained Application Protocol (CoAP) usage for RELOAD: Internet Draft: draft-jimenez-p2psip-coap-reload-10).

But why P2P SIP mode. This is because in P2P:

  • There is no one point of failure.
  • Less cost: avoid having service provider (Paying) + delete nodes/peers on low demand.
  • Capacity: Adding new nodes/peers on high demand.

The peer to peer SIP overlay is very suitable to be built over Openstack cloud where the automated creation and deletion of nodes/peers is based on predefined policies. The peers are defined in an autoscaling group where they are scaled up and down based on the autoscaling policies. When a node is determined to leave the overlay based on the cloud policies, the node starts the leave process where the configuration is delivered to it from the cloud (DELETE lifecycle hook). The trigger to add (<=> join the overlay) or delete server (<=> leave the overlay) are controlled by the cloud orchestration service (the scaling policies). The policies itself are based on CPU utilization, Load,…etc.


Creative Commons License
The content of this blog is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.


First Step In WebRTC

.

Introduction

Here i will show you how to execute very simple WebRTC demo served by Apache web server . The example is how to get the media stream of the local device. I will take as an example the WebRTC “getUserMedia” example from the book “Real-Time Communication with WebRTC by Salvatore Loreto and Simon Pietro(O’Reilly)”. You can find the source code on the book’s GitHub page. Follow these steps:

  • Create a folder for your WebRTC project: # mkdir /var/www/html/webrtc
  • Create subdirector for Javascript files: # mkdir /var/www/html/webrtc/js
  • Open Apache configuration file “/etc/httpd/conf/httpd.conf” and add this line:

Alias  /webc  /var/www/html/webrtc

    Restart Apache: # systemctl restart httpd.service

Screenshot from 2015-03-08 20:20:28To debug your project, open the browser console (e.g. Chrome: More tools –> Javascript Console).

JSFIDDLE Framework

You can use jsfiddle framework to write, save, validate, and run your application online.

Notes

  • Update your browser (bug fixes). Using the developer edition is a good choice (e.g. Firefox Developer Edition).
  • Test your application on different browsers

OpenSIPS Event Interface – Part 1

.

 

External applications can interact with OpenSIPS through Management Interface (MI) which is pull-based mechanism and Event Interface which is push-based mechanism. The event interface acts as a mediator between OpenSIPS regular modules, the configuration script and the transport modules. The events can be published by the modules and the configuration file/script. The external applications subscribes for a certain event and OpenSIPS event interface notifies the external application when that event is raised. OpenSIPS has some hardcoded events such as E_PIKE_BLOCKED which is raised by the “Pike” module), E_CORE_PKG_THRESHOLD which is raised when private memory usage exceeded a threshold,…..etc. Several transport protocols are used to send the event notification to the external subscribed application. Each protocol is provided by separate OpenSIPS module.

Events Raised from OpenSIPS Modules

If the module wants to export events, it must register them in its initialization function (init_function” member in the module interface “struct module_exports”). The function “evi_publish_event” defined in “evi/evi_modules.h” is used to register an event. The function “evi_raise_event” is used to raise an event. The function “evi_probe_event” checks if there is any subscriber for that event. This function is called before building parameters list because if no subscriber has been found, then the delay of building parameters list and other things can be skipped.

event_id_t evi_publish_event(str event_name);

int evi_raise_event(event_id_t id, evi_params_t* params);

int evi_probe_event(event_id_t id);

Events Raised from OpenSIPS Script

To raise an event from script, you have to call the core function “raise_event” from the script. The first parameter of this function is the event name and it is constant string parameter (pseudo variable cannot be used) because the event must be registered at startup and the pseudo variable would have an undefined value. The second and third parameters form together the list of the parameters and they are optional. This is how to call this function:

$avp(attr-name) = “param1”;

$avp(attr-name) = “param2”;

$avp(attr-val) = 1;

$avp(attr-val) = “2”

raise_event(“E_Example”, $avp(attr-name), $avp(attr-val));

Click here to see the full explanation.

The length of both AVPs have the same number of parameters.

Transport Module Interface

This interface is implemented by all transport modules that are used to send the event notification to the external applications. It is defined in the file “evi/evi_transport.h” as following:

typedef struct evi_export_ {
str proto;                      /* protocol name / transport module name */
raise_f *raise;         /* raise function */
parse_f *parse;         /* parse function */
match_f *match;         /* sockets match function */
free_f *free;           /* free a socket */
print_f *print;         /* prints a socket */
unsigned int flags;
} evi_export_t;

The first field is the name of the transport protocol. It is used by the MI subscription command so the event notification will be sent by the transport named in the subscription request. The second filed is the raise function which will be called by the OpenSIPS Event Interface for each subscriber, to send the notification on this transport. This function serializes the event parameters, and send them to the subscriber. The event parameters are passed to the transport module from the module that raised the event.

The third field is the parse function which parses the subscriber socket. The match function is used to update the expire period of the subscription or unsubscribe the application. The rest of fields are easy to understand.

The transport module which implement this interface must be registered to OpenSIPS event interface. This is can be done by calling the function with the an instance of the above structure as a parameter:

int register_event_mod(evi_export_t *ev);

The Availability of the Events

All exported events will be available at startup time after the parsing of the configuration file is completed. The external application can only subscribe to en event after OpenSIPS has been started up.

Event Subscription

To subscribe for events, the external application send this MI subscription command:

event_subscribe Event_Name Transport Socket Expire

Example: #opensipsctl fifo event_subscribe E_PIKE_BLOCKED udp:127.0.0.1:8888 1200

If the subscription succeeded, the application gets 200 OK message and it will start getting notifications for the subscribed event when the event is raised. The external application suppose to do something upon receiving the notification. For example: update the firewall, do some processing, or just printing or send information mail to administrator.


More Information


 

I/O Design Patterns For SIP Routers

.

 

In real-time applications like SIP applications, we have a lot of IO operations (database IO operations, network IO operations,…etc.) and the time to accomplish each IO operation is critical. Using event based architecture with asynchronous IO minimizes the time of IO operation and scale up the performance.

Here i just would like to explain the concept of reactor and proactor patterns for I/O operations. Both the reactor and proacor depend on event firing but the difference who is doing the actual IO operations.

Reactor Pattern

The reactor pattern involves synchronous I/O. The event here is defined as “ready to read or write”. When this kind of IO event is fired, the event demultiplexer passes the event to the registered user event handler/callback. The user event handler/callback function will be called to do the actual IO operation (read or write).

Proactor Pattern

The proactor IO pattern involves asynchronous IO. The event here is defined as “read or write is completed“. When this kind of IO event is fired, the user callback function/event handler will be called to process the data if the operation was “read” and do something else (new async operation) if the operation was “write”. Here the actual IO operation is done by one of the kernel threads which does the actual IO operation using a user defined buffer passed to it when the asynchronous system call is done. When the kernel finished the IO operation, it notifies the event demultiplexer and this will pass the event to the user handler/callback. As you can see the operating system must support the asynchronous IO so it can use the user buffer to do the actual IO operation to save the user application time.


.

OpenSIPS Module Interface – Part 2 (C Development)

.

Introduction

This is Part 2 of the topic “OpenSIPS Module Interface”. In Part 1,I have talked about the definition of the module interface, some fields of its structure, and how to export function from the module to the routing script. So you can call this function in the routing script. Also i have talked about how to export a module parameter. I have talked about the OpenSIPS management interface in the article “OpenSIPS Management Interface (C Development)“. In this article, i will explain a little bit in the module interface and i will show you how to compile a new module.

The definition of the module interface is defined in the header file: “sr_module.h” as following:

struct module_exports {.

……… /* Fields that are explained in the Previous Article are missed here*/

……..  /* Some other fields are missed to be explained in the Next Article */

init_function   init_f;                              /* Initialization function */

destroy_function destroy_f;                /* Function called when the module

.                                                                       should be “destroyed” */
.child_init_function init_child_f;            /* Function called by all processes after the fork */

}

init_f function which will be called at OpenSIPS startup time. In OpenSIPS 1.x which is multi process application, this function will be called when OpenSIPS is just one process (attendant process). For example checking for the database connection can be done here or allocating a memory that will be as shared memory for the forked processes (after forking). You can define your init_f function in the “exports” which is the actual module interface. Lets say “mod_init” is the initialization function in the module interface:

static int mod_init(void){

……

}

The initialization of the exported variables in the routing script will be done before calling the function mod_init. The full OpenSIPS configuration file will be parsed and the own module parameters will be included. For example the parameter “db_url” which will be configured in the routing script:  modparam(“module_name”,”db_url”,”mysql://user:passwd@host/database”) and the “db_url” will be evaluated and can be used in the function “mod_init”. So a connection to the database can be opened in the body of the “mod_init” function.

Also the helper API like shared memory, locking, timer, ..etc. will be available and can be used. Note some resources like new timer processes for the module can only be initialized in the “mod_init” function.

After forking, each process will get a copy of what the OpenSIPS attendant process had before forking (structures/connections).

init_child_f function will be called after OpenSIPS has been forked (It will be called by each process individually). For example if each process will have its separate DB connection, the creation of the DB connection will be in this function.

static int child_init(int rank){

……..

}

The rank parameter allows you to know which process is calling the function. For example it can be called from SIP worker or MI worker (e.g. fifo worker).

destroy_f function will be called when OpenSIPS will shutdown. Here you can do some cleaning of resources(memory, DB connections, and so on). For example you can free the memory for some module-created variables (e.g. pkg_free(Pointer to the Variable)). OpenSIPS 1.X has its own internal memory manager and the function pkg_free is like the function free(pointer to a variable) of glibc standard library. The pointer must not be null otherwise it will crash. Also here in this function, you can save a state that the module have into persistent storage so they can be loaded again at the next OpenSIPS startup. For example in this function, the “dialog” module saves the dialog states in the database.

OpenSIPS 2.X is multi thread application. I will back to it in one or two upcoming articles.

Add Include Directives

You need to add the include statements to the new module code to be able to compile it. For example: the “string.h”, “sr_module.h” which includes the definitions of the structures, “dprint.h” which is used to print stuffs to the syslog or standard error, “mem.h” which is used to allocate/free a memory, and the header file which contains the structures of the new module (e.g. new_module.h).

How to Compile a New OpenSIPS Module

Each OpenSIPS module has its own regular Makefile. It will be run by the master Makefile. The file name is “Makefile” and its content looks like this:

include ../../Makefile.defs
auto_gen=
NAME=new_module.so
LIBS=

include ../../Makefile.modules

The “Makefile.defs” must be included from the OpenSIPS source folder in addition to the name of the shared object “new_module.so” which will be generated and the file “Makefile.modules”. If the module has external libraries dependencies, they should be linked in the module’s Makefile.  Here we have two Makefile variables:

  • DEFS: This is used if the module has additional definitions that you want to pass it to the compiler. Add this line to the module’s Makefile if the module has additional definitions:

DEFS+=-I$(LOCALBASE)/include

OR DEFS+=-I../../../exdef/include

+= is a Makefile operator.

  • LIBS: This is if the  module requires external libraries. So these options:  -llibrary and the -Lpath_to_library will be set to LIBS variable. For example for “cachedb_memcashed module”, add this line to the module’s Makefile:

LIBS=-L$(LOCALBASE)/lib -lmemcashed

To compile the module:

  • The module source files (.c and .h files) and the Makefile should be in the folder “/usr/local/src/opensips_1_11/modules/new_module”
  • From the OpenSIPS ‘s src  folder (“/usr/local/src/opensips_1_11″), execute:

# make include_modules=”new_module” modules OR # make modules modules=modules/new_module

# make install

  • To see if the module is compiled and installed , check having the file “new_module.so” installed in the subdirectory “/usr/local/opensips_1_11/lib64/opensips/modules/“.

Testing

Write log statements (e.g. LM_INFO(….)) in the C code of the new module. Compile it , and restart opensips. Open the log file and search for the name of the module (for example search for “new_module”) to see what is happening to the module “new_module”.


Notes

1- If you’ve got a compilation error (e.g “….. No rule to make target ….”) and you want to clean everything (delete the generated object files and dependencies files (files with extensions “.d”), execute “make proper” and then “make”.

2- If the module depends on external libraries, the module must NOT be compiled by default. To disable this, we edit the file “Makefile.conf.template” which specifies the modules that are not compiles by default.

  • Add a line for your module in “Makefile.conf.template” file in this format:

modulename= Module-Description | module-dependency

For example for the “xcap” module:

xcap= XCAP utility functions for OpenSIPS. | libxml-dev

  • Add the module name to “exclude_modules” list in “Makefile.conf.template” file.
  • After this compile the module as explained in this article.

3- Each OpenSIPS ‘s module has a section in its documentation called “Dependencies -External Libraries” which contains the external libraries that must be installed for this module.


Next

I will continue explaining the module interface.

More Information


OpenSIPS Management Interface (C Development)

.

Introduction

OpenSIPS Management Interface (MI)  is a mechanism which enables the external application (e.g. command line :opensipsctl , web: OpenSIPS-CP) to send commands (MI commands) to OpenSIPS. It is a pull based mechanism which means when you need information you need to query OpenSIPS (i.e. need to do something at a certain time). The MI command allows you to fetch/push data from/to OpenSIPS or trigger some actions there. The core has its own exported MI functions and the module has its own as well. Here you can find some examples of sending mi commands by the external applications “opensipsctl” and “OpenSIPS-CP”

Several transport protocols are used to carry MI commands and their replies between the external application and OpenSIPS. Each protocol is provided by separate OpenSIPS module. The current protocols are mi_fifo, mi_datagram, mi_xmlrpc, mi_http, mi_json, and mi_xmlrpc_ng. These modules require extra processes to avoid disturbing OpenSIPS main processes that are working with SIP.

The module must be loaded (i.e. configured to be loaded in the routing script) so its exported MI functions (exported by the module interface “exports”) are populated and can be called from the external application otherwise you will get an error message “500 command ‘Module_Name’ not available”. To be able to send MI command from the external application, the transport module also must be configured in the routing script to be loaded. For example if you want to connect to MI interface via FIFO file stream, the module MI_FIFO must be right configured and loaded. The same for the rest of transport protocols.

So two modules are needed to be able to call a specific MI function:

  • The module which exports the MI function.
  • The transport protocol which will transport the command to OpenSIPS.

The extra processes that are required for transport issues will be listening on different ports than the SIP ports. OpenSIPS can use multiple transport protocols at the same time (In the routing script, configure them to be loaded).

MI Command Syntax

If you are willing to write an external management application, you have to implement the transport you want to use. Your application will behave like a client which sends MI command in a specific format to OpenSIPS. These are the current formats:

MI Internal Structure

The following is the module_exports structure (Module Interface) defined in the file “sr_module.h” with the parts related to MI.

struct module_exports {.

……… /* Many Fields are Missed For Simplicity */

mi_export_t*     mi_cmds;           /* Array of the exported MI functions */

….

proc_export_t*  procs;     /* Array of the additional processes required by the module */

……

}

procs

Sometime the module needs extra processes like the transport protocol. So it exports

  • The number of required processes to be forked (“no” number).
  • The helper functions (pre_fork_function and post_fork_function) which help the attendant process to create the extra processes.
  • The function which will be executed by the extra processes. What will be done by these extra processes will not interfere with the rest of processes that handle the SIP.

The structure is defined in the file “sr_module.h” in OpenSIPS source directory:

typedef struct cmd_export_ cmd_export_t;

struct proc_export_ {
char *name;
mod_proc_wrapper pre_fork_function;
mod_proc_wrapper post_fork_function;
mod_proc function;
unsigned int no;
unsigned int flags;
};

typedef void (*mod_proc)(int no);

typedef int (*mod_proc_wrapper)();

The flags can be 0 or PROC_FLAG_INITCHILD. If PROC_FLAG_INITCHILD is provided, the function “child_init” from all modules will be run by the new extra processes.

Example of procs NULL terminated array:

static proc_export_t mi_procs[] = {
{“MI Datagram”,  pre_datagram_process,  post_datagram_process,
datagram_process, MI_CHILD_NO, PROC_FLAG_INITCHILD },
{0,0,0,0,0,0}
};
static param_export_t mi_params[] = {
{“children_count”,      INT_PARAM,    &mi_procs[0].no},…….}

mi_cmds

mi_cmds is an array of exported MI function (Type: mi_export_t*). The definition of the type “mi_export_t” is included in the file “mi/mi.h” as following:

typedef struct mi_export_ {
char *name;
char *help;
mi_cmd_f *cmd;
unsigned int flags;
void *param;
mi_child_init_f *init_f;
} mi_export_t;

name is the actual name of the MI function which will be called. help is the information about what this function is doing. cmd is a pointer to the actual mi function:

typedef struct mi_root* (mi_cmd_f)(struct mi_root*, void *param)

The return value is mi_root * (pointer to the root of the MI reply tree).  The first parameter is the MI root parameter(type: mi_root *) and the second parameter is the actual command parameter (void *). the param is the actual parameter of the MI function.

init_f is the child init function for the exported MI function. In some parts, some extra stuffs are needed to be done by the MI workers (processes that are handling the MI requests  like MI_FIFO proces, MI_XMLRPC process,….. This function will be called at the startup after OpenSIPS has been forked (will be called one time by each core process individually).

Example of MI functions NULL terminated array:

static mi_export_t mi_cmds[] = {
{ “mi_get_function“,”This Function is doing bla bla”,mi_get_function,MI_NO_INPUT_FLAG,0,0},
{0,0,0,0,0,0}
}

The string “mi_get_function” is the actual name of the mi command which will be named by when MI command is received and needed to be verified (lookup_mi_cmd(MethodName,..)). The actual function will be like this:

struct mi_root* mi_get_function(struct mi_root*  root, void * param){

……..

}

MI Function Reply Tree

The return value of MI function is a pointer to a tree (reply tree) . The root of this tree is pointer to struct mi_root. This structure is defined in the file “mi/tree.h”:

struct mi_root {
unsigned int  code;
str   reason;
struct mi_handler  * async_hdl;
struct mi_node node;
}

The code is the root code of the response (200 for sucess,500 for error, …) and the node is the starting node (type: mi_node defined in the same file “mi/tree.h”):

struct mi_node {
str value;
str name;
unsigned int flags;
struct mi_node *kids;
struct mi_node *next;
struct mi_node *last;
struct mi_attr *attributes;
}

To initialize MI reply tree in the previous MI function (mi_get_function), the function “init_mi_tree” can be called. To add child node, the function “add_mi_node_child“. To add attribute, the function “add_mi_attr” is called  and so on. Here you can find an example code where a reply tree is built when a MI function is called.

struct mi_root* mi_get_function(struct mi_root*  root, void * param){

………

struct mi_root * rpl_tree = init_mi_tree( 200, MI_SSTR(MI_OK));    /* 200 OK Reply */

……..

}

The transport process like datagram_process, xmlrpc_process,..etc  behaves as a server which accepts connections (UDP connections in case datagram transport , TCP connections in case xmlrpc connections and so on). To serve the MI request, the transport process calls functions from OpenSIPS core (Management Interface: “mi/mi.h”) in addition to some module-defined functions.

When the message is received, the transport process checks if the requested MI function is available (looks up the MI command):

struct mi_cmd*  fu =lookup_mi_cmd((char*)methodName, strlen(methodName))

The “lookup_mi_cmd” is defined in “mi/mi.h”. If “f==0”, the MI command is not available. Otherwise it parses the parameters of the requested MI function into a MI tree. For example in xmlrpc, the function “xr_parse_tree” is defined in the file “modules/mi_xmlrpc/xr_parser.h” and called in “modules/mi_xmlrpc/xr_server.c”. It returns a pointer to the reply tree (mi_root * t).

After this, the actual mi function will be called through the function “run_mi_cmd”. As i explained above the return value of the MI function (reply tree) has the type mi_root*. This is how it is called in xmlrpc transport module “”.

struct mi_root*  mi_rpl=run_mi_cmd( fu , t , (mi_flush_f*)xr_flush_response,env))

The function “run_mi_cmd” is defined in the file “mi/mi.h” and in its body you call the actual MI function as following:

static inline struct mi_root* run_mi_cmd(struct mi_cmd *cmd, struct mi_root * t, mi_flush_f *f, void *param){

struct mi_root *ret;

…..

ret = cmd->f( t, cmd->param);

…..

}

struct mi_cmd {
int id;
str module;
str name;
str help;
mi_child_init_f *init_f;
mi_cmd_f *f;
unsigned int flags;
void *param;
};

Then the reply tree mi_rpl  (Type: mi_root *) will be formatted corresponded to the transport process (transport protocol). For example if the protocol is xmlrpc, then the reply tree will be formatted in xml either as string that contains the name and attributes of each node OR as an array where each element contains a node information (node name and its attributes). Each attribute has name and value.

Finally, the response will be written in fifo file, UDP socket (datagram transport), TCP socket (xmlrpc transport) so it will be sent to the external application.

Test MI Function of a New Module

You can test the MI command by using opensipsctl as following:

# scripts/opensipsctl fifo mi_get_function


 Note

  • The OpenSIPS core MI exported functions (mi_core_cmds) are defined in the file “mi_core.c”.
  • The OpenSIPS Module MI exported functions (mi_cmds) are defind in the file modules/Module_Name/Module_Name.c
  • OpenSIPS has many transport modules. Go to “modules/mi_Transport“, Where Transport can be datagram, xmlrpc, json,…etc.

 More Information


OpenSIPS Memory Debugging

.

Introduction

If you have problem with the memory on your SIP router, do these:

  • Check things related to the memory management in your operating system. This is explained in the article “Linux Tuning For SIP Routers – Part 3“.
  • Check the memory management in the application of your SIP router.

Here i will give you just an example and you will take this as a way of thinking.

OpenSIPS Memory Debugging

If you got an error message related to OpenSIPS memory (e.g out of memory message), you have to debug it. The OpenSIPS Memory manager (NOT standard) supports debugging so what you can do is enabling the debugging:

  • From the OpenSIPS ‘s src  folder (“/usr/local/src/opensips_1_11″), run: “make menuconfig”. You will get this text based interface:

1

  • Press “ENTER” key to enter in “Configure Compile Flags” item:

2

  • Use the arrow keys to go to the sub-item “F_MALLOC“. Press the SPACE key to uncheck it. Then move to DBG_QM_MALLOC and check it (press SPACE key). Use the option “DBG_QM_MALLOC” in development and the option “F_MALLOC” in deployment because “F_MALLOC” is much faster.
  • Press ‘q’ to go back to the previous screen.
  • Press on “Save Changes”.
  • Press ‘q’ again and then move to the item “Compile and Install OpenSIPS” to compile and install OpenSIPS.
  • Open the configuration file and set memlog=1
  • Restart OpenSIPS and wait OpenSIPS to be completely loaded.

Now everything is ready to get the memory dumps (at shutdown and at runtime).

If you want to get the dumps at runtime:

  • Run “ps aux |grep opensips” to get OpenSIPS processes with their PIDs.
  • Run “kill -SIGUSR1 PID”. Where PID is the ID of the process and SIGUSR1 is the signal name which triggers the memory dump in OpenSIPS. The signals handlers (file: “main.c”) are loaded at the first stage of OpenSIPS startup.
  • Wait some time so the messages for private and shared memory will be dumped.

At shutdown: Stop OpenSIPS and go to the log and get the memory messages.


More Information


 

OpenSIPS Module Interface – Part 1 (C Development)

.

Introduction

OpenSIPS consists of two parts: the core and the modules. OpenSIPS module is a shared library (.so file extension) which is configured in the routing script and loaded at OpenSIPS startup time (the loader: /usr/bin/ld). The name of the module is the name of the shared library.

To open and edit the source files, I will use the text editor vim. You can install it on Fedora Red Hat: # yum install vim Or use your own favourite text editor to edit the source files. Then go to the OpenSIPS src folder (It can be “/usr/local/src/opensips_1_11”) and explore the files (# vim File-Name). To enter in write mode, press i. To paste a copied code into an opened file do it like this: press ESC and then write :set paste. To save a file press ESC and them write :wq

If you are reading in a source file and you want to open a file (i.e. the file where its name comes after the “#include” statement), put the curser on the file name and press either “gf” to go to the file or “ctrl-w f” to open the file in a new window.

In this part i will talk about the definition of the module interface, some fields of its structure, and how to export function from the module to the routing script. So you can call this function from the routing script. Also i will talk about exporting a module parameter.

OpenSIPS Module Interface

OpenSIPS module interface is called module_exports and it is used by OpenSIPS core to serve the module. Each OpenSIPS module creates a variable of this type and call it by “exports”.  When OpenSIPS tries to load the module, it takes the module path from the configuration file (loadmodule statement), open the shared object file (.so file), and try to find the “exports” which populates everything about the module (e.g. script functions, parameters, mi functions, ..etc.).

ModuleInterface

The structure of the module interface (module_exports) is defined in the header file “/usr/local/src/opensips_1_11/sr_module.h”. It is a struct C data type which has the declared name module_exports. It defines a grouped list of fields that are populated by the module. I will explain some of these fields and leave the rest to the next article.

struct module_exports {
.          char* name;                                     /*  Module Name */
.          char* version;                                  /*  Module Version */
.          char* compile_flags;                        /*  Compile flags used on the module */
.          unsigned int dlflags;                          /*  Flags for dlopen */

.          cmd_export_t* cmds;                      /* Array of the exported commands */
.          param_export_t* params;                /* Array of the exported module parameters */

.          …  /*Rest of Fields*/
}

The fields are:

name: It is the module name. If the module will be used, its name must be passed to the function “loadmodule” in the configuration file. The corresponding shared library (shared object) will be “new_module.so” and it will be loaded at the startup time. version: It is the module version. It is used to detect the conflict situations where you have a module with version different from the core version. In the same file “sr_module.h”, we have the defined variable “MODULE_VERSION” which is OPENSIPS full version concatenated with the OpenSIPS compilation flags.

#define MODULE_VERSION  OPENSIPS_FULL_VERSION, OPENSIPS_COMPILE_FLAGS

This variable can be set to the field version in the variable “exports” which has the type module_exports and represents the actual module interface. This is shown further below.

compile_flags: You can skip this if you have used the MODULE_VERSION which already has the compile flags after the comma “,“.  Otherwise use them independently. Remember when you define a variable of struct in c language, the “,”  is used to separate between the fields.

dlflags: In the same file “sr_module.h”, we have  the defined variable DEFAULT_DLFLAGS which has the value 0 which means instructs the module loader to use OpenSIPS default dlopen flags.

#define DEFAULT_DLFLAGS  0

This variable can be set to the field dlflags in the actual interface “exports”.

cmds which has the OpenSIPS-defined C-type cmd_export_t*. It is an array of functions that are exported to the routing script. It is defined in the same file “sr_module.h”

typedef  struct cmd_export_   cmd_export_t;

struct cmd_export_ {
char* name;                                    /* null terminated command name */
cmd_function function;                    /* pointer to the corresponding function */
int param_no;                                  /* number of parameters used by the function */
fixup_function fixup;                      /* pointer to the function called to “fix” the parameters */
free_fixup_function free_fixup;     /* pointer to the function called to free the “fixed” parameters */
int flags;                                         /* Function flags */
}

The structure above defines a function in a dynamic and generic way. The name is the function actual name which will be used from the script when calling the function. The function is a pointer to the C code of the function which will be actually called. This pointer is defined as following:

typedef  int (*cmd_function)(struct sip_msg*, char*, char*, char*, char*, char*, char*);

The return value of the function is integer (int). “cmd_function” is the pointer to the function. The rest are the parameters of the function (First parameter is the actual SIP message which triggers the processing and the rest are multiple char pointers. You can have till sex arguments). In the module C-code, you will have something like this:

int new_function(struct sip_msg *msg,char *param) {

   ……… /* Actual Exported Function Code*/

    /* You can manipulate the SIP message Here:

    /* Remove/Add header Fields …….*/

}

This function has one “char *” parameter but as i said you can have up to 6 “char *” parameters. The return value of the exported function can be either a positive number (higher than +1) on success or lower than -1 on failure. The return value must not be zero because this will exit the script.

The param_no is the number of parameters passed to the function. If this number is equal to zero, this means there are no parameters for the function. But keep in mind that the first parameter (SIP message) will always be received as a first parameter of the C code function even if you call the function with no parameters from the routing script (Example new_function() in the script will be something like new_function(msg) in the C code).

These parameters must be evaluated (get the length of each “char *” parameter) and probably processed. To avoid CPU penalty of doing this evaluation and processing many times (For each function call), the evaluation and processing can be done only once at the OpenSIPS startup via fixup function. This reduces the runtime evaluations.

fixup is a pointer to the function which will be called to fix the parameters of the exported function. The structure of fixup pointer is “fixup_function” which is defined in the file “sr_module.h” as following:

typedef  int (*fixup_function)(void** param, int param_no);

The fixup function will receive the current parameter to be processed (param) and its index (param_no). It is important to check the parameter index so the function knows which parameter it is currently evaluating.  The actual code of the fixup function looks like this:

int do_fixup (void** param, int param_no){

/* Parameter Evaluation and probably Processing */ check the parameter you are currentlu evaluating

char * value= (char *)(*param);

str* ret_value;

 ………

if (param_no==1){

ret_value=pkg_malloc(sizeof(str));

 ……..

ret_value->s=value;

ret_value->len=strlen(value);

*param=(void* )ret_value;

return 1;

}

       …….

return 0;

}

OpenSIPS 1.X has its own internal memory manager (NOT standard) and the function pkg_malloc allocates a memory of a given size (here sizeof(str)) and return a pointer to the allocated memory (here ret_value is a pointer to str).  Note that the received parameter (param) is replaced with the evaluated and processed value (ret_value). After the fixup function has been called, the exported function will have the param fixed (type = str not char*) but casting is still required ((str *)param).

Using fixup function is not restricted to constants. It can be also used with variables.

free_fixup is a pointer to the function which is used to free up the fixed parameters (free up the resources used for the fixed parameters).

flags represents the places in the routing script where the function is available and can be called. For example it can be called from REQUEST_ROUTE, FAILURE_ROUTE, BRANCH_ROUTE, ONREPLY_ROUTE, or LOCAL_ROUTE (these routes are defined in the header file “route.h”). If the function is called in forbidden place in the routing script, you will get an error message and OpenSIPS will not start.

The figure below is a graphical representation of the module interface:

The Actual Module Interface

When you create a new module, you need to create a variable of the type struct module_exports which will be the actual interface of the new module. Assume you are writing the module “new_module”, you will add its interface (named “exports”) in its C code. It could be like this:

struct module_exports exports= {
.               “new_module”,                                  /* module name */
.                MODULE_VERSION,                   /* Module Version
.                DEFAULT_DLFLAGS,                  /*  dlopen flags */
.                cmds,                                              /*  exported functions */
.                params,                                           /*  exported parameters */

        …. rest of values

}

How to Export a Function to the Routing Script ?

For example if the module has two functions and you want to export these functions to the routing script, add them to an array of the type cmd_export_t and make the array null terminated ({0,0,0,0,0,0}). As an example assume the first function with one argument and the second function without any argument (same name will be used in the routing script but different names in the C code).

static cmd_export_t cmds[]={

{new_function“,(cmd_function)new_function,1,do_fixup,0, REQUEST_ROUTE|ONREPLY_ROUTE},
{new_function“,(cmd_function)new_function0,0,0,0, REQUEST_ROUTE|ONREPLY_ROUTE},
{0,0,0,0,0,0}
}

In the routing script, these functions can be called only within REQUEST_ROUTE or ONREPLY_ROUTE. Otherwise as i mentioned above, you will get an error messages and OpenSIPS will not start.

The fixup function “do_fixup” is populated here so it will fix up the only one parameter of the exported function “new_function”.

Somewhere in the module C code, You will write the bodies of the actual exported functions: The first function “new_function” which has only one parameter to be passed from the script will look like this:

int new_function(struct sip_msg *msg,char *param){

…….

/* The do_fixup is the fixup function of this function, so we can cast the param to str * as it is evaluated there */

str * param_value = (str *)param; 

……..

}

The second function is new_function0 which has no parameters when it is called from the routing script.

int new_function0(struct sip_msg *msg){

…….

}

How to Export a Parameter to the Routing Script ?

The field “params” in the definition of the module interface is an array of parameters which are exported from the module to the routing script so they can be used in the script and populated by script parser. The exported parameter can be set from the routing script. The OpenSIPS-defined C-Type of the exported parameter is “param_export_t” and it is defined in the file “sr_module.h” as following:

typedef struct param_export_ param_export_t;

struct param_export_ {
char* name;                  /*!< null terminated param. name */
modparam_t type;        /*!< param. type */
void* param_pointer;    /*!< pointer to the param. memory location */
}
The field name is the name of the parameter which will be used in the script. The type can be:

  • A regular string (Flag: STR_PARAM) or regular integer (Flag: INT_PARAM)
  • Or one of the above types (STR_PARAM or  INT_PARAM) combined with the flag USE_FUNC_PARAM (STR_PARAM|USE_FUNC_PARAM) or (INT_PARAM|USE_FUNC_PARAM). This means there is a function will be called every time the parameter is set.

The field param_pointer which is a pointer to either the memory location of the parameter or a pointer to the function which will be called on the value given from the routing script.

All exported parameters will be grouped in “params” in “exports” (actual interface). Remember the OpenSIPS-defined C-Type of the params array is param_export_t*. For example it can be:

static param_export_t params[]={
{ “new_param”, STR_PARAM|USE_FUNC_PARAM, (void *)set_function},
{ “db_url”,     STR_PARAM, &db_url.s},
{0,0,0}
}

The name of the first exported parameter is the string “new_param”. The type of this parameter is “STR_PARAM|USE_FUNC_PARAM” (i.e. “|” is a bit mask).  The function “set_function” will be called each time the parameter “new_param” is set. It look like this:

int set_function(unsigned int type, void *val){

…….

}

In the routing script this parameter can be set as following: modparam(“new_module”,”new_param”,”Test Value”) and the function “set_function” will get the value of “new_param” parameter (“Test Value”) as its argument.


Note

Start playing with OpenSIPS Source code (read in the code, make some changes, write log statements (e.g. LM_INFO(….)), recompile&start OpenSIPS, and read the log and see the results). If you want to build your own module, I suggest you to take as start step the OpenSIPS Example which is available on the OpenSIPS web site and make any OpenSIPS module like dialog module (Files: dialog.c and dialog.h) as a reference to your code (follow the same strategy).


Next

In the Next Part, i will continue explaining the rest of the module interface fields.


 More Information


OpenSIPS Startup Cost

.

klok.To measure the cost of OpenSIPS startup, i will use the environment variable LD_DEBUG which is used to instruct Linux dynamic linker to dump debug information. To display all valid options, execute:

# LD_DEBUG=help cat

Here I am using the default routing script of OpenSIPS. I will set the variable LD_DEBUG to the value “statistics” while starting OpenSIPS:

  • # cd /etc/init.d
  • # LD_DEBUG=statistics ./opensips start

I got:

19179:     runtime linker statistics:
19179:       total startup time in dynamic loader: 274776 clock cycles
19179:                 time needed for relocation: 51512 clock cycles (18.7%)
……..

It took 274776 cycles to startup OpenSIPS on my device.

To decrease the latency after startup, we load all dynamic libraries at startup. Dynamic Libraries can be instructed to load at system startup by setting the environment variable  LD_BIND_NOW to 1.

  • # LD_BIND_NOW=1 LD_DEBUG=statistics ./opensips start

I got:

19211:     runtime linker statistics:
19211:       total startup time in dynamic loader: 308553 clock cycles
19211:                 time needed for relocation: 47668 clock cycles (15.4%)

…….

With LD_BIND_NOW=1 it took 308553 cycles on startup. They are nearly the same. This means OpenSIPS already load all dynamic libraries at startup. This eliminate the latency of loading after the startup while processing SIP messages. Good Job !

What actually happens is: When OpenSIPS parses the configuration file, it loads all the modules that are configured in the configuration file. The statement loadmodule (Module_Name) corresponds to loading the shared library named Module_Name.

After parsing, the configuration file is not needed any more.