OpenSIPS Management Interface (C Development)

.

Introduction

OpenSIPS Management Interface (MI)  is a mechanism which enables the external application (e.g. command line :opensipsctl , web: OpenSIPS-CP) to send commands (MI commands) to OpenSIPS. It is a pull based mechanism which means when you need information you need to query OpenSIPS (i.e. need to do something at a certain time). The MI command allows you to fetch/push data from/to OpenSIPS or trigger some actions there. The core has its own exported MI functions and the module has its own as well. Here you can find some examples of sending mi commands by the external applications “opensipsctl” and “OpenSIPS-CP”

Several transport protocols are used to carry MI commands and their replies between the external application and OpenSIPS. Each protocol is provided by separate OpenSIPS module. The current protocols are mi_fifo, mi_datagram, mi_xmlrpc, mi_http, mi_json, and mi_xmlrpc_ng. These modules require extra processes to avoid disturbing OpenSIPS main processes that are working with SIP.

The module must be loaded (i.e. configured to be loaded in the routing script) so its exported MI functions (exported by the module interface “exports”) are populated and can be called from the external application otherwise you will get an error message “500 command ‘Module_Name’ not available”. To be able to send MI command from the external application, the transport module also must be configured in the routing script to be loaded. For example if you want to connect to MI interface via FIFO file stream, the module MI_FIFO must be right configured and loaded. The same for the rest of transport protocols.

So two modules are needed to be able to call a specific MI function:

  • The module which exports the MI function.
  • The transport protocol which will transport the command to OpenSIPS.

The extra processes that are required for transport issues will be listening on different ports than the SIP ports. OpenSIPS can use multiple transport protocols at the same time (In the routing script, configure them to be loaded).

MI Command Syntax

If you are willing to write an external management application, you have to implement the transport you want to use. Your application will behave like a client which sends MI command in a specific format to OpenSIPS. These are the current formats:

MI Internal Structure

The following is the module_exports structure (Module Interface) defined in the file “sr_module.h” with the parts related to MI.

struct module_exports {.

……… /* Many Fields are Missed For Simplicity */

mi_export_t*     mi_cmds;           /* Array of the exported MI functions */

….

proc_export_t*  procs;     /* Array of the additional processes required by the module */

……

}

procs

Sometime the module needs extra processes like the transport protocol. So it exports

  • The number of required processes to be forked (“no” number).
  • The helper functions (pre_fork_function and post_fork_function) which help the attendant process to create the extra processes.
  • The function which will be executed by the extra processes. What will be done by these extra processes will not interfere with the rest of processes that handle the SIP.

The structure is defined in the file “sr_module.h” in OpenSIPS source directory:

typedef struct cmd_export_ cmd_export_t;

struct proc_export_ {
char *name;
mod_proc_wrapper pre_fork_function;
mod_proc_wrapper post_fork_function;
mod_proc function;
unsigned int no;
unsigned int flags;
};

typedef void (*mod_proc)(int no);

typedef int (*mod_proc_wrapper)();

The flags can be 0 or PROC_FLAG_INITCHILD. If PROC_FLAG_INITCHILD is provided, the function “child_init” from all modules will be run by the new extra processes.

Example of procs NULL terminated array:

static proc_export_t mi_procs[] = {
{“MI Datagram”,  pre_datagram_process,  post_datagram_process,
datagram_process, MI_CHILD_NO, PROC_FLAG_INITCHILD },
{0,0,0,0,0,0}
};
static param_export_t mi_params[] = {
{“children_count”,      INT_PARAM,    &mi_procs[0].no},…….}

mi_cmds

mi_cmds is an array of exported MI function (Type: mi_export_t*). The definition of the type “mi_export_t” is included in the file “mi/mi.h” as following:

typedef struct mi_export_ {
char *name;
char *help;
mi_cmd_f *cmd;
unsigned int flags;
void *param;
mi_child_init_f *init_f;
} mi_export_t;

name is the actual name of the MI function which will be called. help is the information about what this function is doing. cmd is a pointer to the actual mi function:

typedef struct mi_root* (mi_cmd_f)(struct mi_root*, void *param)

The return value is mi_root * (pointer to the root of the MI reply tree).  The first parameter is the MI root parameter(type: mi_root *) and the second parameter is the actual command parameter (void *). the param is the actual parameter of the MI function.

init_f is the child init function for the exported MI function. In some parts, some extra stuffs are needed to be done by the MI workers (processes that are handling the MI requests  like MI_FIFO proces, MI_XMLRPC process,….. This function will be called at the startup after OpenSIPS has been forked (will be called one time by each core process individually).

Example of MI functions NULL terminated array:

static mi_export_t mi_cmds[] = {
{ “mi_get_function“,”This Function is doing bla bla”,mi_get_function,MI_NO_INPUT_FLAG,0,0},
{0,0,0,0,0,0}
}

The string “mi_get_function” is the actual name of the mi command which will be named by when MI command is received and needed to be verified (lookup_mi_cmd(MethodName,..)). The actual function will be like this:

struct mi_root* mi_get_function(struct mi_root*  root, void * param){

……..

}

MI Function Reply Tree

The return value of MI function is a pointer to a tree (reply tree) . The root of this tree is pointer to struct mi_root. This structure is defined in the file “mi/tree.h”:

struct mi_root {
unsigned int  code;
str   reason;
struct mi_handler  * async_hdl;
struct mi_node node;
}

The code is the root code of the response (200 for sucess,500 for error, …) and the node is the starting node (type: mi_node defined in the same file “mi/tree.h”):

struct mi_node {
str value;
str name;
unsigned int flags;
struct mi_node *kids;
struct mi_node *next;
struct mi_node *last;
struct mi_attr *attributes;
}

To initialize MI reply tree in the previous MI function (mi_get_function), the function “init_mi_tree” can be called. To add child node, the function “add_mi_node_child“. To add attribute, the function “add_mi_attr” is called  and so on. Here you can find an example code where a reply tree is built when a MI function is called.

struct mi_root* mi_get_function(struct mi_root*  root, void * param){

………

struct mi_root * rpl_tree = init_mi_tree( 200, MI_SSTR(MI_OK));    /* 200 OK Reply */

……..

}

The transport process like datagram_process, xmlrpc_process,..etc  behaves as a server which accepts connections (UDP connections in case datagram transport , TCP connections in case xmlrpc connections and so on). To serve the MI request, the transport process calls functions from OpenSIPS core (Management Interface: “mi/mi.h”) in addition to some module-defined functions.

When the message is received, the transport process checks if the requested MI function is available (looks up the MI command):

struct mi_cmd*  fu =lookup_mi_cmd((char*)methodName, strlen(methodName))

The “lookup_mi_cmd” is defined in “mi/mi.h”. If “f==0”, the MI command is not available. Otherwise it parses the parameters of the requested MI function into a MI tree. For example in xmlrpc, the function “xr_parse_tree” is defined in the file “modules/mi_xmlrpc/xr_parser.h” and called in “modules/mi_xmlrpc/xr_server.c”. It returns a pointer to the reply tree (mi_root * t).

After this, the actual mi function will be called through the function “run_mi_cmd”. As i explained above the return value of the MI function (reply tree) has the type mi_root*. This is how it is called in xmlrpc transport module “”.

struct mi_root*  mi_rpl=run_mi_cmd( fu , t , (mi_flush_f*)xr_flush_response,env))

The function “run_mi_cmd” is defined in the file “mi/mi.h” and in its body you call the actual MI function as following:

static inline struct mi_root* run_mi_cmd(struct mi_cmd *cmd, struct mi_root * t, mi_flush_f *f, void *param){

struct mi_root *ret;

…..

ret = cmd->f( t, cmd->param);

…..

}

struct mi_cmd {
int id;
str module;
str name;
str help;
mi_child_init_f *init_f;
mi_cmd_f *f;
unsigned int flags;
void *param;
};

Then the reply tree mi_rpl  (Type: mi_root *) will be formatted corresponded to the transport process (transport protocol). For example if the protocol is xmlrpc, then the reply tree will be formatted in xml either as string that contains the name and attributes of each node OR as an array where each element contains a node information (node name and its attributes). Each attribute has name and value.

Finally, the response will be written in fifo file, UDP socket (datagram transport), TCP socket (xmlrpc transport) so it will be sent to the external application.

Test MI Function of a New Module

You can test the MI command by using opensipsctl as following:

# scripts/opensipsctl fifo mi_get_function


 Note

  • The OpenSIPS core MI exported functions (mi_core_cmds) are defined in the file “mi_core.c”.
  • The OpenSIPS Module MI exported functions (mi_cmds) are defind in the file modules/Module_Name/Module_Name.c
  • OpenSIPS has many transport modules. Go to “modules/mi_Transport“, Where Transport can be datagram, xmlrpc, json,…etc.

 More Information


Advertisements

SIP Stress Testing -Part 1 (Opensipsctl & OpenSIPS-CP)

.

What is Stress Testing ?

It is the process of putting so great load on the system and measuring its response. It helps to determine the maximum operating capacity of the system. The behaviour of the system will be predictable under high/peak loads.

I will mention some tools which help you in the testing of OpenSIPS SIP server under stress.

.

Using SIP traffic generator , we can highly load the SIP server and read the statistics. For example under certain high load, the number of dropped packets became unacceptable or the number of SIP transactions that are in use in memory is so large.

Opensipsctl Tool

It is shell script and it is used as management and monitoring tool. It has some options to generate some statistics. This script communicates with OpenSIPS server through either FIFO pipe or unix socket (The control engine is either fifo or unixsocket). The script is present in OpenSIPS sbin directory. Its configuration file opensipsctlrc is stored in OpenSIPS etc directory. Check the name of your fifo file in the configuration file. To display the available commands, type: #./opensipsctl fifo which

Here is a list of commands available in OpenSIPS 1.11.

get_statistics, reset_statistics, uptime, version, pwd, arg, which, ps, kill, debug, cache_store, cache_fetch, cache_remove, event_subscribe, events_list, subscribers_list, list_tcp_conns, help, list_blacklists, t_uac_dlg, t_uac_cancel, t_hash, t_reply, ul_rm, ul_rm_contact, ul_dump, ul_flush, ul_add, ul_show_contact, ul_sync, dlg_list, dlg_list_ctx, dlg_end_dlg, dlg_db_sync, dlg_restore_db, profile_get_size, profile_list_dlgs, profile_get_values, list_all_profiles.

  • For example to execute the command “ps”, type: #./opensipsctl fifo ps. This command display all  processes running by OpenSIPS. The output looks like:

Process::  ID=0 PID=9739 Type=attendant
Process::  ID=1 PID=9747 Type=MI FIFO
Process::  ID=2 PID=9749 Type=SIP receiver udp:10.0.0.9:5060
Process::  ID=3 PID=9751 Type=SIP receiver udp:10.0.0.9:5060
Process::  ID=4 PID=9753 Type=SIP receiver udp:10.0.0.9:5060
Process::  ID=5 PID=9755 Type=SIP receiver udp:10.0.0.9:5060
Process::  ID=6 PID=9756 Type=time_keeper
Process::  ID=7 PID=9757 Type=timer: tm-utimer
Process::  ID=8 PID=9760 Type=TCP receiver
Process::  ID=9 PID=9762 Type=TCP receiver
Process::  ID=10 PID=9764 Type=TCP receiver
Process::  ID=11 PID=9765 Type=TCP receiver
Process::  ID=12 PID=9768 Type=TCP main

  • To get the number of simultaneous calls (=dialogs in case no presence), type: #./opensipsctl fifo get_statistics dialog. Both SIP INVITE message (Establishing a Call) and SUBSCRIBE message (subscription to presence service) creates dialog. If the SIP server provides call management + presence,  the number of active dialogs will not be equal to the number of calls. The length of the call affects the memory of the SIP server not the processing because the dialog remains in the memory till one of the end parties (caller or calle) ends the dialog by sending SIP BYE message.
  • To take information about specific command, use “help” command with the name of the command as an argument. For example if you want to take information about  the command “ps”, type: #./opensipsctl fifo help ps. The output looks like:

Help:: lists all processes used by OpenSIPS
Exported by:: core

Note: Current opensipsctl does not support NoSQL databases.

OpenSIPS Control Panel (OpenSIPS-CP)

You can also use OpenSIPS control panel (OpenSIPS-CP) to do the same things. Open the web interface. Click on “System” –> “MI Commands”. Choose a command from the command list and press “Enter”. Dont forget the parameters.

OpenSIPS-CP Statistics Monitor Page

  • There are two tables needed for statistics monitoring in OpenSIPS-CP. Do the following to download the structures of monitored_stats and monitoring_stats tables in mysql opensips database:

#cd /var/www/opensips-cp/

#mysql -Dopensips -p < config/tools/system/smonitor/tables.mysql

  • On the web interface, click on “System” –> “Statistics Monitor”. You will get the following:

omslag1

For example you can get some real time statistics about transaction module (TM)  by clicking on the corresponding item in the list “Module:tm”. The “inuse_transaction” sub-item gives the number of transactions in use in memory. The transactions should be released quickly and the average value of this number should be stable (transactions created and released quickly). The number of  transactions per second defines the performance of SIP proxy.

Also you can get statistics on the number of TCP/UDP packets which waiting to be processed. By clicking on “Module:net” item, you can see “waiting_udp” and “waiting_tcp” sub-items.

  • Edit the cron_job/get_opensips_stats.php file and check the value of the variable $path_to_smonitor if it is correct according to your opensips-cp.
  • Add a cron job that collects data from the opensips:

# crontab -e

Append the following entry:

* * * * *   root   php /var/www/opensips-cp/cron_job/get_opensips_stats.php > /dev/null

Note

The MI exported functions can be called only if the corresponded modules are loaded.  Otherwise you will get an error message (command not available). For example enabling tracing by calling sip_trace (# ./opensipsctl fifo sip_trace on) will not work if the module “siptrace.so” is not loaded (i.e. configured to be loaded in the configuration file). You will get this error message “500 command ‘sip_trace’ not available”.

Module Not loaded –> No Exported Functions –> Error upon calling “not exist” Function


Next

The Next Part will be about SIPSAK command line tool which is used as a SIP traffic generator to test the performance and the security of the SIP servers or user agents.