SBC_Setup
About
This document covers information about the SBC Setup.
Click here to expand Table of Contents
- 1 Introduction
- 2 Preliminaries
- 3 Installation of FreeSWITCH
- 4 Installation of Kamailio
- 5 Configuration of FreeSWITCH
- 6 Test connectivity between FreeSWITCH and Kamailio
- 7 Optimizations
- 8 Check CPU usage
- 9 Links to Kamailio and carrierroute
Introduction
Below you'll find a step by step setup for installing FS as a SBC. The LCR engine is provided by Kamailio and its module carrierroute. Kamailio is an opensource SIP Proxy (not a B2BUA).
In this setup, the dialplan is detailed only for inbound to outbound traffic, but it could be easily extended for outbound to inbound traffic (or DID).
The design is the following:
FS is configured with an internal and an external profile, each profile listening on a different network interface.
Kamailio is listening on the loopback interface, and is not used by any other process than FS.
Internal--------FS--------External
|
|
Kamailio
When FS receives an INVITE, FS fires this event in the default context, where it always matches the extension named LOOKUP_ROUTE. In this extension, FS sends the INVITE to Kamailio, that will replies with a 302 Redirect SIP message that contains the route FS has to use to reach the number dialed. The call is then transfered in the context ROUTING where FS will match the associated Gateway (in this setup it is an associated code) that will route the call.
If Kamailio has no route for the dialed number, it replies with a "604 unable to route the call" that FS will relay.
So Kamailio is used only for route lookup by FS, and FS is the only equipment that is seen in the SIP transaction.
This design has been choosen cause I needed a way to store big LCR table (it's now more than 450 000 routes on the production server) and LCR module was not available when I began playing with FS (version 1.0.1).
I extensively tested this setup on an Intel Quad Core server (Q9550 running at 2.83Ghz with 8GB RAM. 4GB should have been far enough).
I ran some tests using SIPP with 750 simultaneous calls and between 50 to 100 calls per second (CPS) for more than 10 days with 25%-30% CPU ressource free. FS version was 1.2 at this time.
Advantages
Stable with quite huge routing table: Kamailio was using less than 3% CPU when doing the SIPP tests with 160 000 entries in LCR table.
Scalable: some people are using carrierroute module with LCR table up to 1 million entries.
Correct call per second rate: up to 100cps.
Drawbacks
Each INVITE you send to FS will first send an INVITE to Kamailio, and once FS receives the Kamailio answer, FS will bridge the call to the Peer that will process the call.
I'm feeling that if you want to process more call per seconds (more than 100 CPS), the fact that the Sofia library is monothreaded and that it has to deal with one more session (FS to Kamailio) before processing the call to the Peer could prevent FS to reach a higher CPS rate. (For everybody that has a better internal knowledge of FS than me, feel free to correct this assumption).
Other ways to go to achieve the same (not tested)
Try mod_lcr that has been introduced in FS 1.0.3 and report on the list and Wiki.
Use the XML CURL module to process the call routing using your preferred script language running on yout HTTP server.
Preliminaries
Install Debian stable netinstall (64 bits version, take care you need a 64bit processor)
Upgrade debian distribution
apt-get update
apt-get dist-upgrade
Install SSH Server for remote administration
apt-get install openssh-server
Install Vlan packages
apt-get install vlan
Install Bridge packages
apt-get install bridge-utils
Install Packet Capture packages
apt-get install ngrep
apt-get install tshark
apt-get install tcpdump
Install text editor
apt-get install emacs22-nox
Install screen
apt-get install screen
Disable useless services
/etc/init.d/exim4 stop
update-rc.d -f exim4 remove
/etc/init.d/portmap stop
update-rc.d -f portmap remove
/etc/init.d/nfs-common stop
update-rc.d -f nfs-common remove
Restart server
reboot
Check TCP or UDP services running
netstat -tnlp
only SSH (port 22) should be listening
netstat -unlp
only dhclient3 (port 68) should be listening (I need dhcp for my tests and to get an IP for administration of the test server)
Install utils for compilation
apt-get install build-essential
Installation of FreeSWITCH
Install dependencies
apt-get install subversion subversion-tools automake1.9 gcc-4.1 autoconf make wget libtool g++ libncurses5 libncurses5-dev
apt-get install debhelper automake1.9 autoconf libtool unixodbc-dev libasound2-dev libcurl4-openssl-dev libssl-dev libogg-dev libvorbis-dev libperl-dev libgdbm-dev libdb-dev libgnutls-dev libspandsp-dev libtiff4-dev
Download FreeSWITCH latest Git and put them in /opt/sources/freewitch
cd /opt/sources/
git clone git://git.freeswitch.org/freeswitch.git freeswitch-git
cd freeswitch-git
./bootstrap.sh
Edit modules.conf depending on your needs
./configure -prefix=/opt/freeswitch-git --enable-core-odbc-support
make
make install
make sounds-install
make moh-install
To upgrade to latest Git
cd /opt/sources/freeswitch-git
make current
and that's it you get the latest Git in /opt/freeswitch-git
Prepare Network interface (I'm using a switch with vlans, useful for network isolation)
* Load linux vlan module
modprobe 8021q
* add 2 vlans (one for internal profile ex: vlan 100, the other for external profile ex: vlan 200)
vconfig add eth0 100
vconfig add eth0 200
* bring these interfaces up
ip link set up dev eth0.100
ip link set up dev eth0.200
* bring them IP
ip addr add 10.10.10.254/24 dev eth0.100
ip addr add 10.10.20.254/24 dev eth0.200
Edit sip_profiles to use these IP
cd /opt/freeswitch/conf/sip_profiles
edit internal.xml and replace $${local_ip_v4} with internal IP (10.10.10.254)
edit external.xml and replace $${local_ip_v4} and $${external_rtp_ip} with external IP (10.10.20.254)
Optimizations before launching FS
ulimit -c unlimited
ulimit -d unlimited
ulimit -f unlimited
ulimit -i unlimited
ulimit -n 999999
ulimit -q unlimited
ulimit -u unlimited
ulimit -v unlimited
ulimit -x unlimited
ulimit -s 240
ulimit -l unlimited
ulimit -a
Start FreeSWITCH
/opt/freeswitch/bin/freeswitch -nc -hp
-nc stands for no console
-hp stands for high priority
Connect on FreeSWITCH via fs_cli
/opt/freeswitch/bin/fs_cli
with this tool you'll be able to get a console connection to FS
the log level has not been changed so it could be very verbose
to disable log, type /nolog in console
Check if FS is running
netstat -unlp
you should have this
udp 0 0 10.10.10.254:5060 0.0.0.0:* 9240/freeswitch
udp 0 0 10.10.20.254:5060 0.0.0.0:* 9240/freeswitch
Installation of Kamailio
Installation of Bison, a parser generator
apt-get install bison
Installation of Flex, a fast lexical analyzer generator
apt-get install flex
Installation of MySQL and MySQL dev libraries
apt-get install mysql-server-5.0
You'll be prompt to set the password of your database, let's test with password "1234" or whatever you want
apt-get install libmysql++-dev
Installation of libconfuse, needed for carrierroute module compilation
apt-get install libconfuse-dev
Get sources of Kamailio and put them in /opt/sources
cd /opt/sources
wget http://www.kamailio.org/pub/kamailio/1.5.0/src/kamailio-1.5.0-notls_src.tar.gz
tar -xvzf kamailio-1.5.0-notls_src.tar.gz
cd kamailio-1.5.0-notls
If you need a LCR route table of approximatively 50 000 routes, then bypass this step.
For those who'd like to load a bigger LCR route table in carrierroute (for my needs: 160 000 routes)
emacs /opt/sources/kamailio-1.5.0-notls/config.h
#define PKG_MEM_POOL_SIZE 32*1024*1024 /*!< Used only if PKG_MALLOC is defined*/
#define SHM_MEM_SIZE 4*32 /*!< Used if SH_MEM is defined*/
Note: these values have been set by empirical means and are maybe too high even for 160 000 routes
Compile Kamailio in /opt/kamailio-1.5.0
make prefix=/opt/kamailio-1.5.0
make prefix=/opt/kamailio-1.5.0 modules=modules/db\_mysql modules
make prefix=/opt/kamailio-1.5.0 modules=modules/carrierroute modules
make prefix=/opt/kamailio-1.5.0 install
make prefix=/opt/kamailio-1.5.0 modules=modules/carrierroute install
make prefix=/opt/kamailio-1.5.0 modules=modules/db\_mysql install
Add a symbolic link for /opt/kamailio-1.5.0
ln -sf /opt/kamailio-1.5.0/ /opt/kamailio
Edit Kamailio Database Settings
cd /opt/kamailio/etc/kamailio/
emacs kamctlrc
# this parameter...
DBENGINE=MYSQL
## database host
DBHOST=localhost
## database name (for ORACLE this is TNS name)
DBNAME=openser
# database path used by dbtext or db_berkeley
# DB_PATH="/usr/local/etc/kamailio/dbtext"
## database read/write user
DBRWUSER=openser
## password for database read/write user
DBRWPW="1234"
## database read only user
DBROUSER=openserro
## password for database read only user
DBROPW="1234"
## database super user (for ORACLE this is 'scheme-creator' user)
DBROOTUSER="root"
# user name column
USERCOL="username"
Then launch Kamailio utils to create database
/opt/kamailio/sbin/kamdbctl create
You'll be prompted for root sql password (remember you used 1234 or ...) and don't forget to set yes for carrierroute table creation
Populate carrierroute database with some routes
Connect to mysql
mysql -u root -p
Enter your root password
In mysql console:
use openser;
INSERT INTO carrier_name (carrier) VALUES ("default");
INSERT INTO domain_name (domain) VALUES ("default");
INSERT INTO carrierroute (carrier,domain,scan_prefix,flags,mask,prob,strip,rewrite_host,rewrite_prefix,rewrite_suffix,description) VALUES("1","1","1000","0","0","1","0","PEER-01","","","FRANCE");
exit
You have inserted your first route, that will match prefix 1000 and route this to PEER_01 with a description, FRANCE in this example. PEER_01 will be defined later in FS dialplan. The description parameter could be exported, for billing purpose for example. In this setup, I'll export the description to an AREA variable in FS.
Configuration of Kamailio
cd /opt/kamailio/etc/kamailio
mv kamailio.cfg kamailio.cfg.default
emacs kamailio.cfg
and put the config below:
# KAMAILIO basic configuration script for use with carrierroute and FS
####### Global Parameters #########
debug=1
log_stderror=no
log_facility=LOG_LOCAL0
fork=yes
children=8
/* uncomment the next line to disable TCP (default on) */
disable_tcp=yes
/* uncomment the next line to disable the auto discovery of local aliases
based on revers DNS on IPs (default on) */
auto_aliases=no
/* uncomment and configure the following line if you want opensips to
bind on a specific interface/port/proto (default bind on all available) */
listen=udp:127.0.0.1:5062
####### Modules Section ########
#set module path
mpath="/opt/kamailio/lib64/kamailio/modules/"
/* uncomment next line for MySQL DB support */
loadmodule "db_mysql.so"
loadmodule "sl.so"
loadmodule "tm.so"
loadmodule "rr.so"
loadmodule "maxfwd.so"
loadmodule "textops.so"
loadmodule "mi_fifo.so"
loadmodule "xlog.so"
loadmodule "carrierroute.so"
loadmodule "pv.so"
loadmodule "avpops.so"
#----------------- setting module-specific parameters ---------------
# ----- mi_fifo params -----
modparam("mi_fifo", "fifo_name", "/tmp/kamailio_fifo")
# ------ Carrierroute ------
modparam("carrierroute", "config_source", "db")
modparam("carrierroute", "db_url", "mysql://openser:1234@localhost/openser")
modparam("carrierroute", "carrierroute_table", "carrierroute")
modparam("carrierroute", "default_tree", "default")
modparam("carrierroute", "fetch_rows", 2000)
####### Routing Logic ########
# main request routing logic
route{
if (!mf_process_maxfwd_header("10")) {
sl_send_reply("483","Too Many Hops");
exit;
}
t_check_trans();
if ($rU==NULL) {
# request with no Username in RURI
sl_send_reply("484","Address Incomplete");
exit;
}
# LOOKUP ROUTE TABLE WHEN ASKED BY HEADER: X-ROUTE:LOOKUP
if (is_method("INVITE") && $hdr(X-ROUTE)=="LOOKUP"){
if(!cr_route("default", "default", "$rU", "$rU", "call_id","$avp(s:route_desc)")){
#xlog("ROUTING FAILED: no route found for $rU");
sl_send_reply("604", "Unable to route this call");
exit;
} else {
#xlog("LOOKUP FOUND: $rd $avp(s:route_desc)");
avp_pushto("$ru/username", "$avp(s:route_desc)");
sl_send_reply("302", "$rd");
exit;
}
}
}
Start Kamailio
/opt/kamailio/sbin/kamctl start
Check Kamailio is running
netstat -unlp
You should see something like this
udp 0 0 127.0.0.1:5062 0.0.0.0:* 16269/kamailio
Check your route table
Seems broken in Kamailio 1.5.0, check SQL table
/opt/kamailio/sbin/kamctl cr show
You should see a route with PEER_01
Configuration of FreeSWITCH
Dialplan configuration for FS
cd /opt/freeswitch/conf/dialplan
emacs default.xml
Put this
<?xml version="1.0" encoding="utf-8"?>
<!-- http://wiki.freeswitch.org/wiki/Dialplan_XML -->
<include>
<context name="default">
<extension name="LOOKUP_ROUTE">
<condition field="destination_number" expression="(\d+)$">
<action application="set" data="hangup_after_bridge=true"/>
<action application="set" data="continue_on_fail=true"/>
<action application="export" data="sip_h_X-ROUTE=LOOKUP"/>
<action application="bridge" data="sofia/internal/${destination_number}@127.0.0.1:5062"/>
<action application="set" data="ROUTE_GW=${sip_redirect_contact_host_0}"/>
<action application="set" data="AREA=${sip_redirect_contact_user_0}"/>
<action application="transfer" data="${destination_number} XML ROUTING"/>
</condition>
</extension>
</context>
<context name="ROUTING">
<extension name="PEER_01">
<condition field="${ROUTE_GW}" expression="PEER_01">
<action application="set" data="hangup_after_bridge=true"/>
<action application="set" data="continue_on_fail=NORMAL_TEMPORARY_FAILURE,TIMEOUT,NO_ROUTE_DESTINATION"/>
<action application="set" data="PEER=1.1.1.1"/>
<action application="bridge" data="sofia/external/${destination_number}@1.1.1.1"/>
<action application="set" data="PEER=2.2.2.2"/>
<action application="bridge" data="sofia/external/${destination_number}@2.2.2.2"/>
<action application="set" data="PEER=3.3.3.3"/>
<action application="bridge" data="sofia/external/${destination_number}@3.3.3.3"/>
</condition>
</extension>
</context>
</include>
In fact PEER_01 is used as an index for a combination of peers. I do this to have a failover mechanism to a backup peer if the 1st peer is down.
You can load balance to different PEER combinations by using probability in carrierroute table. Ex: using 2 routes pointing to PEER_01 and PEER_02 with a probability of 0.5 and you will share the load between these 2 peers combination.
Test connectivity between FreeSWITCH and Kamailio
Test that FS and Kamailio are talking to each others
Register an IP phone using credentials in /opt/freeswitch/conf/directory/default/1001 for example
to dump traffic on the loopback interface for SIP messages between FS and Kamailio
ngrep -d lo -nn -W byline
When your phone is registered, dial 1000
You should see this in the ngrep trace
U 10.10.10.254:5060 -> 127.0.0.1:5062
INVITE sip:1000@127.0.0.1:5062 SIP/2.0.
Via: SIP/2.0/UDP 10.10.10.254;rport;branch=z9hG4bKZjpQ3tQ8SN7gc.
Max-Forwards: 69.
From: "1001" <sip:1001@10.10.10.254>;tag=pZ4egZ0m43Q3c.
To: <sip:1000@127.0.0.1:5062>.
Call-ID: 7015d499-6c86-122c-80be-001b21009ccd.
CSeq: 110711449 INVITE.
Contact: <sip:mod_sofia@10.10.10.254:5060>.
User-Agent: FreeSWITCH-mod_sofia/1.0.trunk-11592M.
Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, PRACK, MESSAGE, SUBSCRIBE, NOTIFY, REFER, UPDATE, REGISTER, INFO.
Supported: timer, precondition, path, replaces.
Allow-Events: talk, refer.
Min-SE: 120.
Content-Type: application/sdp.
Content-Disposition: session.
Content-Length: 333.
X-ROUTE: LOOKUP.
Remote-Party-ID: "1001" <sip:1001@10.10.10.254>;screen=yes;privacy=off.
.
v=0.
o=FreeSWITCH 3223313286554167397 4405097820346134292 IN IP4 10.10.10.254.
s=FreeSWITCH.
c=IN IP4 10.10.10.254.
t=0 0.
m=audio 31686 RTP/AVP 9 0 8 3 101 13.
a=rtpmap:9 G722/8000.
a=rtpmap:0 PCMU/8000.
a=rtpmap:8 PCMA/8000.
a=rtpmap:3 GSM/8000.
a=rtpmap:101 telephone-event/8000.
a=fmtp:101 0-16.
a=rtpmap:13 CN/8000.
a=ptime:20.
#
U 127.0.0.1:5062 -> 10.10.10.254:5060
SIP/2.0 302 PEER_01.
Via: SIP/2.0/UDP 10.10.10.254;rport=5060;branch=z9hG4bKZjpQ3tQ8SN7gc.
From: "1001" <sip:1001@10.10.10.254>;tag=pZ4egZ0m43Q3c.
To: <sip:1000@127.0.0.1:5062>;tag=458fb4012080e656b6742c09466dabcd.1740.
Call-ID: 7015d499-6c86-122c-80be-001b21009ccd.
CSeq: 110711449 INVITE.
Contact: sip:1000@PEER_01.
Server: Kamailio (1.4.3-notls (x86_64/linux)).
Content-Length: 0.
.
#
U 10.10.10.254:5060 -> 127.0.0.1:5062
ACK sip:1000@127.0.0.1:5062 SIP/2.0.
Via: SIP/2.0/UDP 10.10.10.254;rport;branch=z9hG4bKZjpQ3tQ8SN7gc.
Max-Forwards: 69.
From: "1001" <sip:1001@10.10.10.254>;tag=pZ4egZ0m43Q3c.
To: <sip:1000@127.0.0.1:5062>;tag=458fb4012080e656b6742c09466dabcd.1740.
Call-ID: 7015d499-6c86-122c-80be-001b21009ccd.
CSeq: 110711449 ACK.
Content-Length: 0.
.
Optimizations
Stop FreeSWITCH
/opt/freeswitch/bin/freeswitch -stop
Lowering FS Log Level
emacs /opt/freeswitch/conf/autoload_configs/switch.conf.xml
<!-- Default Global Log Level - value is one of debug,info,notice,warning,err,crit,alert -->
<param name="loglevel" value="err"/>
Set Call Admission Control
emacs /opt/freeswitch/conf/autoload_configs/switch.conf.xml
<!--Most channels to allow at once -->
<param name="max-sessions" value="1000"/>
<!--Most channels to create per second -->
<param name="sessions-per-second" value="100"/>
With these settings, we have defined a maximum of 500 bridged calls (2*500 channels) and a maximum of 100 new sessions per second. You should set these parameters according to your hardware
Creation of a ramdisk for the FS database
mv /opt/freeswitch/db /opt/freeswitch/db_old
mkdir /opt/freeswitch/db
mount -t tmpfs tmpfs /opt/freeswitch/db
Use Proxy Media to keep FS in the RTP media path (topology hiding) but without analyzing RTP or DTMF
emacs /opt/freeswitch/conf/sip_profiles/internal.xml
<param name="inbound-late-negotiation" value="true"/>
Add directive proxy media in dialplan
<extension name="PEER_01">
<condition field="${sip_h_X-ROUTE}" expression="PEER_01">
<action application="set" data="proxy_media=true"/>
<action application="set" data="hangup_after_bridge=true"/>
<action application="set" data="continue_on_fail=NORMAL_TEMPORARY_FAILURE,TIMEOUT,NO_ROUTE_DESTINATION"/>
<action application="set" data="PEER=1.1.1.1"/>
<action application="bridge" data="sofia/external/${destination_number}@1.1.1.1"/>
<action application="set" data="PEER=2.2.2.2"/>
<action application="bridge" data="sofia/external/${destination_number}@2.2.2.2"/>
<action application="set" data="PEER=3.3.3.3"/>
<action application="bridge" data="sofia/external/${destination_number}@3.3.3.3"/>
</condition>
</extension>
Disable Presence Support in SIP_Profiles
emacs /opt/freeswitch/conf/sip_profiles/internal.xml
<param name="manage-presence" value="false"/>
emacs /opt/freeswitch/conf/sip_profiles/external.xml
<param name="manage-presence" value="false"/>
Check that your Network Interfaces Cards (NIC) are not sharing IRQ
cat /proc/interrupts
ideally, your ethernet cards should be put on different IRQ, and if you have more than one CPU, you can force affinity of a NIC with a specific CPU (I will add a note later on this)
Check CPU usage
Install some useful utils for checking CPU consumptions
apt-get install sysstat
You can check the load of your CPUs using
mpstat -P ALL 1
13:13:13 CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
13:13:14 all 0,00 0,00 0,00 0,00 0,00 0,00 0,00 100,00 162,00
13:13:14 0 0,00 0,00 0,00 0,00 0,00 0,00 0,00 100,00 162,00
The %idle represents the idle percentage of the CPU, higher is better as it means that you have free CPU ressources
vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 80 234312 93496 1273928 0 0 1 32 55 65 2 3 95 0
0 0 80 234296 93496 1273928 0 0 0 0 106 1299 0 0 100 0
The last 2 columns are interesting:
id: idle, same as before for mpstat, the higher is better
wa: waiting IO, when this number is high it means that your CPU is waiting for IO (ex: HDD with too much access) and it will have a negative impact on performance
top
I think that I do not have to introduce this tool