- Spelling checked
[sip-router] / doc / serdev / serdev.sgml
1 <!-- $Id$ -->
2 <!DOCTYPE Book PUBLIC "-//OASIS//DTD DocBook V4.2//EN">
3
4 <book label="serdev" id="serdev" lang="EN">
5 <?dbhtml filename="index.html">
6     <title>SIP Express Router v0.8.8 - Developer's Guide</title>
7     <bookinfo>
8         <authorgroup>
9             <author>
10                 <firstname>Jan</firstname>
11                 <surname>Janak</surname>
12                 <email>J.Janak@sh.cvut.cz</email>
13             </author>
14             <author>
15                 <firstname>Jiri</firstname>
16                 <surname>Kuthan</surname>
17                 <email>jiri@iptel.org</email>
18             </author>
19             <author>
20                 <firstname>Bogdan</firstname>
21                 <surname>Iancu</surname>
22                 <email>Bogdan.Iancu@fokus.fraunhofer.de</email>
23             </author>
24         </authorgroup>
25         <copyright>
26             <year>2001</year>
27             <year>2002</year>
28             <holder>FhG Fokus</holder>
29         </copyright>
30         <legalnotice>
31             <para>
32                 This documentation is free software; you can redistribute
33                 it and/or modify it under the terms of the GNU General Public
34                 License as published by the Free Software Foundation; either
35                 version 2 of the License, or (at your option) any later
36                 version.
37             </para>
38             
39             <para>
40                 This program is distributed in the hope that it will be
41                 useful, but WITHOUT ANY WARRANTY; without even the implied
42                 warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
43                 See the GNU General Public License for more details.
44             </para>
45             
46             <para>
47                 You should have received a copy of the GNU General Public
48                 License along with this program; if not, write to the Free
49                 Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
50                 MA 02111-1307 USA
51             </para>
52             
53             <para>
54                 For more details see the file COPYING in the source
55                 distribution of SER.
56             </para>         
57         </legalnotice>
58
59         <abstract>
60             <para>
61                 The document describes the SIP Express Router internals and algorithms. It describes
62                 overall server architecture, request processing, configuration, memory management,
63                 interprocess locking, module interface and selected modules in detail.
64             </para>
65
66             <para>
67                 The document is intended mainly for module developers wishing to implement
68                 a new module for the server. Other people like developers of SIP related
69                 software or students might be interested too.
70             </para>
71         </abstract>
72
73     </bookinfo>
74     
75     <toc></toc>
76     
77     <chapter id="server-startup">
78         <title>The Server Startup</title>
79         <para>
80             The <function moreinfo="none">main</function> function in file <filename moreinfo="none">main.c</filename>
81             is the first function called upon server startup. It's purpose is to initialize the server and
82             enter main loop. The server initialization will be described in the following sections.
83         </para>
84         
85         <para>
86             Particular initialization steps are described in order in which they appear in <function>main</function> function.
87         </para>
88         
89         <section id="signal-installation">
90             <title>Installation Of New Signal Handlers</title>
91             <para>
92                 The first step in the initialization process is the installation of new signal handlers. We need
93                 our own signal handlers to be able to do graceful shutdown, print server statistics and so on. There is
94                 only one signal handler function which is function <function moreinfo="none">sig_usr</function> in
95                 file <filename moreinfo="none">main.c</filename>.
96             </para>
97             <para>
98                 The following signals are handled by the function: SIGINT, SIGPIPE, SIGUSR1, SIGCHLD, SIGTERM, SIGHUP and
99                 SIGUSR2.
100             </para>
101         </section> <!-- signal-installation -->
102         
103         <section id="cmd-line-params">
104             <title>Processing Command Line Parameters</title>
105             <para>
106                 <acronym>SER</acronym> utilizes the <function moreinfo="none">getopt
107                 </function>function to parse command line parameters.
108                 The function is extensively described in the man pages.
109             </para>
110         </section> <!-- cmd-line-params -->
111         
112         <section id="parser-init">
113             <title>Parser Initialization</title>
114             <para>
115                 <acronym>SER</acronym> contains a fast 32-bit parser. The parser uses pre-calculated hash table that needs to be
116                 filled in upon startup. The initialization is done here, there are two functions that do the
117                 job. Function <function moreinfo="none">init_hfname_parser</function> initializes hash table
118                 in header field name parser and function <function moreinfo="none">init_digest_parser</function>
119                 initializes hash table in digest authentication parser. The parser's internals will be described
120                 later.
121             </para>
122         </section> <!-- parser-init -->
123         
124 <!--
125         <section id="hash-init">
126             <title>Hash Table Initialization</title>
127             <para>TBD.</para>
128         </section>
129 -->
130         
131         <section id="malloc-init">
132             <title>Malloc Initialization</title>
133             <para>
134                 To make <acronym>SER</acronym> even faster we decided to re-implement memory allocation routines. The new 
135                 <function moreinfo="none">malloc</function>
136                 better fits our needs and speeds up the server a lot. The memory management subsystem needs
137                 to be initialized upon server startup. The initialization mainly creates internal data
138                 structures and allocates memory region to be partitioned.
139             </para>
140             
141             <important>
142                 <para>
143                     The memory allocation code must be initialized <emphasis>BEFORE</emphasis> any of its function is called !
144                 </para>
145             </important>
146         </section> <!-- malloc-init -->
147         
148         <section id="timer-init">
149             <title>Timer Initialization</title>
150             <para>
151                 Various subsystems of the server must be called periodically regardless of the incoming
152                 requests. That's what timer is for. Function <function moreinfo="none">init_timer</function>
153                 initializes the timer subsystem. The function is called from <filename moreinfo="none">main.c</filename>
154                 and can be found in <filename moreinfo="none">timer.c</filename> The timer subsystem will
155                 be described later.
156             </para>
157             <warning>
158                 <para>
159                     Timer subsystem must be initialized before config file is parsed !
160                 </para>
161             </warning>
162         </section> <!-- timer-init -->
163
164         <section id="fifo-init">
165             <title><acronym>FIFO</acronym> Initialization</title>
166             <para>
167                 <acronym>SER</acronym> has built-in support for <acronym>FIFO</acronym> control. It means that the 
168                 running server can accept
169                 commands over a FIFO special file (a named pipe). Function <function moreinfo="none">register_core_fifo</function> 
170                 initializes <acronym>FIFO</acronym> subsystem and registers basic commands, that are processed by the core
171                 itself. The function can be found in file <filename moreinfo="none">fifo_server.c</filename>.
172             </para>
173             <para>
174                 The <acronym>FIFO</acronym> server will be described in another chapter.
175             </para>
176         </section> <!-- fifo-init -->
177         
178         <section id="builtin-mod-reg">
179             <title>Built-in Module Initialization</title>
180             <para>
181                 Modules can be either loaded dynamically at runtime or compiled in statically. When a module
182                 is loaded at runtime, it is registered
183                 <footnote id="regfoot">
184                     <para>
185                         Module registration is a process when the core tries to find what functions and
186                         parameters are offered by the module.
187                     </para>
188                 </footnote>
189                 immediately with the 
190                 core. When the module is compiled in statically, the registration<footnoteref linkend="regfoot"> 
191                 must be performed during the server startup. Function 
192                 <function moreinfo="none">register_builtin_modules</function> does the job.
193             </para>
194         </section> <!-- builtin-mod-reg -->
195
196         <section id="ser-config">
197             <title>Server Configuration</title>
198             <para>
199                 The server is configured through a configuration file. The configuration file
200                 is C-Shell like script which defines how incoming requests should be processed.
201                 
202                 The file cannot be interpreted directly because that would be very slow. Instead
203                 of that the file is translated into an internal binary representation. The process
204                 is called compilation and will be described in the following sections.
205             </para>
206             <note>
207                 <para>
208                     The following sections only describe how the internal binary representation is
209                     being constructed from the config file. The way how the binary representation is used
210                     upon a request arrival will be described later.
211                 </para>
212             </note>
213             <para>The compilation can be divided in several steps:</para>
214             
215             <section id="lex-analysis">
216                 <title>Lexical Analysis</title>
217                 <para>
218                     Lexical analysis is process of converting the input (the configuration file in this
219                     case) into a stream of tokens. A token is a set of characters that
220                     'belong' together. A program that can turn the input into stream of tokens is
221                     called scanner. For example, when scanner encounters a number in the config
222                     file, it will produce token NUMBER.
223                 </para>
224                 <para>
225                     There is no need to implement the scanner from scratch, it can be done automatically.
226                     There is a utility called flex. Flex accepts a configuration file and generates
227                     scanner according to the configuration file. The configuration file for flex consists
228                     of several lines - each line describing one token. The tokens are described using
229                     regular expressions. For more details, see flex manual page or info documentation.
230                 </para>
231                 <para>
232                     Flex input file for the <acronym>SER</acronym> config file is in file 
233                     <filename moreinfo="none">cfg.lex</filename>. The file is processed by flex when the
234                     server is being compiled and the result is written in file 
235                     <filename moreinfo="none">lex.yy.c</filename>. The output file contains the scanner
236                     implemented in the C language.
237                 </para>
238             </section> <!-- lex-analysis -->
239             
240             <section id="syntax-analysis">
241                 <title>Syntactical Analysis</title>
242                 <para>
243                     The second stage of configuration file processing is called syntactical analysis. Purpose
244                     of syntactical analysis is to check if the configuration file has been well formed, doesn't
245                     contain syntactical errors and perform various actions at various stages of the analysis.
246                     Program performing syntactical analysis is called parser.
247                 </para>
248                 <para>
249                     Structure of the configuration file is described using grammar. Grammar is a set of rules
250                     describing valid 'order' or 'combination' of tokens. If the file isn't conformable with it's
251                     grammar, it is syntactically invalid and cannot be further processed. In that case an error
252                     will be issued and the server will be aborted.
253                 </para>
254                 <para>
255                     There is a utility called yacc. Input of the utility is a file containing the grammar of
256                     the configuration file, in addition to the grammar, you can describe what action the parser
257                     should do at various stages of parsing. For example, you can instruct the parser to create a
258                     structure describing an <acronym>IP</acronym> address every time it finds an <acronym>IP</acronym> 
259                     address in the configuration file and convert the address to its binary representation.
260                 </para>
261                 <para>For more information see yacc documentation.</para>
262                 <para>
263                     yacc creates the parser when the server is being compiled from the sources.
264                     Input file for yacc is <filename moreinfo="none">cfg.y</filename>. The file contains grammar
265                     of the config file along with actions that create the binary representation of the file.
266                     Yacc will write its result into file <filename moreinfo="none">cfg.tab.c</filename>. The
267                     file contains function <function moreinfo="none">yyparse</function> which will parse the whole
268                     configuration file and construct the binary representation. For more information about the
269                     bison input file syntax see bison documentation.
270                 </para>
271             </section> <!-- syntax-analysis -->
272
273             <section id="config-file-struct">
274                 <title>Config File Structure</title>
275                 <para>
276                     The configuration file consist of three sections, each of the sections will be
277                     described separately.
278                 </para>
279                 <itemizedlist>
280                     <listitem>
281                         <para>
282                             <emphasis>Route Statement</emphasis> - The statement describes how incoming requests 
283                             will be processed.
284                             When a request is received, commands in one or more <quote>route</quote> sections will be executed
285                             step by step. The config file must always contain one main <quote>route</quote> statement and 
286                             may contain several additional <quote>route</quote> statements. Request processing always starts 
287                             at the beginning
288                             of the main <quote>route</quote> statement. Additional <quote>route</quote> statements can be 
289                             called from the main
290                             one or another additional <quote>route</quote> statements (It it similar to function calling).
291                         </para>
292                     </listitem>
293                     <listitem>
294                         <para>
295                             <emphasis>Assign Statement</emphasis> - There are many configuration variables across 
296                             the server and this
297                             statement makes it possible to change their value. Generally it is a list of assignments,
298                             each assignment on a separate line.
299                         </para>
300                     </listitem>
301                     <listitem>
302                         <para>
303                             <emphasis>Module Statement</emphasis> - Additional functionality of the server is 
304                             available through separate
305                             modules. Each module is a shared object that can be loaded at runtime. Modules can
306                             export functions, that can be called from the configuration file and variables, that
307                             can be configured from the config file. The module statement makes it possible to load
308                             modules and configure them. There are two commands in the statement - 
309                             <function moreinfo="none">loadmodule</function> and <function moreinfo="none">modparam</function>. 
310                             The first can load a module. The second one can configure module's internal
311                             variables.
312                         </para>
313                     </listitem>
314                 </itemizedlist>
315                 <para>
316                     In the following sections we will describe in detail how the three sections are being processed 
317                     upon server startup.
318                 </para>
319
320                 <section id="route-stm">
321                     <title>Route Statement</title>
322                     <para>The following grammar snippet describes how the route statement is constructed</para>
323                     <programlisting format="linespecific">
324 route_stm = "route" "{" actions "}"  
325
326     $$ = push($3, &amp;rlist[DEFAULT_RT]); 
327 }
328                         
329 actions = actions action { $$ = append_action($1, $2}; }
330         | action { $$ = $1; }
331
332 action = cmd SEMICOLON { $$ = $1; }
333        | SEMICOLON { $$ = 0; }
334
335 cmd = "forward" "(" host ")" { $$ = mk_action(FORWARD_T, STRING_ST, NUMBER_ST, $3, 0)
336     | ...
337 </programlisting>
338                     <para>
339                         A config file can contain one or more "route" statements. "route" statement without
340                         number will be executed first and is called the main route statement. There can be
341                         additional route statements identified by number, these additional route statements
342                         can be called from the main route statement or another additional route statements.
343                     </para>
344                     <para>
345                         Each route statement consists of a set of actions. Actions in the route statement are
346                         executed step by step in the same order in which they appear in the config file. Actions in the 
347                         route statement are delimited by semicolon.
348                     </para>
349                     <para>
350                         Each action consists of one and only one command (cmd in the grammar). There are many types
351                         of commands defined. We don't list all of them here because the list would be too long and all the
352                         commands are processed in the same way. Therefore we show only one example (forward) and
353                         interested readers might look in <filename moreinfo="none">cfg.y</filename> file for
354                         full list of available commands.
355                     </para>
356                     <para>
357                         Each rule in the grammar contains a section enclosed in curly braces. The section is the C code
358                         snippet that will be executed every time the parser recognizes that rule in the config file.
359                     </para>
360                     <para>
361                         For example, when the parser finds <function moreinfo="none">forward</function> command, 
362                         <function moreinfo="none">mk_action</function> function (as specified in the
363                         grammar snippet above) will be called. The function creates a new structure with 
364                         <structfield>type</structfield> field set to FORWARD_T representing the command. Pointer 
365                         to the structure will be returned as the return value of the rule.
366                     </para>
367                     <para>
368                         The pointer propagates through <function moreinfo="none">action</function> rule to 
369                         <function moreinfo="none">actions</function> rule. <function moreinfo="none">Actions</function> rule 
370                         will create linked list of all commands. The linked list will be then inserted into  
371                         <structfield>rlist</structfield> table.
372                         (Function <function moreinfo="none">push</function> in rule <function moreinfo="none">route_stm</function>).
373                         Each element of the table represents one <quote>route</quote> statement of the config file.
374                     </para>
375                     <para>
376                         Each route statement of the configuration file will be represented by a linked list of all
377                         actions in the statement. Pointers to all the lists will be stored in rlist array. Additional
378                         route statements are identified by number. The number also serves as index to the array.
379                     </para>
380                     <para>
381                         When the core is about to execute route statement with number n, it will look in the array
382                         at position n. If the element at position n is not null then there is a linked list of
383                         commands and the commands will be executed step by step.
384                     </para>
385                     <para>
386                         Reply-Route statement is compiled in the same way. Main differences are:
387                         <itemizedlist>
388                             <listitem>
389                                 <para>
390                                     Reply-Route statement is executed when a <acronym>SIP</acronym> <emphasis>REPLY</emphasis> 
391                                     comes (not ,<acronym>SIP</acronym> <emphasis>REQUEST</emphasis>).
392                                 </para>
393                             </listitem>
394                             <listitem>
395                                 <para>
396                                     Only subset of commands is allowed in the reply-route statement.
397                                     (See file <filename moreinfo="none">cfg.y</filename> for more details).
398                                 </para>
399                             </listitem>
400                             <listitem>
401                                 <para>Reply-route statement has it's own array of linked-lists.</para>
402                             </listitem>
403                         </itemizedlist>
404                     </para>                 
405                 </section> <!-- route-stm -->
406                 
407                 <section id="assign-stm">
408                     <title>Assign Statement</title>
409                     <para>
410                         The server contains many configuration variables. There is a section of the
411                         config file in which the variables can be assigned new value. The section
412                         is called The Assign Statement. The following grammar snippet describes how the section
413                         is constructed (only one example will be shown):
414                     </para>
415                     <programlisting format="linespecific">
416 assign_stm = "children" '=' NUMBER { children_no=$3; }
417            | "children" '=' error  { yyerror("number expected"); } 
418 ...
419 </programlisting>
420                     <para>
421                         The number in the config file is assigned to <varname>children_no</varname> variable. 
422                         The second statement will be executed if the parameter is not number or is in invalid 
423                         format and will issue an error and abort the server.
424                     </para>
425                 </section> <!-- assign-stm -->
426                 
427                 <section id="module-stm">
428                     <title>Module Statement</title>
429                     <para>
430                         The module statement allows module loading and configuration. There are two commands:
431                     </para>
432                     <itemizedlist>
433                         <listitem>
434                             <para>
435                                 <emphasis>loadmodule</emphasis> - Load the specified module in form of a shared object. 
436                                 The shared object will be loaded using <function moreinfo="none">dlopen</function>.
437                             </para>
438                         </listitem>
439                         <listitem>
440                             <para>
441                                 <emphasis>modparam</emphasis> - It is possible to configure a module using this 
442                                 command. The command accepts 3 parameters: 
443                                 <emphasis>module name</emphasis>, <emphasis>variable name</emphasis> 
444                                 and <emphasis>variable value</emphasis>.
445                             </para>
446                         </listitem>
447                     </itemizedlist>
448                     <para>The following grammar snippet describes the module statement:</para>
449                     <programlisting format="linespecific">
450 module_stm = "loadmodule" STRING 
451
452     DBG("loading module %s\n", $2);
453     if (load_module($2)!=0) {
454         yyerror("failed to load module");
455     }
456 }
457            | "loadmodule" error  { yyerror("string expected");  }
458            | "modparam" "(" STRING "," STRING "," STRING ")" 
459                         {
460                             if (set_mod_param($3, $5, STR_PARAM, $7) != 0) {
461                                 yyerror("Can't set module parameter");
462                             }
463                         }
464            | "modparam" "(" STRING "," STRING "," NUMBER ")" 
465                         {
466                             if (set_mod_param($3, $5, INT_PARAM, (void*)$7) != 0) {
467                                 yyerror("Can't set module parameter");
468                             }
469                         }
470            | MODPARAM error { yyerror("Invalid arguments"); }
471 </programlisting>
472                     <para>
473                         When the parser finds <function moreinfo="none">loadmodule</function> command, it will execute 
474                         statement in curly braces.
475                         The statement will call <function moreinfo="none">load_module</function> function.
476                         The function will load the specified filename using <function moreinfo="none">dlopen</function>.
477                         If <function moreinfo="none">dlopen</function> was successful, the server will look for
478                         <structname>exports</structname> structure describing the module's interface and register the 
479                         module. For more details see <link linkend="module-interface">module section</link>.
480                     </para>
481                     <para>
482                         If the parser finds <function moreinfo="none">modparam</function> command, it will try to configure 
483                         the specified variable in the
484                         specified module. The module must be loaded using <function moreinfo="none">loadmodule</function>
485                         before <function moreinfo="none">modparam</function> for the module can be used !
486                         Function <function moreinfo="none">set_mod_param</function> will be called and will
487                         configure the variable in the specified module.
488                     </para>
489                 </section> <!-- module-stm -->
490             </section> <!-- config-file-struct -->
491         </section> <!-- ser-config -->
492         
493         <section id="iface-config">
494             <title>Interface Configuration</title>
495             <para>
496                 The server will try to obtain list of all configured interfaces of the host it is running on. If it
497                 fails the server tries to convert hostname to <acronym>IP</acronym> address and will use interface with 
498                 the <acronym>IP</acronym> address only.
499             </para>
500             <para>
501                 Function <function moreinfo="none">add_interfaces</function> will add all configured interfaces to the
502                 array.
503             </para>
504             <para>
505                 Try to convert all interface names to IP addresses, remove duplicates...
506             </para>
507         </section> <!-- iface-config -->
508         
509         <section id="daemon">
510             <title>Turning into a Daemon</title>
511             <para>
512                 When configured so, <acronym>SER</acronym> becomes a daemon during startup. A process is called daemon
513                 when it hasn't associated controlling terminal. See function 
514                 <function moreinfo="none">daemonize</function> in file 
515                 <filename moreinfo="none">main.c</filename> for more details.
516                 The function does the following:
517             </para>
518             <itemizedlist>
519                 <listitem>
520                     <para>
521                         <emphasis>chroot</emphasis> is performed if necessary. That ensures that the server will have
522                         access to a particular directory and its subdirectories only.
523                     </para>
524                 </listitem>
525                 <listitem>
526                     <para>
527                         Server's working directory is changed if the new working directory was
528                         specified (usually it is /).
529                     </para>
530                 </listitem>
531                 <listitem>
532                     <para>
533                         If command line parameter -g was used, the server's group ID is changed
534                         to that value.
535                     </para>
536                 </listitem>
537                 <listitem>
538                     <para>
539                         If command line parameter -u was used, the server's user ID is changed
540                         to that value.
541                     </para>
542                 </listitem>
543                 <listitem>
544                     <para>
545                         Perform <emphasis>fork</emphasis>, let the parent process exit. This ensures that we 
546                         are not a group leader.
547                     </para>
548                 </listitem>
549                 <listitem>
550                     <para>
551                         Perform <emphasis>setsid</emphasis> to become a session leader and drop the controlling 
552                         terminal.
553                     </para>
554                 </listitem>
555                 <listitem>
556                     <para>
557                         Fork again to drop group leadership.
558                     </para>
559                 </listitem>
560                 <listitem>
561                     <para>Create a pid file.</para>
562                 </listitem>
563                 <listitem>
564                     <para>Close all opened file descriptors.</para>
565                 </listitem>
566             </itemizedlist>
567         </section> <!-- daemon -->
568         
569         <section id="module-init">
570             <title>Module Initialization</title>
571             <para>
572                 The whole config file was parsed, all modules were loaded already and can
573                 be initialized now. A module can tell the core that it needs to be initialized
574                 by exporting <function moreinfo="none">mod_init</function> function.
575                 <function moreinfo="none">mod_init</function> function of all loaded modules
576                 will be called now.
577             </para>
578         </section> <!-- module-init -->
579
580         <section id="rt-list-fix">
581             <title>Routing List Fixing</title>
582             <para>
583                 After the whole routing list was parsed, there might be still places that can be
584                 further processed to speed-up the server. For example, several commands accept
585                 regular expression as one of their parameters. The regular expression can be compiled
586                 too and processing of compiled expression will be much faster.
587             </para>
588             <para>
589                 Another example might be string as parameter of a function. For example if you call
590                 <function moreinfo="none">append_hf("Server: SIP Express Router\r\n")</function> from the routing 
591                 script, the function will 
592                 append a new header field after the last one. In this case, the function needs to know length 
593                 of the string parameter. It could call <function moreinfo="none">strlen</function> every time it is
594                 called, but that is not a very good idea because <function moreinfo="none">strlen</function> would
595                 be called every time a message is processed and that is not necessary.
596             </para>
597             <para>
598                 Instead of that the length of the string parameter could be pre-calculated upon server startup, saved
599                 and reused later. The processing of the request will be faster because 
600                 <function moreinfo="none">append_hf</function> doesn't need
601                 to call <function moreinfo="none">strlen</function> every time, I can just reuse the saved value.
602             </para>
603             <para>
604                 This can be used also for string to int conversions, hostname lookups, expression evaluation and so on.
605             </para>
606             <para>
607                 This process is called Routing List Fixing and will be done as one of last steps of the server startup.
608             </para>
609             <para>
610                 Every loaded module can export one or more functions. Each such function can have associated a fixup
611                 function, which should do fixing as described in this section. All such fixups of all loaded modules
612                 will be called here. That makes it possible for module functions to fix their parameters too if necessary.
613             </para>
614         </section> <!-- rt-list-fix -->
615
616         <section id="stat-init">
617             <title>Statistics Initialization</title>
618             <para>
619                 If compiled-in, the core can produce some statistics about itself and traffic processed. The statistics
620                 subsystem gets initialized here, see function <function moreinfo="none">init_stats</function>.
621             </para>
622         </section> <!-- stat-init -->
623
624         <section id="socket-init">
625             <title>Socket Initialization</title>
626             <para>
627                 UDP socket initialization depends on <varname>dont_fork</varname> variable. If this variable is set 
628                 (only one process will
629                 be processing incoming requests) and there are multiple listen interfaces, only the first one will be used. This
630                 mode is mainly for debugging.
631             </para>
632             <para>
633                 If the variable is not set, then sockets for all configured interfaces will be created and initialized.
634                 See function <function moreinfo="none">udp_init</function> in file
635                 <filename moreinfo="none">udp_server.c</filename> for more details.
636             </para>
637         </section> <!-- socke-init -->
638         
639         <section id="forking">
640             <title>Forking</title>
641             <para>
642                 The rest of the initialization process depends on value of <varname>dont_fork</varname> variable. 
643                 <varname>dont_fork</varname> is a global variable defined in <filename moreinfo="none">main.c</filename>.
644                 We will describe both variants separately.
645             </para>
646
647             <section id="dont-fork-set">
648                 <title><varname>dont_fork</varname> variable is set (not zero)</title>
649                 <para>
650                     If <varname>dont_fork</varname> variable is set, the server will be operating in special mode. 
651                     There will be only one process processing incoming requests. This is very slow and was intended mainly 
652                     for debugging purposes. The main process will be processing all incoming requests itself.
653                 </para>
654                 <para>
655                     The server still needs additional children:
656                 </para>
657                 <itemizedlist>
658                     <listitem>
659                         <para>
660                             One child is for the timer subsystem, the child will be processing timers independently of 
661                             the main process.
662                         </para>
663                     </listitem>
664                     <listitem>
665                         <para>
666                             <acronym>FIFO</acronym> server will spawn another child if enabled. The child will be 
667                             processing all commands coming through the fifo interface.
668                         </para>
669                     </listitem>
670                     <listitem>
671                         <para>If <acronym>SNMP</acronym> support was enabled, another child will be created.</para>
672                     </listitem>
673                 </itemizedlist>
674                 
675                 <para>
676                     The following initialization will be performed in dont fork mode.
677                     (look into function <function moreinfo="none">main_loop</function> in file 
678                     <filename moreinfo="none">main.c</filename>.
679                 </para>
680
681                 <itemizedlist>
682                     <listitem>
683                         <para>Another child will be forked for the timer subsystem.</para>
684                     </listitem>
685                     <listitem>
686                         <para>
687                             Initialize the <acronym>FIFO</acronym> server if enabled, this will fork another child.
688                             For more info about the <acronym>FIFO</acronym> server, 
689                             see section The <acronym>FIFO</acronym> server.
690                         </para>
691                     </listitem>
692                     <listitem>
693                         <para>
694                             Call <function moreinfo="none">init_child(0).</function>
695                             The function performs per-child specific initialization of all loaded modules.
696                             A module can be initialized though <function moreinfo="none">mod_init</function> function.
697                             The function is called <emphasis>BEFORE</emphasis> the server forks and thus is common for all 
698                             children.
699                         </para>
700                         <para>
701                             If there is anything, that needs to be initialized in every child separately (for example
702                             if each child needs to open its own file descriptor), it cannot be done in 
703                             <function>mod_init</function>.
704                             To make such initialization possible, a module can export another initialization function
705                             called <function moreinfo="none">init_child</function>. The function will be called in
706                             all children <emphasis>AFTER</emphasis> fork of the server.
707                         </para>
708                         <para>
709                             And since we are in <quote>dont fork</quote> mode and there will no children processing 
710                             requests (remember the main process will be processing all requests), the 
711                             <function moreinfo="none">init_child</function> wouldn't be called.
712                         </para>
713                         <para>
714                             That would be bad, because <function moreinfo="none">child_init</function> might do some
715                             initialization that must be done otherwise modules might not work properly.
716                         </para>
717                         <para>
718                             To make sure that module initialization is complete we will call 
719                             <function moreinfo="none">init_child</function> here for the main process even if we
720                             are not going to fork.
721                         </para>
722                     </listitem>
723                 </itemizedlist>
724                 <para>
725                     That's it. Everything has been initialized properly and as the last step we will call
726                     <function moreinfo="none">udp_rcv_loop</function> which is the main loop function. The function
727                     will be described later.
728                 </para>
729             </section> <!-- dont-fork-set -->
730
731             <section id="dont-fork-not-set">
732                 <title><varname>dont_fork</varname> is not set (zero)</title>
733                 <para>
734                     <varname>dont_fork</varname> is not set. That means that the server will fork children and the children
735                     will be processing incoming requests. How many children will be created depends on
736                     the configuration (<varname>children</varname> variable). The main process will be sleeping and 
737                     handling signals only.
738                 </para>
739
740                 <para>
741                     The main process will then initialize the <acronym>FIFO</acronym> server. The <acronym>FIFO</acronym> server 
742                     needs another child to handle communication over <acronym>FIFO</acronym> and thus another child will be 
743                     created. The <acronym>FIFO</acronym> server will be described in more detail later.
744                 </para>
745
746                 <para>
747                     Then the main process will perform another fork for the timer attendant. The child will
748                     take care of timer lists and execute specified function when a timer hits.
749                 </para>
750                 <para>
751                     The main process is now completely initialized, it will sleep in <function moreinfo="none">pause</function>
752                     function until a signal comes and call <function moreinfo="none">handle_sigs</function> when such
753                     condition occurs.
754                 </para>
755                 
756                 <para>
757                     The following initialization will be performed by each child separately:
758                 </para>
759                 <para>
760                     Each child executes <function moreinfo="none">init_child</function> function. The function
761                     will sequentially call <function moreinfo="none">child_init</function> functions of all loaded modules.
762                 </para>
763                 <para>
764                     Because the function is called in each child separately, it can initialize per-child
765                     specific data. For example if a module needs to communicate with database, it must open
766                     a database connection. If the connection would be opened in <function moreinfo="none">mod_init</function>
767                     function, all the children would share the same connection and locking would be necessary
768                     to avoid conflicts. On the other hand if the connection was opened in
769                     <function moreinfo="none">child_init</function> function, each child will have its own
770                     connection and concurrency conflicts will be handled by the database server.
771                 </para>
772                 <para>
773                     And last, but not least, each child executes <function moreinfo="none">udp_rcv_loop</function> function
774                     which contains the main loop logic.
775                 </para>
776                 
777             </section> <!-- dont-fork-not-set -->
778         </section> <!-- forking -->
779     </chapter> <!-- server-startup -->
780     
781     <chapter id="main-loop">
782         <title>Main Loop</title>
783         <para>
784             Upon startup, all children execute <function moreinfo="none">recvfrom</function> function. The process will
785             enter the kernel mode. When there is no data to be processed at the moment, the kernel will put the process on
786             list of processes waiting for data and the process will be put asleep. 
787         </para>
788         <para>
789             When data to be processed was received, the first process on the list will be removed from the list and woken
790             up. After the process finished processing of the data, it will call <function moreinfo="none">recvfrom</function>
791             again and will be put by the kernel at the end of the list.
792         </para>
793         <para>
794             When next data arrives, the first process on the list will be removed, processes the data and will be put on the
795             end of the list again. And so on...
796         </para>
797
798         <para>
799             The main loop logic can be found in function <function moreinfo="none">udp_rcv_loop</function> in file
800             <filename moreinfo="none">udp_server.c</filename>.
801         </para>
802         <para>
803             The message is received using <function moreinfo="none">recvfrom</function> function. The received data is
804             stored in buffer and zero terminated. 
805         </para>
806         <para>
807             If configured so, basic sanity checks over the received message will be performed.
808         </para>
809         <para>
810             The message is then processed by <function moreinfo="none">receive_msg</function> function and 
811             <function moreinfo="none">recvfrom</function> is called again.
812         </para>
813         
814         <section id="recv-message">
815             <title><function moreinfo="none">receive_msg</function> Function</title>
816             <para>
817                 The function can be found in <filename moreinfo="none">receive.c</filename> file.
818             </para>
819             
820             <itemizedlist>
821                 <listitem>
822                     <para>
823                         In the server, a request or response is represented by <structname>sip_msg</structname>
824                         structure. The structure is allocated in this function. The original message is stored in 
825                         <structfield>buf</structfield>
826                         attribute of the structure and is zero terminated. Then, another copy of the received message will be
827                         created and the copy will be stored in <structfield>orig</structfield> field. The original copy will be not
828                         modified during the server operation. All changes will be made to the copy in <structfield>buf</structfield>
829                         field. The second copy of the message will be removed in the future.
830                     </para>
831                 </listitem>
832                 <listitem>
833                     <para>
834                         The message will be parsed (function <function moreinfo="none">parse_msg</function>). We don't
835                         need the whole message header to be parsed at this stage. Only the first line and first Via
836                         header need to be parsed. The server needs to know if the message is request or response - hence
837                         the first line. The server also needs the first Via to be able to add its own Via - hence
838                         the first Via. Nothing else will be parsed at the moment - this saves time. (Message parser
839                         as well as <structname>sip_msg</structname> structure will be described later).
840                     </para>
841                 </listitem>
842                 <listitem>
843                     <para>
844                         A module may register callbacks. Each callback have associated an event, that
845                         will trigger the callback. One such callback is <emphasis>pre-script</emphasis> callback. Such
846                         callback will be called immediately before the routing part of the config file will be executed. 
847                         If there are such callbacks registered, they will be executed now.
848                     </para>
849                 </listitem>
850                 <listitem>
851                     <para>
852                         As the next step we will determine type of the message. If the message being processed is a REQUEST
853                         then basic sanity checks will be performed (make sure that there is the first Via and parsing was
854                         successful) and the message will be passed to routing engine.
855                         The routing engine is one of the most complicated parts of the server and will be in detail
856                         described in chapter <link linkend="routing-engine">The Routing Engine</link>.
857                     </para>
858                 </listitem>
859                 <listitem>
860                     <para>
861                         If the message is a RESPONSE, it will be simply forwarded to its destination.
862                     </para>
863                 </listitem>
864                 <listitem>
865                     <para>
866                         After all, <emphasis>post-script</emphasis> callbacks will be executed if any and the 
867                         structure representing the message will be released.
868                     </para>
869                 </listitem>
870                 <listitem>
871                     <para>
872                         Processing of the message is done now and the process is ready for another 
873                         <acronym>SIP</acronym> message.
874                     </para>
875                 </listitem>
876             </itemizedlist>
877         </section> <!-- recv-message -->
878     </chapter> <!-- main-loop -->
879
880     <chapter id="server-shutdown">
881         <title>The Server Shutdown</title>
882         <para>
883             The server shutdown can be triggered by sending a signal to the server. The server will behave differently upon 
884             receiving various types of signals, here is a brief summary:
885         </para>
886         <itemizedlist>
887             <listitem>
888                 <para>
889                     <emphasis>SIGINT, SIGPIPE, SIGTERM, SIGCHLD</emphasis> will terminate the server.
890                 </para>
891             </listitem>
892             <listitem>
893                 <para>
894                     <emphasis>SIGUSR1</emphasis> will print statistics and let the server continue.
895                 </para>
896             </listitem>
897             <listitem>
898                 <para>
899                     <emphasis>SIGHUP, SIGUSR2</emphasis> will be ignored.
900                 </para>
901             </listitem>
902         </itemizedlist>
903         
904         <para>
905             There is only one common signal handler for all signals - function <function moreinfo="none">sig_usr</function>
906             in file <filename moreinfo="none">main.c</filename>.
907         </para>
908
909         <para>
910             In normal mode of operation (<varname>dont_fork</varname> variable is not set), the main server is not processing any 
911             requests, it calls <function moreinfo="none">pause</function> function and will be waiting for signals only. What
912             happens when a signal arrives is shown in the previous paragraph.
913         </para>
914         <para>
915             When in normal mode (<varname>dont_fork</varname> is not set), the signal handler of the main process will 
916             only store number of the signal received.
917             All the processing logic will be executed by the main process outside the signal handler (function
918             <function moreinfo="none">handle_sigs</function>) The function will be called immediately after the signal
919             handler finish. The main
920             process usually does some cleanup and running such things outside the signal handler is much
921             more safe than doing it from the handler itself. Children only print statistics and exit
922             or ignore the signal completely, that is quite safe and can be done directly from the signal handler of
923             children.
924         </para>
925         <para>
926             When <varname>dont_fork</varname> is set, all the cleanup will be done directly from the signal handler, 
927             because there is only one process - the main process. This is not so safe as the previous case, but this mode 
928             should be  used for debugging only and such shortcoming doesn't harm in that case.
929         </para>
930         <para>
931             Upon receipt of SIGINT, SIGPIPE or SIGTERM <function moreinfo="none">destroy_modules</function> will be called.
932             Each module may register so-called <function moreinfo="none">destroy</function> function if it needs to do some 
933             cleanup when the server is terminating (flush of cache to disk  for example). 
934             <function moreinfo="none">destroy_modules</function> will call destroy function of all loaded modules.
935         </para>
936
937         <para>
938             If you need to terminate the server and all of its children, the best way how to do it is to send SIGTERM
939             to the main process, the main process will in turn send the same signal to its children.
940         </para>
941
942         <para>
943             The main process and its children are in the same process group. Therefore the main process can kill all
944             its children simply by sending a signal to pid 0, sending to pid 0 will send the signal to all processes
945             in the same process group as the sending process. This is how the main process will terminate all children 
946             when it is going to shut down.
947         </para>
948
949         <para>
950             If one child exited during normal operation, the whole server will be shut down. This is better than let
951             the server continue - a dead child might hold a lock and that could block the whole server, such situation
952             cannot be avoided easily. Instead of that it is better to shutdown the whole server and let it restart.
953         </para>
954     </chapter> <!-- server-shutdown -->
955
956     <chapter id="internal-data-struct">
957         <title>Internal Data Structures</title>
958         <para>
959             There are some data structures that are important and widely used in the server. We will describe them
960             in detail in this section.
961         </para>
962         
963         <note>
964             <para>
965                 There are many more structures and types defined across the server and modules. We will describe
966                 only the most important and common data structure here. The rest will be described in other sections
967                 if needed.
968             </para>
969         </note>
970
971         <section id="str">
972             <title>Type <type>str</type></title>
973             <para>
974                 One of our main goals was to make <acronym>SER</acronym> really fast. There are many functions across
975                 the server that need to work with strings. Usually these functions need to know string length. We wanted
976                 to avoid using of <function moreinfo="none">strlen</function> because the function is relatively slow. It
977                 must scan the whole string and find the first occurrence of zero character. To avoid this, we created
978                 <type>str</type> type. The type has 2 fields, field <structfield>s</structfield> is pointer
979                 to the beginning of the string and field <structfield>len</structfield> is length of the string. We then
980                 calculate length of the string only once and later reuse saved value.
981             </para>
982             <important>
983                 <para>
984                     <type>str</type> structure is quite important because it is widely used in
985                     <acronym>SER</acronym> (most functions accept <type>str</type> instead of <type>char*</type>).
986                 </para>
987             </important>
988
989             
990             <para><emphasis><type>str</type> Type Declaration</emphasis></para>
991             <programlisting format="linespecific">
992 struct _str{
993     char* s;
994     int len;
995 };
996
997 typedef struct _str str;                    
998 </programlisting>
999
1000             <para>
1001                 The declaration can be found in header file <filename moreinfo="none">str.h</filename>.
1002             </para>
1003
1004             <warning>
1005                 <para>
1006                     Because we store string lengths, there is no need to zero terminate them. Some strings in the
1007                     server are still zero terminated, some are not. Be careful when using functions like 
1008                     <function moreinfo="none">snprintf</function> that rely on the ending zero. You can print 
1009                     variable of type <type>str</type> this way:
1010                     <programlisting format="linespecific">printf("%.*s", mystring->len, mystring->s);</programlisting>
1011                     That ensures that the string will be printed correctly even if there is no zero character at
1012                     the end.
1013                 </para>
1014             </warning>
1015         </section> <!-- str -->
1016         
1017         <section id="hdr-field">
1018             <title>Structure <structname>hdr_field</structname></title>
1019             <para>
1020                 The structure represents a header field of a <acronym>SIP</acronym> message. A header field
1021                 consist of <emphasis>name</emphasis> and <emphasis>body</emphasis> separated by a double colon.
1022                 For example: <quote>Server: SIP Express Router\r\n</quote> is one header field. <quote>Server</quote> 
1023                 is header field name and <quote>SIP Express Router\r\n</quote> is header field body.
1024             </para>
1025             <para>
1026                 The structure is defined in file <filename moreinfo="none">hf.h</filename> under
1027                 <filename moreinfo="none">parser</filename> subdirectory.
1028             </para>
1029             <para>
1030                 <emphasis>Structure Declaration</emphasis>
1031             </para>
1032             <programlisting format="linespecific">
1033 struct hdr_field {   
1034     int type;                /* Header field type */
1035     str name;                /* Header field name */
1036     str body;                /* Header field body */
1037     void* parsed;            /* Parsed data structures */
1038     struct hdr_field* next;  /* Next header field in the list */
1039 };
1040 </programlisting>
1041             <para>
1042                 <emphasis>Field Description:</emphasis>
1043             </para>
1044             <itemizedlist>
1045                 <listitem>
1046                     <para>
1047                         <structfield>type</structfield> - Type of the header field, the following header field
1048                         types are defined (and recognized by the parser): 
1049                     </para>
1050                     <para>
1051                         HDR_VIA1, HDR_VIA2, HDR_TO, HDR_FROM, HDR_CSEQ, HDR_CALLID, HDR_CONTACT, HDR_MAXFORWARDS, 
1052                         HDR_ROUTE, HDR_RECORDROUTE, HDR_CONTENTTYPE, HDR_CONTENTLENGTH, HDR_AUTHORIZATION, 
1053                         HDR_EXPIRES, HDR_PROXYAUTH, HDR_WWWAUTH, HDR_SUPPORTED, HDR_REQUIRE, HDR_PROXYREQUIRE, 
1054                         HDR_UNSUPPORTED, HDR_ALLOW, HDR_EVENT, HDR_OTHER.
1055                     </para>
1056                     <para>
1057                         Their meaning is self explanatory. HDR_OTHER marks header field not recognized by the parser.
1058                     </para>
1059                 </listitem>
1060                 <listitem>
1061                     <para>
1062                         <structfield>name</structfield> - Name of the header field (the part before colon)
1063                     </para>
1064                 </listitem>
1065                 <listitem>
1066                     <para>
1067                         <structfield>body</structfield> - body of the header field (the part after colon)
1068                     </para>
1069                 </listitem>
1070                 <listitem>
1071                     <para>
1072                         <structfield>parsed</structfield> - Each header field body can be further parsed. The field
1073                         contains pointer to parsed structure if the header field was parsed already. The pointer is
1074                         of type <type>void*</type> because it can point to different types of structure depending on 
1075                         the header field type.
1076                     </para>
1077                 </listitem>
1078                 <listitem>
1079                     <para>
1080                         <structfield>next</structfield> - Pointer to the next header field in linked list.
1081                     </para>
1082                 </listitem>
1083             </itemizedlist>
1084         </section> <!-- hdr-field -->
1085
1086         <section id="sip-uri"> 
1087             <title>Structure <structname>sip_uri</structname></title>
1088             <para>This structure represents parsed <acronym>SIP</acronym> <acronym>URI</acronym>.</para>
1089             <programlisting format="linespecific">
1090 struct sip_uri {
1091     str user;     /* Username */
1092     str passwd;   /* Password */
1093     str host;     /* Host name */
1094     str port;     /* Port number */
1095     str params;   /* Parameters */
1096     str headers;  
1097 };
1098 </programlisting>
1099             <para><emphasis>Field Description:</emphasis></para>
1100             <itemizedlist>
1101                 <listitem>
1102                     <para>
1103                         <structfield>user</structfield> - Username if found in the <acronym>URI</acronym>.
1104                     </para>
1105                 </listitem>
1106                 <listitem>
1107                     <para>
1108                         <structfield>passwd</structfield> - Password if found in the <acronym>URI</acronym>.
1109                     </para>
1110                 </listitem>
1111                 <listitem>
1112                     <para>
1113                         <structfield>host</structfield> - Hostname of the <acronym>URI</acronym>.
1114                     </para>
1115                 </listitem>
1116                 <listitem>
1117                     <para>
1118                         <structfield>params</structfield> - Parameters of the <acronym>URI</acronym> if any.
1119                     </para>
1120                 </listitem>
1121                 <listitem>
1122                     <para>
1123                         <structfield>headers</structfield> - See the <acronym>SIP</acronym> <acronym>RFC</acronym>.
1124                     </para>
1125                 </listitem>
1126             </itemizedlist>
1127         </section> <!-- sip-uri -->
1128         
1129         <section id="via-body">
1130             <title>Structure <structname>via_body</structname></title>
1131             <para>
1132                 The structure represents parsed Via header field. See file <filename moreinfo="none">parse_via.h</filename>
1133                 under <filename moreinfo="none">parser</filename> subdirectory for more details.
1134             </para>
1135             <programlisting format="linespecific">
1136 struct via_body { 
1137     int error;
1138     str hdr;                      /* Contains "Via" or "v" */
1139     str name;
1140     str version;   
1141     str transport;
1142     str host;
1143     int port;
1144     str port_str;
1145     str params;
1146     str comment;
1147     int bsize;                    /* body size, not including hdr */
1148     struct via_param* param_lst;  /* list of parameters*/
1149     struct via_param* last_param; /*last via parameter, internal use*/
1150
1151     /* shortcuts to "important" params*/
1152     struct via_param* branch;
1153     struct via_param* received;
1154
1155     struct via_body* next;        /* pointer to next via body string if
1156     compact via or null */
1157 };
1158 </programlisting>
1159             <para><emphasis>Field Description:</emphasis></para>
1160             <itemizedlist>
1161                 <listitem>
1162                     <para>
1163                         <structfield>error</structfield> - The field contains error code when the parser was unable
1164                         to parse the header field.
1165                     </para>
1166                 </listitem>
1167                 <listitem>
1168                     <para>
1169                         <structfield>hdr</structfield>- Header field name, it can be <quote>Via</quote> or <quote>v</quote>
1170                         in this case.
1171                     </para>
1172                 </listitem>
1173                 <listitem>
1174                     <para>
1175                         <structfield>name</structfield> - Protocol name (<quote>SIP</quote> in this case).
1176                     </para>
1177                 </listitem>
1178                 <listitem>
1179                     <para>
1180                         <structfield>version</structfield> - Protocol version (for example <quote>2.0</quote>).
1181                     </para>
1182                 </listitem>
1183                 <listitem>
1184                     <para>
1185                         <structfield>transport</structfield> - Transport protocol name (<quote>TCP</quote>, <quote>UDP</quote>
1186                         and so on).
1187                     </para>
1188                 </listitem>
1189                 <listitem>
1190                     <para>
1191                         <structfield>host</structfield> - Hostname or IP address contained in the Via header field.
1192                     </para>
1193                 </listitem>
1194                 <listitem>
1195                     <para>
1196                         <structfield>port</structfield> - Port number as integer.
1197                     </para>
1198                 </listitem>
1199                 <listitem>
1200                     <para>
1201                         <structfield>port_str</structfield> - Port number as string.
1202                     </para>
1203                 </listitem>
1204                 <listitem>
1205                     <para>
1206                         <structfield>params</structfield> - Unparsed parameters (as one string containing all the parameters).
1207                     </para>
1208                 </listitem>
1209                 <listitem>
1210                     <para>
1211                         <structfield>comment</structfield> - Comment.
1212                     </para>
1213                 </listitem>
1214                 <listitem>
1215                     <para>
1216                         <structfield>bsize</structfield> - Size of the body (not including hdr).
1217                     </para>
1218                 </listitem>
1219                 <listitem>
1220                     <para>
1221                         <structfield>param_lst</structfield> - Linked list of all parameters.
1222                     </para>
1223                 </listitem>
1224                 <listitem>
1225                     <para>
1226                         <structfield>last_param</structfield> - Last parameter in the list.
1227                     </para>
1228                 </listitem>
1229                 <listitem>
1230                     <para>
1231                         <structfield>branch</structfield> - Branch parameter.
1232                     </para>
1233                 </listitem>
1234                 <listitem>
1235                     <para>
1236                         <structfield>received</structfield> - Received parameter.
1237                     </para>
1238                 </listitem>
1239                 <listitem>
1240                     <para>
1241                         <structfield>next</structfield> - If the Via is in compact form (more Vias in the same header
1242                         field), this field contains pointer to the next Via.
1243                     </para>
1244                 </listitem>
1245             </itemizedlist>
1246         </section> <!-- via-body -->
1247
1248         <section id="ip-addr">
1249             <title>Structure <structname>ip_addr</structname></title>
1250             <para>
1251                 The structure represents <acronym>IPv4</acronym> or <acronym>IPv6</acronym> address. It is defined in 
1252                 <filename moreinfo="none">ip_addr.h</filename>.
1253             </para>
1254             <programlisting format="linespecific">
1255 struct ip_addr{
1256     unsigned int af;     /* address family: AF_INET6 or AF_INET */
1257     unsigned int len;    /* address len, 16 or 4 */
1258                 
1259     /* 64 bits aligned address */
1260     union {
1261         unsigned int   addr32[4];
1262         unsigned short addr16[8];
1263         unsigned char  addr[16];
1264     }u;
1265 };
1266 </programlisting>
1267         </section> <!-- ip-addr -->
1268
1269         <section id="lump">
1270             <title>Structure <structname>lump</structname></title>
1271             <para>
1272                 The structure describes modifications that should be made to the message before
1273                 the message will be sent. 
1274             </para>
1275             <para>
1276                 The structure will be described in more detail later in chapter 
1277                 SIP Message Modifications.
1278             </para>
1279         </section> <!-- lump -->
1280         
1281         <section id="lump-rpl">
1282             <title>Structure <structname>lump_rpl</structname></title>
1283             <para>
1284                 The structure represents text that should be added to reply. List of such data is
1285                 kept in the request and processed when the request is being turned into reply. 
1286             </para>
1287             <para>
1288                 The structure will be described in more detail later in chapter
1289                 SIP Message Modifications.
1290             </para>
1291         </section> <!-- lump-rpl -->
1292
1293         <section id="msg-start">
1294             <title>Structure <structname>msg_start</structname></title>
1295             <para>
1296                 The structure represents the first line of a <acronym>SIP</acronym> request or response. 
1297             </para>
1298             <para>
1299                 The structure is defined in file <filename moreinfo="none">parse_fline.h</filename> under 
1300                 <filename moreinfo="none">parser</filename> subdirectory.
1301             </para>
1302
1303             <para>
1304                 <emphasis>Structure Declaration</emphasis>
1305             </para>
1306             <programlisting format="linespecific">
1307 struct msg_start {
1308     int type;                        /* Type of the Message - Request/Response */
1309     union {
1310         struct {
1311             str method;              /* Method string */
1312             str uri;                 /* Request URI */
1313             str version;             /* SIP version */
1314             int method_value;        /* Parsed method */
1315         } request;
1316         struct {
1317             str version;             /* SIP version */
1318             str status;              /* Reply status */
1319             str reason;              /* Reply reason phrase */
1320             unsigned int statuscode; /* Status code */
1321         } reply;
1322     }u;
1323 };
1324 </programlisting>
1325             
1326             <para>
1327                 <emphasis>Description of Request Related Fields:</emphasis>
1328             </para>
1329
1330             <itemizedlist>
1331                 <listitem>
1332                     <para>
1333                         <structfield>type</structfield> - Type of the message - REQUEST or RESPONSE.
1334                     </para>
1335                 </listitem>
1336                 <listitem>
1337                     <para>
1338                         <structfield>method</structfield> - Name of method (same as in the message).
1339                     </para>
1340                 </listitem>
1341                 <listitem>
1342                     <para>
1343                         <structfield>uri</structfield> - Request <acronym>URI</acronym>.
1344                     </para>
1345                 </listitem>
1346                 <listitem>
1347                     <para>
1348                         <structfield>version</structfield> - Version string.
1349                     </para>
1350                 </listitem>
1351                 <listitem>
1352                     <para>
1353                         <structfield>method_value</structfield> - Parsed method. Field method which is of
1354                         type <type>str</type> will be converted to integer and stored here. This is good for comparison,
1355                         integer comparison is much faster then string comparison.
1356                     </para>
1357                 </listitem>
1358             </itemizedlist>
1359
1360             <para>
1361                 <emphasis>Description of Response Related Fields:</emphasis>
1362             </para>
1363
1364             <itemizedlist>
1365                 <listitem>
1366                     <para>
1367                         <structfield>version</structfield> - Version string.
1368                     </para>
1369                 </listitem>
1370                 <listitem>
1371                     <para>
1372                         <structfield>status</structfield> - Response status code as string.
1373                     </para>
1374                 </listitem>
1375                 <listitem>
1376                     <para>
1377                         <structfield>reason</structfield> - Response reason string as in the message.
1378                     </para>
1379                 </listitem>
1380                 <listitem>
1381                     <para>
1382                         <structfield>statuscode</structfield> - Response status code converted to integer.
1383                     </para>
1384                 </listitem>
1385             </itemizedlist>
1386
1387         </section> <!-- msg-start -->
1388
1389         <section id="sip-msg">
1390             <title>Structure <structname>sip_msg</structname></title>
1391             <para>
1392                 This is the most important structure in the whole server. This structure represents
1393                 a <acronym>SIP</acronym> message. When a message is received, it is immediately converted
1394                 into this structure and all operations are performed over the structure. After the server
1395                 finished processing, this structure is converted back to character array
1396                 buffer and the buffer is sent out.
1397             </para>
1398             <para><emphasis>Structure Declaration:</emphasis></para>
1399             <programlisting format="linespecific">
1400 struct sip_msg {
1401     unsigned int id;               /* message id, unique/process*/
1402     struct msg_start first_line;   /* Message first line */
1403     struct via_body* via1;         /* The first via */
1404     struct via_body* via2;         /* The second via */
1405     struct hdr_field* headers;     /* All the parsed headers*/
1406     struct hdr_field* last_header; /* Pointer to the last parsed header*/
1407     int parsed_flag;               /* Already parsed header field types */
1408
1409     /* Via, To, CSeq, Call-Id, From, end of header*/
1410     /* first occurrence of it; subsequent occurrences 
1411      * saved in 'headers' 
1412      */
1413
1414     struct hdr_field* h_via1;
1415     struct hdr_field* h_via2;
1416     struct hdr_field* callid;
1417     struct hdr_field* to;
1418     struct hdr_field* cseq;
1419     struct hdr_field* from;
1420     struct hdr_field* contact;
1421     struct hdr_field* maxforwards;
1422     struct hdr_field* route;
1423     struct hdr_field* record_route;
1424     struct hdr_field* content_type;
1425     struct hdr_field* content_length;
1426     struct hdr_field* authorization;
1427     struct hdr_field* expires;
1428     struct hdr_field* proxy_auth;
1429     struct hdr_field* www_auth;
1430     struct hdr_field* supported;
1431     struct hdr_field* require;
1432     struct hdr_field* proxy_require;
1433     struct hdr_field* unsupported;
1434     struct hdr_field* allow;
1435     struct hdr_field* event;
1436
1437     char* eoh;        /* pointer to the end of header (if found) or null */
1438     char* unparsed;   /* here we stopped parsing*/
1439
1440     struct ip_addr src_ip;
1441     struct ip_addr dst_ip;
1442                 
1443     char* orig;       /* original message copy */
1444     char* buf;        /* scratch pad, holds a modified message,
1445                        *  via, etc. point into it 
1446                        */
1447     unsigned int len; /* message len (orig) */
1448
1449     /* modifications */
1450                 
1451     str new_uri;               /* changed first line uri*/
1452     int parsed_uri_ok;         /* 1 if parsed_uri is valid, 0 if not */
1453     struct sip_uri parsed_uri; /* speed-up > keep here the parsed uri*/
1454                 
1455     struct lump* add_rm;         /* used for all the forwarded 
1456                                   * requests */
1457     struct lump* repl_add_rm;    /* used for all the forwarded replies */
1458     struct lump_rpl *reply_lump; /* only for locally generated replies !!!*/
1459
1460     char add_to_branch_s[MAX_BRANCH_PARAM_LEN];
1461     int add_to_branch_len;
1462                 
1463     /* index to TM hash table; stored in core to avoid unnecessary calcs */
1464     unsigned int  hash_index;
1465                 
1466     /* allows to set various flags on the message; may be used for 
1467      * simple inter-module communication or remembering processing state
1468      * reached 
1469      */
1470     flag_t flags;       
1471 };
1472 </programlisting>
1473             <para>
1474                 <emphasis>Field Description:</emphasis>
1475             </para>
1476             <itemizedlist>
1477                 <listitem>
1478                     <para>
1479                         <structfield>id</structfield> - Unique ID of the message within a process context.
1480                     </para>
1481                 </listitem>
1482                 <listitem>
1483                     <para>
1484                         <structfield>first_line</structfield> - Parsed first line of the message.
1485                     </para>
1486                 </listitem>
1487                 <listitem>
1488                     <para>
1489                         <structfield>via1</structfield> - The first Via - parsed.
1490                     </para>
1491                 </listitem>
1492                 <listitem>
1493                     <para>
1494                         <structfield>via2</structfield> - The second Via - parsed.
1495                     </para>
1496                 </listitem>
1497                 <listitem>
1498                     <para>
1499                         <structfield>headers</structfield> - Linked list of all parsed headers.
1500                     </para>
1501                 </listitem>
1502                 <listitem>
1503                     <para>
1504                         <structfield>last_header</structfield> - Pointer to the last parsed header (parsing is incremental,
1505                         that means that the parser will stop if all requested headers were found and next time it will
1506                         continue at the place where it stopped previously. Therefore this field will not point to the
1507                         last header of the message if the whole message hasn't been parsed yet).
1508                     </para>
1509                 </listitem>
1510                 <listitem>
1511                     <para>
1512                         <structfield>parsed_flag</structfield> - Already parsed header field types (bitwise OR).
1513                     </para>
1514                 </listitem>
1515             </itemizedlist>
1516
1517             <para>
1518                 The following fields are set to zero if the corresponding header field was not found in the
1519                 message or hasn't been parsed yet. (These fields are called hooks - they always point to the first
1520                 occurrence if there is more than one header field of the same type).
1521             </para>
1522             <itemizedlist>
1523                 <listitem>
1524                     <para>
1525                         <structfield>h_via1</structfield> - Pointer to the first Via header field.
1526                     </para>
1527                 </listitem>
1528                 <listitem>
1529                     <para>
1530                         <structfield>h_via2</structfield> - Pointer to the second Via header field.
1531                     </para>
1532                 </listitem>
1533                 <listitem>
1534                     <para>
1535                         <structfield>callid</structfield> - Pointer to the first Call-ID header field.
1536                     </para>
1537                 </listitem>
1538                 <listitem>
1539                     <para>
1540                         <structfield>to</structfield> - Pointer to the first To header field.
1541                     </para>
1542                 </listitem>
1543                 <listitem>
1544                     <para>
1545                         <structfield>cseq</structfield> - Pointer to the first CSeq header field.
1546                     </para>
1547                 </listitem>
1548                 <listitem>
1549                     <para>
1550                         <structfield>from</structfield> - Pointer to the first From header field.
1551                     </para>
1552                 </listitem>
1553                 <listitem>
1554                     <para>
1555                         <structfield>contact</structfield> - Pointer to the first Contact header field.
1556                     </para>
1557                 </listitem>
1558                 <listitem>
1559                     <para>
1560                         <structfield>maxforwards</structfield> - Pointer to the first Max-Forwards header field.
1561                     </para>
1562                 </listitem>
1563                 <listitem>
1564                     <para>
1565                         <structfield>route</structfield> - Pointer to the first Route header field.
1566                     </para>
1567                 </listitem>
1568                 <listitem>
1569                     <para>
1570                         <structfield>record_route</structfield> - Pointer to the first Record-Route header field.
1571                     </para>
1572                 </listitem>
1573                 <listitem>
1574                     <para>
1575                         <structfield>content_type</structfield> - Pointer to the first Content-Type header field.
1576                     </para>
1577                 </listitem>
1578                 <listitem>
1579                     <para>
1580                         <structfield>content_length</structfield> - Pointer to the first Content-Length header field.
1581                     </para>
1582                 </listitem>
1583                 <listitem>
1584                     <para>
1585                         <structfield>authorization</structfield> - Pointer to the first Authorization header field.
1586                     </para>
1587                 </listitem>
1588                 <listitem>
1589                     <para>
1590                         <structfield>expires</structfield> - Pointer to the first Expires header field.
1591                     </para>
1592                 </listitem>
1593                 <listitem>
1594                     <para>
1595                         <structfield>proxy_auth</structfield> - Pointer to the first Proxy-Authorize header field.
1596                     </para>
1597                 </listitem>
1598                 <listitem>
1599                     <para>
1600                         <structfield>www_auth</structfield> - Pointer to the first WWW-Authorize header field.
1601                     </para>
1602                 </listitem>
1603                 <listitem>
1604                     <para>
1605                         <structfield>supported</structfield> - Pointer to the first Supported header field.
1606                     </para>
1607                 </listitem>
1608                 <listitem>
1609                     <para>
1610                         <structfield>require</structfield> - Pointer to the first Require header field.
1611                     </para>
1612                 </listitem>
1613                 <listitem>
1614                     <para>
1615                         <structfield>proxy_require</structfield> - Pointer to the first Proxy-Require header field.
1616                     </para>
1617                 </listitem>
1618                 <listitem>
1619                     <para>
1620                         <structfield>unsupported</structfield> - Pointer to the first Unsupported header field.
1621                     </para>
1622                 </listitem>
1623                 <listitem>
1624                     <para>
1625                         <structfield>allow</structfield> - Pointer to the first Allow header field.
1626                     </para>
1627                 </listitem>
1628                 <listitem>
1629                     <para>
1630                         <structfield>event</structfield> - Pointer to the first Event header field.
1631                     </para>
1632                 </listitem>
1633             </itemizedlist>
1634
1635             <para>
1636                 The following fields are mostly used internally by the server and should be modified through dedicated
1637                 functions only.
1638             </para>
1639
1640             <itemizedlist>
1641                 <listitem>
1642                     <para>
1643                         <structfield>eoh</structfield> - Pointer to the End of Header or null if not found yet (the field
1644                         will be set if and only if the whole message was parsed already).
1645                     </para>
1646                 </listitem>
1647                 <listitem>
1648                     <para>
1649                         <structfield>unparsed</structfield> - Pointer to the first unparsed character in the message.
1650                     </para>
1651                 </listitem>
1652                 <listitem>
1653                     <para>
1654                         <structfield>src_ip</structfield> - Sender's <acronym>IP</acronym> address.
1655                     </para>
1656                 </listitem>
1657                 <listitem>
1658                     <para>
1659                         <structfield>dst_ip</structfield> - Destination's <acronym>IP</acronym> address.
1660                     </para>
1661                 </listitem>
1662                 <listitem>
1663                     <para>
1664                         <structfield>orig</structfield> - Original (unmodified) message copy, this field will hold
1665                         unmodified copy of the message during the whole message lifetime.
1666                     </para>
1667                 </listitem>
1668                 <listitem>
1669                     <para>
1670                         <structfield>buf</structfield> - Message scratch-pad (modified copy of the message) - All modifications
1671                         made to the message will be done here.
1672                     </para>
1673                 </listitem>
1674                 <listitem>
1675                     <para>
1676                         <structfield>len</structfield> - Length of the message (unmodified).
1677                     </para>
1678                 </listitem>
1679             </itemizedlist>
1680             
1681             <itemizedlist>
1682                 <listitem>
1683                     <para>
1684                         <structfield>new_uri</structfield> - New Request-URI to be used when forwarding the message.
1685                     </para>
1686                 </listitem>
1687                 <listitem>
1688                     <para>
1689                         <structfield>parsed_uri_ok</structfield> - 1 if <structfield>parsed_uri</structfield> is 
1690                         valid, 0 if not.
1691                     </para>
1692                 </listitem>
1693                 <listitem>
1694                     <para>
1695                         <structfield>parsed_uri</structfield> - The original parsed Request <acronym>URI</acronym>, sometimes 
1696                         it might be necessary to revert changes made to the Request <acronym>URI</acronym> and therefore we
1697                         store the original <acronym>URI</acronym> here.
1698                     </para>
1699                 </listitem>
1700             </itemizedlist>
1701
1702             <itemizedlist>
1703                 <listitem>
1704                     <para>
1705                         <structfield>add_rm</structfield> - Linked list describing all modifications that will be made to
1706                         <emphasis>REQUEST</emphasis> before it will be forwarded. The list will be processed when the request
1707                         is being converted to character array (i.e. immediately before the request will be send out).
1708                     </para>
1709                 </listitem>
1710                 <listitem>
1711                     <para>
1712                         <structfield>repl_add_rm</structfield> - Linked list describing all modifications 
1713                         that will be made to
1714                         <emphasis>REPLY</emphasis> before it will be forwarded. the list will be processed when the reply
1715                         is being converted to character array (i.e. immediately before the request will be send out).
1716                     </para>
1717                 </listitem>
1718                 <listitem>
1719                     <para>
1720                         <structfield>reply_lump</structfield> - This is list of data chunks that should be appended to 
1721                         locally generated reply, i.e. when the server is generating local reply out of the request. A local
1722                         reply is reply generated by the server. For example, when processing of a request fails for some
1723                         reason, the server might generate an error reply and send it back to sender.
1724                     </para>
1725                 </listitem>
1726             </itemizedlist>
1727             
1728             <itemizedlist>
1729                 <listitem>
1730                     <para>
1731                         <structfield>add_to_branch_s</structfield> - String to be appended to branch parameter.
1732                     </para>
1733                 </listitem>
1734                 <listitem>
1735                     <para>
1736                         <structfield>add_to_branch_len</structfield> - Length of the string.
1737                     </para>
1738                 </listitem>
1739             </itemizedlist>
1740             
1741             <itemizedlist>
1742                 <listitem>
1743                     <para>
1744                         <structfield>hash_index</structfield> - Index to a hash table in TM module.
1745                     </para>
1746                 </listitem>
1747                 <listitem>
1748                     <para>
1749                         <structfield>flags</structfield> - Allows to set various flags on the message. May be used 
1750                         for simple inter-module communication or remembering processing state reached.
1751                     </para>
1752                 </listitem>
1753             </itemizedlist>
1754         </section> <!-- sip_msg -->
1755     </chapter> <!-- internal-data-struct -->
1756         
1757     <chapter id="routing-engine">
1758         <title>The Routing Engine</title>
1759         <para>
1760             In a previous section we discussed how routing part of a config file gets translated into binary
1761             representation. In this section, we will discuss how the binary representation is used during
1762             message processing.
1763         </para>
1764         <para>
1765             Upon a <acronym>SIP</acronym> message receipt, the server performs some basic sanity checks and converts
1766             the message into <structname>sip_msg</structname> structure. After that the Routing Engine will start
1767             processing the message.
1768         </para>
1769         <para>
1770             The routing engine can be found in file <filename moreinfo="none">action.c</filename>.
1771         </para>
1772         <para>
1773             The main function is <function moreinfo="none">run_actions.</function> The 
1774             function accepts two parameters. The first parameter is list of actions to
1775             be processed (Remember, the config file gets translated into array of linked
1776             lists. Each linked list in the array represents one "route" part of the config
1777             file). The second parameter is <structname>sip_msg</structname> structure
1778             representing the message to be processed.
1779         </para>
1780         <para>
1781             Upon a receipt of a request, the linked list representing the main route
1782             part will be processed so the first parameter will be <varname>rlist[0]</varname>. (The linked
1783             list of main route part is always at index 0).
1784         </para>
1785         <para>
1786             The function will then sequentially call 
1787             <function moreinfo="none">do_action</function> function for each element
1788             of the linked list. Return value of the function is important. If the function
1789             returns 0, processing of the list will be stopped. By returning 0 a command
1790             can indicate that processing of the message should be stopped and the message will be dropped.
1791         </para>
1792         <para>
1793             Modules may export so-called <emphasis>on_break handlers</emphasis>.
1794             <emphasis>on_break</emphasis> handler is a function, that will be called when
1795             processing of the linked-list is interrupted (ret == 0). All such handlers
1796             will be called when processing of the linked-list is finished and ret == 0.
1797         </para>
1798         <section id="do-action">
1799             <title><function moreinfo="none">do_action</function> Function</title>
1800             <para>
1801                 <function moreinfo="none">do_action</function> function is core of the
1802                 routing engine. There is a big <function moreinfo="none">switch</function>
1803                 statement. Each case of the statements is one command handled by the
1804                 server core itself.
1805             </para>
1806             <para>
1807                 The following commands are handled by the <acronym>SER</acronym> core
1808                 itself: 
1809                 <function moreinfo="none">drop</function>,
1810                 <function moreinfo="none">forward</function>,
1811                 <function moreinfo="none">send</function>,
1812                 <function moreinfo="none">log</function>,
1813                 <function moreinfo="none">append_branch</function>,
1814                 <function moreinfo="none">len_gt</function>,
1815                 <function moreinfo="none">setflag</function>,
1816                 <function moreinfo="none">resetflag</function>,
1817                 <function moreinfo="none">isflagset</function>,
1818                 <function moreinfo="none">error</function>,
1819                 <function moreinfo="none">route</function>,
1820                 <function moreinfo="none">exec</function>,
1821                 <function moreinfo="none">revert_uri</function>,
1822                 <function moreinfo="none">set_host</function>,
1823                 <function moreinfo="none">set_hostport</function>,
1824                 <function moreinfo="none">set_user</function>,
1825                 <function moreinfo="none">set_userpass</function>,
1826                 <function moreinfo="none">set_port</function>,
1827                 <function moreinfo="none">set_uri</function>,
1828                 <function moreinfo="none">prefix</function>,
1829                 <function moreinfo="none">strip</function>,
1830                 <function moreinfo="none">if</function>,
1831                 <function moreinfo="none">module</function>.
1832             </para>
1833             <para>
1834                 Each of the commands is represented by a <emphasis>case</emphasis> statement in the switch.
1835                 (For example, if you are interested in implementation of 
1836                 <function>drop</function> command,
1837                 look at <quote>case DROP_T:</quote> statement in the function.
1838             </para>
1839             <para>
1840                 The respective commands will be described now.
1841             </para>
1842             <itemizedlist>
1843                 <listitem>
1844                     <para>
1845                         <function moreinfo="none">drop</function> - This command is very
1846                         simple, it simply returns 0 which will result in abortion of
1847                         processing of the request. No other commands after
1848                         <function moreinfo="none">drop</function> will be executed.
1849                     </para>
1850                 </listitem>
1851                 <listitem>
1852                     <para>
1853                         <function moreinfo="none">forward</function> - The function will
1854                         forward the message further. The message will be either forwarded
1855                         to the Request <acronym>URI</acronym> of the message or to 
1856                         <acronym>IP</acronym> or host given as parameter.
1857                     </para>
1858                     <para>
1859                         In the first case, host in the Request <acronym>URI</acronym>
1860                         must be converted into corresponding <acronym>IP</acronym> address.
1861                         Function <function moreinfo="none">mk_proxy</function> converts
1862                         hostname to corresponding <acronym>IP</acronym> address.
1863                         The message is then sent out using 
1864                         <function moreinfo="none">forward_request</function> function.
1865                     </para>
1866                     <para>
1867                         In the second case, hostname was converted to <acronym>IP</acronym>
1868                         address in fixup i.e. immediately after the config file was compiled
1869                         into its binary representation. The first parameter is pointer to
1870                         <structname>proxy</structname> structure created in the fixup and
1871                         therefore we only need to call 
1872                         <function moreinfo="none">forward_request</function> here to
1873                         forward the message further.
1874                     </para>
1875                 </listitem>
1876                 <listitem>
1877                     <para>
1878                         <function moreinfo="none">send</function> - This functions sends
1879                         the message to a third-party host. The message will be sent out
1880                         as is - i.e. without Request <acronym>URI</acronym> and Via
1881                         altering.
1882                     </para>
1883                     <para>
1884                         Hostname or <acronym>IP</acronym> address of the third-party host
1885                         is specified as a parameter of the function.
1886                     </para>
1887                     <para>
1888                         The message will be sent out using 
1889                         <function moreinfo="none">udp_send</function> directly.
1890                     </para>
1891                 </listitem>
1892                 <listitem>
1893                     <para>
1894                         <function moreinfo="none">log</function> - The message given as
1895                         a parameter will be logged using system logger. It can be either
1896                         <command moreinfo="none">syslog</command> or <varname>stderr</varname> 
1897                         (depends on configuration). The message is logged
1898                         using <function moreinfo="none">LOG</function> which is a macro
1899                         defined in <filename moreinfo="none">dprint.h</filename> header
1900                         file.
1901                     </para>
1902                 </listitem>
1903                 <listitem>
1904                     <para>
1905                         <function moreinfo="none">append_branch</function> - Append a new
1906                         <acronym>URI</acronym> for forking.
1907                     </para>
1908                     <para>
1909                         More than one destinations may be associated with a single
1910                         <acronym>SIP</acronym> request. If the server was configured so,
1911                         it will use all the destinations and fork the request. 
1912                     </para>
1913                     <para>
1914                         The server keeps an array of all destinations, that should be
1915                         used when forking. The array and related functions can be found
1916                         in file <filename moreinfo="none">dset.c</filename>. There is
1917                         function <function moreinfo="none">append_branch</function> which adds a new destination to the set.
1918                     </para>
1919                     <para>
1920                         This command simply calls 
1921                         <function moreinfo="none">append_branch</function> function and
1922                         adds a new destination to the destination set.
1923                     </para>
1924                 </listitem>
1925                 <listitem>
1926                     <para>
1927                         <function moreinfo="none">len_gt</function> - The command accepts
1928                         one number as a parameter. It then compares the number with length
1929                         of the message. If the message length is greater or equal then
1930                         the number then 1 will be returned otherwise the function returns -1.
1931                     </para>
1932                 </listitem>
1933                 <listitem>
1934                     <para>
1935                         <function moreinfo="none">setflag</function> - Sets a flag in the
1936                         message. The command simply 
1937                         calls <function moreinfo="none">setflags</function> function that
1938                         will set the flag. Fore more information see file
1939                         <filename moreinfo="none">flag.c</filename>.
1940                     </para>
1941                 </listitem>
1942                 <listitem>
1943                     <para>
1944                         <function moreinfo="none">resetflag</function> - Same as command
1945                         <function moreinfo="none">setflag</function> - only resetflag
1946                         will be called instead of setflag.
1947                     </para>
1948                 </listitem>
1949                 <listitem>
1950                     <para>
1951                         <function moreinfo="none">isflagset</function> - Test if the flag
1952                         is set or not.
1953                     </para>
1954                 </listitem>
1955                 <listitem>
1956                     <para>
1957                         <function moreinfo="none">error</function> - Log a message with
1958                         NOTICE log level.
1959                     </para>
1960                 </listitem>
1961                 <listitem>
1962                     <para>
1963                         <function moreinfo="none">route</function> - Execute another
1964                         route statement.
1965                     </para>
1966                     <para>
1967                         As we have mentioned already, there can be more that one route
1968                         statement in the config file. One of them is main (without
1969                         number), the other are additional. This command makes it possible
1970                         to execute an additional route statement.
1971                     </para>
1972                     <para>
1973                         The command accepts one parameter which is route statement number.
1974                         First sanity checks over the parameter will be performed. If
1975                         the checks passed, function 
1976                         <function moreinfo="none">run_actions</function> will be called.
1977                         The function accepts two parameters. The first one is linked list
1978                         to execute, the second one is <structname>sip_msg</structname>
1979                         structure representing the message to be processed.
1980                     </para>
1981                     <para>
1982                         As you might remember, each route statement was compiled into
1983                         linked list of commands to be executed and head of the linked
1984                         list was stored in <varname>rlist</varname> array. For example, head of linked list
1985                         representing route statement with number 4 will be stored at
1986                         position 4 in the array (position 0 is reserved for the main
1987                         route statement).
1988                     </para>
1989                     <para>
1990                         So the command will simply call 
1991                         <function moreinfo="none">run_actions(rlist[a->p1.number], msg)</function>
1992                         and that will execute route statement with number given as parameter.
1993                     </para>
1994                 </listitem>
1995                 <listitem>
1996                     <para>
1997                         <function moreinfo="none">exec</function> - Execute a shell command.
1998                     </para>
1999                     <para>
2000                         The command accepts one parameter of type <type>char*</type>. The string given as
2001                         parameter will be passed to <function moreinfo="none">system</function>
2002                         function which will in turn execute
2003                         <function moreinfo="none">/bin/sh -c &lt;string&gt;</function>.
2004                     </para>
2005                 </listitem>
2006                 <listitem>
2007                     <para>
2008                         <function moreinfo="none">revert_uri</function> - Revert changes made to
2009                         the Request <acronym>URI</acronym>.
2010                     </para>
2011                     <para>
2012                         If there is a new <acronym>URI</acronym> stored in 
2013                         <structfield>new_uri</structfield> of <structname>sip_msg</structname> structure,
2014                         it will be freed. The original Request <acronym>URI</acronym> will be used
2015                         when forwarding the message.
2016                     </para>
2017                     <para>
2018                         If there is a valid <acronym>URI</acronym> in 
2019                         <structfield>parsed_uri</structfield> field of <structname>sip_msg</structname>
2020                         structure (indicated by <structfield>parsed_uri_ok</structfield> field), it
2021                         will be freed too.
2022                     </para>
2023                 </listitem>
2024                 <listitem>
2025                     <para>
2026                         <function moreinfo="none">set_host</function> - Change hostname of Request
2027                         <acronym>URI</acronym> to value given as parameter.
2028                     </para>
2029                     <para>
2030                         If there is a <acronym>URI</acronym> in <structfield>new_uri</structfield>
2031                         field, it will be modified, otherwise the original Request
2032                         <acronym>URI</acronym> will be modified.
2033                     </para>
2034                 </listitem>
2035                 <listitem>
2036                     <para>
2037                         <function moreinfo="none">set_hostport</function> - change hostname and port
2038                         of Request <acronym>URI</acronym> to value given as string parameter.
2039                     </para>
2040                     <para>
2041                         If there is a <acronym>URI</acronym> in <structfield>new_uri</structfield>
2042                         field, it will be modified, otherwise the original Request
2043                         <acronym>URI</acronym> will be modified.
2044                     </para>                     
2045                 </listitem>
2046                 <listitem>
2047                     <para>
2048                         <function moreinfo="none">set_user</function> - Set username part of Request
2049                         <acronym>URI</acronym> to string given as parameter.
2050                     </para>
2051                     <para>
2052                         If there is a <acronym>URI</acronym> in <structfield>new_uri</structfield>
2053                         field, it will be modified, otherwise the original Request
2054                         <acronym>URI</acronym> will be modified.
2055                     </para>
2056                 </listitem>
2057                 <listitem>
2058                     <para>
2059                         <function moreinfo="none">set_userpass</function> - Set username and password
2060                         part of Request <acronym>URI</acronym> to string given as parameter.
2061                     </para>
2062                     <para>
2063                         If there is a <acronym>URI</acronym> in <structfield>new_uri</structfield>
2064                         field, it will be modified, otherwise the original Request
2065                         <acronym>URI</acronym> will be modified.
2066                     </para>                     
2067                 </listitem>
2068                 <listitem>
2069                     <para>
2070                         <function moreinfo="none">set_port</function> - Set port of Request
2071                         <acronym>URI</acronym> to value given as parameter.
2072                     </para>
2073                     <para>
2074                         If there is a <acronym>URI</acronym> in <structfield>new_uri</structfield>
2075                         field, it will be modified, otherwise the original Request
2076                         <acronym>URI</acronym> will be modified.
2077                     </para>                     
2078                 </listitem>
2079                 <listitem>
2080                     <para>
2081                         <function moreinfo="none">set_uri</function> - Set a new Request
2082                         <acronym>URI</acronym>.
2083                     </para>
2084                     <para>
2085                         If there is a <acronym>URI</acronym> in <structfield>new_uri</structfield> field,
2086                         it will be freed. If there is a valid <acronym>URI</acronym> in 
2087                         <structfield>parsed_uri</structfield> field, it will be freed too.
2088                     </para>
2089                     <para>
2090                         Then <acronym>URI</acronym> given as parameter will be stored in 
2091                         <structfield>new_uri</structfield> field. (If <structfield>new_uri</structfield>
2092                         contains a <acronym>URI</acronym> it will be used instead of Request
2093                         <acronym>URI</acronym> when forwarding the message).
2094                     </para>
2095                 </listitem>
2096                 <listitem>
2097                     <para>
2098                         <function moreinfo="none">prefix</function> - Set the parameter as username
2099                         prefix.
2100                     </para>
2101                     <para>
2102                         The string will be put immediately after <quote>sip:</quote> part of the 
2103                         Request <acronym>URI</acronym>.
2104                     </para>
2105                     <para>
2106                         If there is a <acronym>URI</acronym> in <structfield>new_uri</structfield>
2107                         field, it will be modified, otherwise the original Request
2108                         <acronym>URI</acronym> will be modified.
2109                     </para>                     
2110                 </listitem>
2111                 <listitem>
2112                     <para>
2113                         <function moreinfo="none">strip</function> - Remove first n characters of
2114                         username in Request <acronym>URI</acronym>.
2115                     </para>
2116                     <para>
2117                         If there is a <acronym>URI</acronym> in <structfield>new_uri</structfield>
2118                         field, it will be modified, otherwise the original Request
2119                         <acronym>URI</acronym> will be modified.
2120                     </para>                     
2121                 </listitem>
2122                 <listitem>
2123                     <para>
2124                         <function moreinfo="none">if</function> - if Statement.
2125                     </para>
2126                     <para>
2127                         There is an expression associated with the command and one or two linked lists of
2128                         commands. The expression is a regular expression compiled into binary form
2129                         in the fixup when the config file was compiled.
2130                     </para>
2131                     <para>
2132                         The expression will be evaluated now. If the result is &gt; 0, the first linked
2133                         list will be executed using <function moreinfo="none">run_action</function>
2134                         function. The linked list represents command enclosed in curly braces of 
2135                         <function moreinfo="none">if</function> command.
2136                     </para>
2137                     <para>
2138                         Otherwise, if there is the second list, it will be executed in the same way. The
2139                         second list represents commands of <function moreinfo="none">else</function>
2140                         statement.
2141                     </para>
2142                 </listitem>
2143                 <listitem>
2144                     <para>
2145                         <function moreinfo="none">module</function> - Execute a function exported by
2146                         a module.
2147                     </para>
2148                     <para>
2149                         When a command in a route statement is not recognized by the core itself
2150                         (i.e. it is not one of commands handled by the core itself), list of exported
2151                         functions of all loaded modules will be searched for a function with corresponding
2152                         name and number of parameters.
2153                     </para>
2154                     <para>
2155                         If the function was found, <function moreinfo="none">module</function> command
2156                         (this one) will be created and pointer to the function will be stored in 
2157                         <structfield>p1.data</structfield> field.
2158                     </para>
2159                     <para>
2160                         So, this command will simply call function whose pointer is in
2161                         <structfield>p1.data</structfield> field and will pass 2 parameters to the
2162                         function. If one or both of the parameters were not used, 0 will be passed
2163                         instead. 
2164                     </para>
2165                     <para>
2166                         Return value of the function will be returned as return value of
2167                         <function moreinfo="none">module</function> command.
2168                     </para>
2169                     <para>
2170                         This command makes <acronym>SER</acronym> pretty extensible while the core
2171                         itself is still reasonably small and clean. Additional functionality is put
2172                         in modules and loaded only when needed.
2173                     </para>
2174                 </listitem>
2175             </itemizedlist>
2176         </section> <!-- do-action -->
2177     </chapter> <!-- routing-engine -->
2178     
2179 <!--
2180     <chapter id="memory-management">
2181         <title>Memory Management</title>
2182         <para>
2183             This section should be written by somebody who really knows how it works. Any volunteers ?
2184             TBD.
2185         </para>
2186     </chapter>
2187 -->
2188
2189     <chapter id="msg-parser">
2190         <title>The Message Parser</title>
2191         <para>
2192             In this section we will discuss internals of the <acronym>SIP</acronym> message header
2193             parser implemented in the server. Message parsing is very important and one of the most
2194             time consuming operations of a <acronym>SIP</acronym> server. We have been trying to
2195             make the parser as fast as possible.
2196         </para>
2197         
2198         <para>
2199             A header field parser can be either in the server core or in a module. By convention,
2200             parser that is needed by the core itself or is needed by at least two modules will be
2201             in the core. Parsers contained in modules will be not described in this section.
2202         </para>
2203         <para>
2204             There is a <filename moreinfo="none">parser</filename> subdirectory that contains all
2205             the parsers and related stuff.
2206         </para>
2207         <para>
2208             The following parsers can be found under <filename moreinfo="none">parser</filename>
2209             subdirectory:
2210         </para>
2211
2212         <section id="sip-msg-structure">
2213             <title>Structure of a <acronym>SIP</acronym> Message</title>
2214             <para>
2215                 A <acronym>SIP</acronym> message consists of message header and optional message body. 
2216                 The header is separated from the body with a empty line (containing CRLF only).
2217             </para>
2218             <para>
2219                 Message header consists of the first line and one or more header fields. The first line
2220                 determines type of the message. Header fields provide additional information that is
2221                 needed by clients and servers to be able to process the message.
2222             </para>
2223             <para>
2224                 Each header field consists of header field name and header field body. Header field name
2225                 is delimited from header field body by a colon (<quote>:</quote>). For example,
2226                 <quote>Server: SIP Express Router</quote> - in this case <quote>Server</quote> is 
2227                 header field name and <quote>SIP Express Router</quote> is header field body.
2228             </para>
2229         </section> <!-- sip-msg-structure -->
2230         
2231
2232         <section id="parser-organization">
2233             <title>The Parser Organization</title>
2234
2235             <itemizedlist>
2236                 <listitem>
2237                     <para>First Line Parser - Parses the first line of a <acronym>SIP</acronym> message.</para>
2238                 </listitem>
2239                 <listitem>
2240                     <para>Header Name Parser- Parsers Name part of a header field (part before colon).</para>
2241                 </listitem>             
2242                 <listitem>
2243                     <para>To Header Parser - Parses body of To header field.</para>
2244                 </listitem>
2245                 <listitem>
2246                     <para>From Header Parser - Parses body of From header field.</para>
2247                 </listitem>
2248                 <listitem>
2249                     <para>CSeq Header Parser - Parses body of CSeq header field.</para>
2250                 </listitem>
2251                 <listitem>
2252                 <para>Event Header Parser - Parses body of Event header field.</para>
2253                 </listitem>
2254                 <listitem>
2255                     <para>Expires Header Parser - Parses body of Expires header field.</para>
2256                 </listitem>
2257                 <listitem>
2258                     <para>Via Header Parser - Parses body of Via header field.</para>
2259                 </listitem>
2260                 <listitem>
2261                     <para>Contact Header Parser - Parses body of Contact header field.</para>
2262                 </listitem>
2263                 <listitem>
2264                     <para>Digest Parser - Parses digest response.</para>
2265                 </listitem>
2266             </itemizedlist>
2267             
2268             <para>
2269                 The server implements what we call <emphasis>incremental parsing</emphasis>. It means that
2270                 a header field will be not parsed unless it is really needed. There is a minimal set of
2271                 header that will be parsed every time. The set includes:
2272                 <itemizedlist>
2273                     <listitem>
2274                         <para>
2275                             The first line - the server must know if the message is request or response
2276                         </para>
2277                     </listitem>
2278                     <listitem>
2279                         <para>
2280                             Via header field - Via will be needed for sure. We must add ourself to Via
2281                             list when forwarding the message.
2282                         </para>
2283                     </listitem>
2284                 </itemizedlist>
2285             </para>
2286         
2287             <section id="fline-parser">
2288                 <title>The First Line Parser</title>
2289                 <para>
2290                     Purpose of the parser is to parse the first line of a <acronym>SIP</acronym>
2291                     message. The first line is represented by <structname>msg_start</structname>
2292                     structure define in file <filename moreinfo="none">parse_fline.h</filename>
2293                     under <filename moreinfo="none">parser</filename> subdirectory.
2294                 </para>
2295                 <para>
2296                     The main function of the first line parser is 
2297                     <function moreinfo="none">parse_first_line</function>, the function will fill in
2298                     <structname>msg_start</structname> structure.
2299                 </para>
2300                 <para>
2301                     Follow inline comments in the function if you want to add support for a new
2302                     message type.
2303                 </para>
2304             </section> <!-- fline-parser -->
2305             <section id="hfname-parser">
2306                 <title>The Header Field Name Parser</title>
2307                 <para>
2308                     The purpose of the header field type parser is to recognize type of a header field.
2309                     The following types of header field will be recognized:
2310                 </para>
2311                 <para>
2312                     Via, To, From, CSeq, Call-ID, Contact, Max-Forwards, Route, Record-Route,
2313                     Content-Type, Content-Length, Authorization, Expires, Proxy-Authorization,
2314                     WWW-Authorization, supported, Require, Proxy-Require, Unsupported, Allow, Event.
2315                 </para>
2316                 <para>
2317                     All other header field types will be marked as HDR_OTHER.
2318                 </para>
2319                 <para>
2320                     Main function of header name parser is <function moreinfo="none">parse_hname2</function>. The
2321                     function can be found in file <filename moreinfo="none">parse_hname.c</filename>. The function
2322                     accepts pointers to begin and end of a header field and fills in <structname>hdf_field</structname>
2323                     structure. <structfield>name</structfield> field will point to the header field name,
2324                     <structfield>body</structfield> field will point to the header field body and 
2325                     <structfield>type</structfield> field will contain type of the header field if known and HDR_OTHER
2326                     if unknown.
2327                 </para>
2328                 <para>
2329                     The parser is 32-bit, it means, that it processes 4 characters of header field name at time. 4
2330                     characters of a header field name are converted to an integer and the integer is then compared. This
2331                     is much faster than comparing byte by byte. Because the server is compiled on at least 32-bit architectures,
2332                     such comparison will be compiled into one instruction instead of 4 instructions.
2333                 </para>
2334                 <para>
2335                     We did some performance measurement and 32-bit parsing is about 3 times faster for a typical 
2336                     <acronym>SIP</acronym> message than corresponding automaton comparing byte by byte. Performance may vary
2337                     depending on the message size, parsed header fields and header fields type. Test showed that it was always
2338                     as fast as corresponding 1-byte comparing automaton.
2339                 </para>
2340                 <para>
2341                     Since comparison must be case insensitive in case of header field names, it is necessary to convert it
2342                     to lower case first and then compare. Since converting byte by byte would slow down the parser a lot, we
2343                     have implemented a hash table, that can again convert 4 bytes at once. Since set of keys that need to be
2344                     converted to lowercase is known (the set consists of all possible 4-byte parts of all recognized header
2345                     field names) we can pre-calculate size of the hash table to be synonym-less. That will simplify (and speed
2346                     up) the lookup a lot. The hash table must be initialized upon the server startup (function
2347                     <function moreinfo="none">init_hfname_parser</function>).
2348                 </para>
2349                 <para>
2350                     The header name parser consists of several files, all of them are under 
2351                     <filename moreinfo="none">parser</filename> subdirectory. Main file is 
2352                     <filename moreinfo="none">parse_hname2.c</filename> - this files contains the parser itself and functions
2353                     used to initialize and lookup the hash table. File <filename moreinfo="none">keys.h</filename> contains
2354                     automatically generated set of macros. Each macro is a group of 4 bytes converted to integer. The macros
2355                     are used for comparison and the hash table initialization. For example, for Max-Forwards header field
2356                     name, the following macros are defined in the file: 
2357                 </para>
2358                 <programlisting>
2359 #define _max__ 0x2d78616d   /* "max-" */
2360 #define _maX__ 0x2d58616d   /* "maX-" */
2361 #define _mAx__ 0x2d78416d   /* "mAx-" */
2362 #define _mAX__ 0x2d58416d   /* "mAX-" */
2363 #define _Max__ 0x2d78614d   /* "Max-" */
2364 #define _MaX__ 0x2d58614d   /* "MaX-" */
2365 #define _MAx__ 0x2d78414d   /* "MAx-" */
2366 #define _MAX__ 0x2d58414d   /* "MAX-" */
2367
2368 #define _forw_ 0x77726f66   /* "forw" */
2369 #define _forW_ 0x57726f66   /* "forW" */
2370 #define _foRw_ 0x77526f66   /* "foRw" */
2371 #define _foRW_ 0x57526f66   /* "foRW" */
2372 #define _fOrw_ 0x77724f66   /* "fOrw" */
2373 #define _fOrW_ 0x57724f66   /* "fOrW" */
2374 #define _fORw_ 0x77524f66   /* "fORw" */
2375 #define _fORW_ 0x57524f66   /* "fORW" */
2376 #define _Forw_ 0x77726f46   /* "Forw" */
2377 #define _ForW_ 0x57726f46   /* "ForW" */
2378 #define _FoRw_ 0x77526f46   /* "FoRw" */
2379 #define _FoRW_ 0x57526f46   /* "FoRW" */
2380 #define _FOrw_ 0x77724f46   /* "FOrw" */
2381 #define _FOrW_ 0x57724f46   /* "FOrW" */
2382 #define _FORw_ 0x77524f46   /* "FORw" */
2383 #define _FORW_ 0x57524f46   /* "FORW" */
2384
2385 #define _ards_ 0x73647261   /* "ards" */
2386 #define _ardS_ 0x53647261   /* "ardS" */
2387 #define _arDs_ 0x73447261   /* "arDs" */
2388 #define _arDS_ 0x53447261   /* "arDS" */
2389 #define _aRds_ 0x73645261   /* "aRds" */
2390 #define _aRdS_ 0x53645261   /* "aRdS" */
2391 #define _aRDs_ 0x73445261   /* "aRDs" */
2392 #define _aRDS_ 0x53445261   /* "aRDS" */
2393 #define _Ards_ 0x73647241   /* "Ards" */
2394 #define _ArdS_ 0x53647241   /* "ArdS" */
2395 #define _ArDs_ 0x73447241   /* "ArDs" */
2396 #define _ArDS_ 0x53447241   /* "ArDS" */
2397 #define _ARds_ 0x73645241   /* "ARds" */
2398 #define _ARdS_ 0x53645241   /* "ARdS" */
2399 #define _ARDs_ 0x73445241   /* "ARDs" */
2400 #define _ARDS_ 0x53445241   /* "ARDS" */
2401 </programlisting>
2402                 <para>
2403                     As you can see, Max-Forwards name was divided into three 4-byte chunks: Max-, Forw, ards. The file
2404                     contains macros for every possible lower and upper case character combination of the chunks. Because
2405                     the name (and therefore chunks) can contain colon (<quote>:</quote>), minus or space and these 
2406                     characters are not allowed in macro name, they must be substituted. Colon is substituted by 
2407                     <quote>1</quote>, minus is substituted by underscore (<quote>_</quote>) and space is substituted
2408                     by <quote>2</quote>.
2409                 </para>
2410
2411                 <para>
2412                     When initializing the hash table, all these macros will be used as keys to the hash table. One of
2413                     each upper and lower case combinations will be used as value. Which one ?
2414                 </para>
2415                 <para>
2416                     There is a convention that each word of a header field name starts with a upper case character. For
2417                     example, most of user agents will send <quote>Max-Forwards</quote>, messages containing some other 
2418                     combination of upper and lower case characters (for example: <quote>max-forwards</quote>,
2419                     <quote>MAX-FORWARDS</quote>, <quote>mAX-fORWARDS</quote>) are very rare (but it is possible).
2420                 </para>
2421                 <para>
2422                     Considering the previous paragraph, we optimized the parser for the most common case. When all header
2423                     fields have upper and lower case characters according to the convention, there is no need to do hash
2424                     table lookups, which is another speed up.
2425                 </para>
2426                 <para>
2427                     For example suppose we are trying to figure out if the header field name is Max-Forwards and the
2428                     header field name is formed according to the convention (i.e. <quote>Max-Forwards</quote>):
2429                     <itemizedlist>
2430                         <listitem>
2431                             <para>
2432                                 Get the first 4 bytes of the header field name (<quote>Max-</quote>), convert it to an 
2433                                 integer and compare to <quote>_Max__</quote> macro. Comparison succeeded, continue
2434                                 with the next step.
2435                             </para>
2436                         </listitem>
2437                         <listitem>
2438                             <para>
2439                                 Get next 4 bytes of the header field name (<quote>Forw</quote>), convert it to an
2440                                 integer and compare to <quote>_Forw_</quote> macro. Comparison succeeded, continue
2441                                 with the next step.
2442                             </para>
2443                         </listitem>
2444                         <listitem>
2445                             <para>
2446                                 Get next 4 bytes of the header field name (<quote>ards</quote>), convert it to an
2447                                 integer and compare to <quote>_ards_</quote> macro. Comparison succeeded, continue
2448                                 with the next step.
2449                             </para>
2450                         </listitem>
2451                         <listitem>
2452                             <para>
2453                                 If the following characters are spaces and tabs followed by a colon (or colon directly
2454                                 without spaces and tabs), we found Max-Forwards header field name and can set
2455                                 <structfield>type</structfield> field to HDR_MAXFORWARDS. Otherwise (other characters
2456                                 than colon, spaces and tabs) it is some other header field and set 
2457                                 <structfield>type</structfield> field to HDR_OTHER.
2458                             </para>
2459                         </listitem>
2460                     </itemizedlist>
2461                 </para>
2462                 <para>
2463                     As you can see, there is no need to do hash table lookups if the header field was formed according
2464                     to the convention and the comparison was very fast (only 3 comparisons needed !).
2465                 </para>
2466                 <para>
2467                     Now lets consider another example, the header field was not formed according to the convention, for
2468                     example <quote>MAX-forwards</quote>:
2469                     <itemizedlist>
2470                         <listitem>
2471                             <para>
2472                                 Get the first 4 bytes of the header field name (<quote>MAX-</quote>), convert it to an
2473                                 integer and compare to <quote>_Max__</quote> macro.
2474                             </para>
2475                             <para>
2476                                 Comparison failed, try to lookup <quote>MAX-</quote> converted to integer in the hash
2477                                 table. It was found, result is <quote>Max-</quote> converted to integer.
2478                             </para>
2479                             <para>
2480                                 Try to compare the result from the hash table to <quote>_Max__</quote> macro. Comparison
2481                                 succeeded, continue with the next step.
2482                             </para>
2483                         </listitem>
2484                         <listitem>
2485                             <para>
2486                                 Compare next 4 bytes of the header field name (<quote>forw</quote>), convert it to an
2487                                 integer and compare to <quote>_Max__</quote> macro.
2488                             </para>
2489                             <para>
2490                                 Comparison failed, try to lookup <quote>forw</quote> converted to integer in the hash
2491                                 table. It was found, result is <quote>Forw</quote> converted to integer.
2492                             </para>
2493                             <para>
2494                                 Try to compare the result from the hash table to <quote>Forw</quote> macro. Comparison
2495                                 succeeded, continue with the next step.
2496                             </para>
2497                         </listitem>
2498                         <listitem>
2499                             <para>
2500                                 Compare next 4 bytes of the header field name (<quote>ards</quote>), convert it to
2501                                 integer and compare to <quote>ards</quote> macro. Comparison succeeded, continue
2502                                 with the next step.
2503                             </para>
2504                         </listitem>
2505                         <listitem>
2506                             <para>
2507                                 If the following characters are spaces and tabs followed by a colon (or colon directly
2508                                 without spaces and tabs), we found Max-Forwards header field name and can set
2509                                 <structfield>type</structfield> field to HDR_MAXFORWARDS. Otherwise (other characters
2510                                 than colon, spaces and tabs) it is some other header field and set 
2511                                 <structfield>type</structfield> field to HDR_OTHER.                                 
2512                             </para>
2513                         </listitem>
2514                     </itemizedlist>
2515                 </para>
2516                 <para>
2517                     In this example, we had to do 2 hash table lookups and 2 more comparisons. Even this variant is still
2518                     very fast, because the hash table lookup is synonym-less, lookups are very fast.
2519                 </para>
2520             </section> <!-- hfname-parser -->
2521
2522             <section id="body-parsers">
2523                 <title>The Header Field Body Parsers</title>
2524                 <section id="to-parser">
2525                     <title>To HF Body Parser</title>
2526                     <para>
2527                         Purpose of this parser is to parse body of To header field. The parser can be found in file
2528                         <filename moreinfo="none">parse_to.c</filename> under <filename moreinfo="none">parser</filename>
2529                         subdirectory.
2530                     </para>
2531                     <para>
2532                         Main function is <function moreinfo="none">parse_to</function> but there is no need to call the
2533                         function explicitly. Every time the parser finds a To header field, this function will be called
2534                         automatically. Result of the parser is <structname>to_body</structname> structure. Pointer to the
2535                         structure will be stored in <structfield>parsed</structfield> field of 
2536                         <structname>hdr_field</structname> structure. Since the pointer is <type>void*</type>, there is a
2537                         convenience macro <function moreinfo="none">get_to</function> in file 
2538                         <filename moreinfo="none">parse_to.h</filename> that will do the necessary type-casting and will
2539                         return pointer to <structname>to_body</structname> structure.
2540                     </para>
2541                     <para>
2542                         The parser itself is a finite state machine that will parse To body according to the grammar defined
2543                         in <acronym>RFC3261</acronym> and store result in <structname>to_body</structname> structure.
2544                     </para>
2545                     <para>
2546                         The parser gets called automatically from function <function moreinfo="none">get_hdr_field</function>
2547                         in file <filename moreinfo="none">msg_parser.c</filename>. The function first creates and initializes 
2548                         an instance of <structname>to_body</structname> structure, then calls 
2549                         <function moreinfo="none">parse_to</function> function with the structure as a parameter and if
2550                         everything went OK, puts the pointer to the structure in <structfield>parsed</structfield> field of
2551                         <structname>hdr_field</structname> structure representing the parsed To header field.
2552                     </para>
2553                     <para>
2554                         The newly created structure will be freed when the message is being destroyed, see function
2555                         <function moreinfo="none">clean_hdr_field</function> in file <filename moreinfo="none">hf.c</filename>
2556                         for more details.
2557                     </para>
2558                     
2559                     <section id="to-body-struct">
2560                         <title>Structure <structname>to_body</structname></title>
2561                         <para>
2562                             The structure represents parsed To body. The structure is declared in 
2563                             <filename moreinfo="none">parse_to.h</filename> file.
2564                         </para>
2565                         <para><emphasis>Structure Declaration:</emphasis></para>
2566                         <programlisting format="linespecific">
2567 struct to_param{
2568     int type;              /* Type of parameter */
2569     str name;              /* Name of parameter */
2570     str value;             /* Parameter value */
2571     struct to_param* next; /* Next parameter in the list */
2572 };
2573
2574
2575 struct to_body{
2576     int error;                    /* Error code */
2577     str body;                     /* The whole header field body */
2578     str uri;                      /* URI */
2579     str tag_value;                /* Value of tag */
2580     struct to_param *param_lst;   /* Linked list of parameters */
2581     struct to_param *last_param;  /* Last parameter in the list */
2582 };
2583 </programlisting>
2584                         <para>
2585                             Structure <structname>to_param</structname> is a temporary structure representing
2586                             a To <acronym>URI</acronym> parameter. Right now only TAG parameter will be marked
2587                             in <structfield>type</structfield> field. All other parameters will have the same
2588                             type.
2589                         </para>
2590                         <para><emphasis>Field Description:</emphasis></para>
2591                         <itemizedlist>
2592                             <listitem>
2593                                 <para>
2594                                     <structfield>error</structfield> - Error code will be put here when parsing of To
2595                                     body fails.
2596                                 </para>
2597                             </listitem>
2598                             <listitem>
2599                                 <para>
2600                                     <structfield>body</structfield> - The whole header field body.
2601                                 </para>
2602                             </listitem>
2603                             <listitem>
2604                                 <para>
2605                                     <structfield>uri</structfield> - <acronym>URI</acronym> of the To header field.
2606                                 </para>
2607                             </listitem>
2608                             <listitem>
2609                                 <para>
2610                                     <structfield>tag_value</structfield> - Value of tag parameter if present.
2611                                 </para>
2612                             </listitem>
2613                             <listitem>
2614                                 <para>
2615                                     <structfield>param_lst</structfield> - Linked list of all parameters.
2616                                 </para>
2617                             </listitem>
2618                             <listitem>
2619                                 <para>
2620                                     <structfield>last_param</structfield> - Pointer to the last parameter in the linked list.
2621                                 </para>
2622                             </listitem>
2623                         </itemizedlist>
2624                     </section> <!-- to-body-struct -->
2625                     
2626                 </section> <!-- to-parser -->
2627
2628                 <section id="from-parser"> 
2629                     <title>From HF Body Parser</title>
2630                     <para>
2631                         This parser is only a wrapper to the To header field parser. Since bodies of both header fields
2632                         are identical, From parser only calls To parser.
2633                     </para>
2634                     <para>
2635                         The wrapper can be found in file <filename moreinfo="none">parse_from.c</filename> under
2636                         <filename moreinfo="none">parser</filename> subdirectory. There is only one function called
2637                         <function moreinfo="none">parse_from_header</function>. The function accepts one parameter which
2638                         is pointer to structure representing the From header field to be parsed. The function creates an
2639                         instance of <structname>to_body</structname> structure and initializes it. It then calls
2640                         <function moreinfo="none">parse_to</function> function and if everything went OK, the pointer to
2641                         the newly created structure will be put in <structfield>parsed</structfield> field of the structure
2642                         representing the parsed header field.
2643                     </para>
2644                     <para>
2645                         The newly created structure will be freed when the whole message is being destroyed. 
2646                         (See To header field parser description for more details).
2647                     </para>
2648                     <para>
2649                         From parser <emphasis>must be called explicitly</emphasis> !
2650                     </para>
2651                     <para>
2652                         If the main parser finds a From header field, it will not parse the header field body
2653                         automatically. It is up to you to call the 
2654                         <function moreinfo="none">parse_from_header</function> when you want to parse a From
2655                         header field body.
2656                     </para>
2657                 </section> <!-- from-parser -->
2658
2659                 <section id="cseq-parser">
2660                     <title>CSeq HF Body Parser</title>
2661                     <para>
2662                         Purpose of this parser is to parse body of CSeq header field. The parser can be found in file
2663                         <filename moreinfo="none">parse_cseq.c</filename> under <filename moreinfo="none">parser</filename>
2664                         subdirectory.
2665                     </para>
2666                     <para>
2667                         Main function is <function moreinfo="none">parse_cseq</function> but there is no need to call the
2668                         function explicitly. Every time the parser finds a CSeq header field, this function will be called
2669                         automatically. Result of the parser is <structname>cseq_body</structname> structure. Pointer to the
2670                         structure will be stored in <structfield>parsed</structfield> field of 
2671                         <structname>hdr_field</structname> structure. Since the pointer is <type>void*</type>, there is a
2672                         convenience macro <function moreinfo="none">get_cseq</function> in file 
2673                         <filename moreinfo="none">parse_cseq.h</filename> that will do the necessary type-casting and will
2674                         return pointer to <structname>cseq_body</structname> structure.
2675                     </para>
2676                     <para>
2677                         The parser will parse&nbs