Annotation of embedaddon/lighttpd/doc/outdated/fastcgi.txt, revision 1.1.1.1

1.1       misho       1: =====================
                      2: the FastCGI Interface
                      3: =====================
                      4: 
                      5: -------------------
                      6: Module: mod_fastcgi
                      7: -------------------
                      8: 
                      9: :Author: Jan Kneschke
                     10: :Date: $Date: 2004/11/03 22:26:05 $
                     11: :Revision: $Revision: 1.3 $
                     12: 
                     13: :abstract:
                     14:   The FastCGI interface is the fastest and most secure way
                     15:   to interface external process-handlers like Perl, PHP and
                     16:   your self-written applications.
                     17: 
                     18: .. meta::
                     19:   :keywords: lighttpd, FastCGI
                     20: 
                     21: .. contents:: Table of Contents
                     22: 
                     23: Description
                     24: ===========
                     25: 
                     26: lighttpd provides an interface to a external programs that
                     27: support the FastCGI interface. The FastCGI Interface is
                     28: defined by http://www.fastcgi.com/ and is a
                     29: platform-independent and server independent interface between
                     30: a web-application and a webserver.
                     31: 
                     32: This means that FastCGI programs that run with the Apache
                     33: Webserver will run seamlessly with lighttpd and vice versa.
                     34: 
                     35: 
                     36: FastCGI
                     37: -------
                     38: 
                     39: FastCGI is removes a lot of the limitations of CGI programs.
                     40: CGI programs have the problem that they have to be restarted
                     41: by the webserver for every request which leads to really bad
                     42: performance values.
                     43: 
                     44: FastCGI removes this limitation by keeping the process running
                     45: and handling the requests by this always running process. This
                     46: removes the time used for the fork() and the overall startup
                     47: and cleanup time which is necessary to create and destroy a
                     48: process.
                     49: 
                     50: While CGI programs communicate to the server over pipes,
                     51: FastCGI processes use Unix-Domain-Sockets or TCP/IP to talk
                     52: with the webserver. This gives you the second advantage over
                     53: simple CGI programs: FastCGI don't have to run on the Webserver
                     54: itself but everywhere in the network.
                     55: 
                     56: lighttpd takes it a little bit further by providing a internal
                     57: FastCGI load-balancer which can be used to balance the load
                     58: over multiple FastCGI Servers. In contrast to other solutions
                     59: only the FastCGI process has to be on the cluster and not the
                     60: whole webserver. That gives the FastCGI process more resources
                     61: than a e.g. load-balancer+apache+mod_php solution.
                     62: 
                     63: If you compare FastCGI against a apache+mod_php solution you
                     64: should note that FastCGI provides additional security as the
                     65: FastCGI process can be run under different permissions that
                     66: the webserver and can also live a chroot which might be
                     67: different than the one the webserver is running in.
                     68: 
                     69: Options
                     70: =======
                     71: 
                     72: lighttpd provides the FastCGI support via the fastcgi-module
                     73: (mod_fastcgi) which provides 2 options in the config-file:
                     74: 
                     75: fastcgi.debug
                     76:   a value between 0 and 65535 to set the debug-level in the
                     77:   FastCGI module. Currently only 0 and 1 are used. Use 1 to
                     78:   enable some debug output, 0 to disable it.
                     79: 
                     80: fastcgi.map-extensions
                     81:   map multiple extensions to the same fastcgi server
                     82: 
                     83:   Example: ::
                     84: 
                     85:     fastcgi.map-extensions = ( ".php3" => ".php" )
                     86: 
                     87: fastcgi.server
                     88:   tell the module where to send FastCGI requests to. Every
                     89:   file-extension can have it own handler. Load-Balancing is
                     90:   done by specifying multiple handles for the same extension.
                     91: 
                     92:   structure of fastcgi.server section: ::
                     93: 
                     94:     ( <extension> =>
                     95:       (
                     96:         ( "host" => <string> ,
                     97:           "port" => <integer> ,
                     98:           "socket" => <string>,       # either socket
                     99:                                       # or host+port
                    100:           "bin-path" => <string>,     # OPTIONAL
                    101:           "bin-environment" => <array>, # OPTIONAL
                    102:           "bin-copy-environment" => <array>, # OPTIONAL
                    103:           "mode" => <string>,         # OPTIONAL
                    104:           "docroot" => <string> ,     # OPTIONAL if "mode"
                    105:                                       # is not "authorizer"
                    106:           "check-local" => <string>,  # OPTIONAL
                    107:           "max-procs" => <integer>,   # OPTIONAL
                    108:           "broken-scriptfilename" => <boolean>, # OPTIONAL
                    109:           "disable-time" => <integer>, # optional
                    110:           "allow-x-send-file" => <boolean>, # optional
                    111:           "kill-signal" => <integer>, # OPTIONAL
                    112:           "fix-root-scriptname" => <boolean>,
                    113:                                       # OPTIONAL
                    114:         ( "host" => ...
                    115:         )
                    116:       )
                    117:     )
                    118: 
                    119:   :<extension>: is the file-extension or prefix
                    120:                 (if started with "/")
                    121:   :"host":      is hostname/ip of the FastCGI process
                    122:   :"port":      is tcp-port on the "host" used by the FastCGI
                    123:                 process
                    124:   :"bin-path":  path to the local FastCGI binary which should be
                    125:                 started if no local FastCGI is running
                    126:   :"socket":    path to the unix-domain socket
                    127:   :"mode":      is the FastCGI protocol mode.
                    128:                 Default is "responder", also "authorizer"
                    129:                 mode is implemented.
                    130:   :"docroot":   is optional and is the docroot on the remote
                    131:                 host for default "responder" mode. For
                    132:                 "authorizer" mode it is MANDATORY and it points
                    133:                 to docroot for authorized requests. For security
                    134:                 reasons it is recommended to keep this docroot
                    135:                 outside of server.document-root tree.
                    136:   :"check-local": is optional and may be "enable" (default) or
                    137:                 "disable". If enabled the server first check
                    138:                 for a file in local server.document-root tree
                    139:                 and return 404 (Not Found) if no such file.
                    140:                 If disabled, the server forward request to
                    141:                 FastCGI interface without this check.
                    142:   :"broken-scriptfilename": breaks SCRIPT_FILENAME in a wat that
                    143:                 PHP can extract PATH_INFO from it (default: disabled)
                    144:   :"disable-time": time to wait before a disabled backend is checked
                    145:                 again
                    146:   :"allow-x-send-file": controls if X-LIGHTTPD-send-file headers
                    147:                 are allowed
                    148:   :"fix-root-scriptname": fix broken path-info split for "/" extension ("prefix")
                    149: 
                    150:   If bin-path is set:
                    151: 
                    152:   :"max-procs": the upper limit of the processess to start
                    153:   :"bin-environment": put an entry into the environment of
                    154:                 the started process
                    155:   :"bin-copy-environement": clean up the environment and copy
                    156:                 only the specified entries into the fresh
                    157:                 environment of the spawn process
                    158:   :"kill-signal": signal to terminate the FastCGI process with,
                    159:                 defauls to SIGTERM
                    160: 
                    161: Examples
                    162: --------
                    163: 
                    164:   Multiple extensions for the same host ::
                    165: 
                    166:     fastcgi.server = ( ".php" =>
                    167:       (( "host" => "127.0.0.1",
                    168:          "port" => 1026,
                    169:          "bin-path" => "/usr/local/bin/php"
                    170:       )),
                    171:       ".php4" =>
                    172:       (( "host" => "127.0.0.1",
                    173:          "port" => 1026
                    174:       ))
                    175:     )
                    176: 
                    177:   Example with prefix: ::
                    178: 
                    179:     fastcgi.server = ( "/remote_scripts/" =>
                    180:       (( "host" => "192.168.0.3",
                    181:          "port" => 9000,
                    182:          "check-local" => "disable",
                    183:          "docroot" => "/" # remote server may use
                    184:                           # it's own docroot
                    185:       ))
                    186:     )
                    187: 
                    188:   The request `http://my.host.com/remote_scripts/test.cgi` will
                    189:   be forwarded to fastcgi server at 192.168.0.3 and the value
                    190:   "/remote_scripts/test.cgi" will be used for the SCRIPT_NAME
                    191:   variable. Remote server may prepend it with its own
                    192:   document root. The handling of index files is also the
                    193:   resposibility of remote server for this case.
                    194: 
                    195:   In the case that the prefix is not terminated with a slash
                    196:   the prefix will be handled as file and /test.cgi would become
                    197:   a PATH_INFO instead of part of SCRIPT_NAME.
                    198: 
                    199: 
                    200:   Example for "authorizer" mode: ::
                    201: 
                    202:     fastcgi.server = ( "/remote_scripts/" =>
                    203:       (( "host" => "10.0.0.2",
                    204:          "port" => 9000,
                    205:          "docroot" => "/path_to_private_docs",
                    206:          "mode" => "authorizer"
                    207:       ))
                    208:     )
                    209: 
                    210:   Note that if "docroot" is specified then its value will be
                    211:   used in DOCUMENT_ROOT and SCRIPT_FILENAME variables passed
                    212:   to FastCGI server.
                    213: 
                    214: Load-Balancing
                    215: ==============
                    216: 
                    217: The FastCGI plugin provides automaticly a load-balancing between
                    218: multiple FastCGI servers. ::
                    219: 
                    220:   fastcgi.server = ( ".php" =>
                    221:      (( "host" => "10.0.0.2", "port" => 1030 ),
                    222:       ( "host" => "10.0.0.3", "port" => 1030 ))
                    223:     )
                    224: 
                    225: 
                    226: To understand how the load-balancing works you can enable the
                    227: fastcgi.debug option and will get a similar output as here: ::
                    228: 
                    229:   proc: 127.0.0.1 1031  1 1 1 31454
                    230:   proc: 127.0.0.1 1028  1 1 1 31442
                    231:   proc: 127.0.0.1 1030  1 1 1 31449
                    232:   proc: 127.0.0.1 1029  1 1 2 31447
                    233:   proc: 127.0.0.1 1026  1 1 2 31438
                    234:   got proc: 34 31454
                    235:   release proc: 40 31438
                    236:   proc: 127.0.0.1 1026  1 1 1 31438
                    237:   proc: 127.0.0.1 1028  1 1 1 31442
                    238:   proc: 127.0.0.1 1030  1 1 1 31449
                    239:   proc: 127.0.0.1 1031  1 1 2 31454
                    240:   proc: 127.0.0.1 1029  1 1 2 31447
                    241: 
                    242: Even if this for multiple FastCGI children on the local machine
                    243: the following explaination is valid for remote connections too.
                    244: 
                    245: The output shows:
                    246: 
                    247: - IP, port, unix-socket (is empty here)
                    248: - is-local, state (0 - unset, 1 - running, ... )
                    249: - active connections (load)
                    250: - PID
                    251: 
                    252: As you can see the list is always sorted by the load field.
                    253: 
                    254: Whenever a new connection is requested, the first entry (the one
                    255: with the lowest load) is selected, the load is increased (got proc: ...)
                    256: and the list is sorted again.
                    257: 
                    258: If a FastCGI request is done or the connection is dropped, the load on the
                    259: FastCGI proc decreases and the list is sorted again (release proc: ...)
                    260: 
                    261: This behaviour is very light-weight in code and still very efficient
                    262: as it keeps the fastcgi-servers equally loaded even if they have different
                    263: CPUs.
                    264: 
                    265: Adaptive Process Spawning
                    266: =========================
                    267: 
                    268: .. note:: This feature is disabled in 1.3.14 again. min-procs is
                    269:           ignored in that release
                    270: 
                    271: Starting with 1.3.8 lighttpd can spawn processes on demand if
                    272: a bin-path is specified and the FastCGI process runs locally.
                    273: 
                    274: If you want to have a least one FastCGI process running and
                    275: more of the number of requests increases you can use min-procs
                    276: and max-procs.
                    277: 
                    278: A new process is spawned as soon as the average number of
                    279: requests waiting to be handle by a single process increases the
                    280: max-load-per-proc setting.
                    281: 
                    282: The idle-timeout specifies how long a fastcgi-process should wait
                    283: for a new request before it kills itself.
                    284: 
                    285: Example
                    286: -------
                    287: ::
                    288: 
                    289:   fastcgi.server = ( ".php" =>
                    290:     (( "socket" => "/tmp/php.socket",
                    291:        "bin-path" => "/usr/local/bin/php",
                    292:        "min-procs" => 1,
                    293:        "max-procs" => 32,
                    294:        "max-load-per-proc" => 4,
                    295:        "idle-timeout" => 20
                    296:     ))
                    297:    )
                    298: 
                    299: Disabling Adaptive Spawning
                    300: ---------------------------
                    301: 
                    302: Adaptive Spawning is a quite new feature and it might misbehave
                    303: for your setup. There are several ways to control how the spawing
                    304: is done:
                    305: 
                    306: 1. ``"max-load-per-proc" => 1``
                    307:    if that works for you, great.
                    308: 
                    309: 2. If not set ``min-procs == max-procs``.
                    310: 
                    311: 3. For PHP you can also use: ::
                    312: 
                    313:      $ PHP_FCGI_CHILDREN=384 ./lighttpd -f ./lighttpd.conf
                    314: 
                    315:      fastcgi.server = ( ".php" =>
                    316:         (( "socket" => "/tmp/php.socket",
                    317:            "bin-path" => "/usr/local/bin/php",
                    318:            "min-procs" => 1,
                    319:            "max-procs" => 1,
                    320:            "max-load-per-proc" => 4,
                    321:            "idle-timeout" => 20
                    322:         ))
                    323:       )
                    324: 
                    325:    It will create one socket and let's PHP create the 384 processes itself.
                    326: 
                    327: 4. If you don't want lighttpd to manage the fastcgi processes, remove the
                    328:    bin-path and use spawn-fcgi to spawn them itself.
                    329: 
                    330: 
                    331: FastCGI and Programming Languages
                    332: =================================
                    333: 
                    334: Preparing PHP as a FastCGI program
                    335: ----------------------------------
                    336: 
                    337: One of the most important application that has a FastCGI
                    338: interface is php which can be downloaded from
                    339: http://www.php.net/ . You have to recompile the php from
                    340: source to enable the FastCGI interface as it is normally
                    341: not enabled by default in the distributions.
                    342: 
                    343: If you already have a working installation of PHP on a
                    344: webserver execute a small script which just contains ::
                    345: 
                    346:   <?php phpinfo(); ?>
                    347: 
                    348: and search for the line in that contains the configure call.
                    349: You can use it as the base for the compilation.
                    350: 
                    351: You have to remove all occurences of `--with-apxs`, `--with-apxs2`
                    352: and the like which would build PHP with Apache support.  Add the
                    353: next three switches to compile PHP with FastCGI support::
                    354: 
                    355:   $ ./configure \
                    356:     --enable-fastcgi \
                    357:     --enable-force-cgi-redirect \
                    358:     ...
                    359: 
                    360: After compilation and installation check that your PHP
                    361: binary contains FastCGI support by calling: ::
                    362: 
                    363:   $ php -v
                    364:   PHP 4.3.3RC2-dev (cgi-fcgi) (built: Oct 19 2003 23:19:17)
                    365: 
                    366: The important part is the (cgi-fcgi).
                    367: 
                    368: 
                    369: Starting a FastCGI-PHP
                    370: ----------------------
                    371: 
                    372: Starting with version 1.3.6 lighttpd can spawn the FastCGI
                    373: processes locally itself if necessary: ::
                    374: 
                    375:   fastcgi.server = ( ".php" =>
                    376:     (( "socket" => "/tmp/php-fastcgi.socket",
                    377:        "bin-path" => "/usr/local/bin/php"
                    378:     ))
                    379:   )
                    380: 
                    381: PHP provides 2 special environment variables which control the number of
                    382: spawned workes under the control of a single watching process
                    383: (PHP_FCGI_CHILDREN) and the number of requests what a single worker
                    384: handles before it kills itself. ::
                    385: 
                    386:   fastcgi.server = ( ".php" =>
                    387:      (( "socket" => "/tmp/php-fastcgi.socket",
                    388:         "bin-path" => "/usr/local/bin/php",
                    389:         "bin-environment" => (
                    390:           "PHP_FCGI_CHILDREN" => "16",
                    391:           "PHP_FCGI_MAX_REQUESTS" => "10000"
                    392:         )
                    393:      ))
                    394:    )
                    395: 
                    396: To increase the security of the started process you should only pass
                    397: the necessary environment variables to the FastCGI process. ::
                    398: 
                    399:   fastcgi.server = ( ".php" =>
                    400:      (( "socket" => "/tmp/php-fastcgi.socket",
                    401:         "bin-path" => "/usr/local/bin/php",
                    402:         "bin-environment" => (
                    403:            "PHP_FCGI_CHILDREN" => "16",
                    404:            "PHP_FCGI_MAX_REQUESTS" => "10000" ),
                    405:         "bin-copy-environment" => (
                    406:            "PATH", "SHELL", "USER" )
                    407:      ))
                    408:    )
                    409: 
                    410: Configuring PHP
                    411: ---------------
                    412: 
                    413: If you want to use PATH_INFO and PHP_SELF in you PHP scripts you have to
                    414: configure php and lighttpd. The php.ini needs the option: ::
                    415: 
                    416:   cgi.fix_pathinfo = 1
                    417: 
                    418: and the option ``broken-scriptfilename`` in your fastcgi.server config: ::
                    419: 
                    420:   fastcgi.server = ( ".php" =>
                    421:      (( "socket" => "/tmp/php-fastcgi.socket",
                    422:         "bin-path" => "/usr/local/bin/php",
                    423:         "bin-environment" => (
                    424:           "PHP_FCGI_CHILDREN" => "16",
                    425:           "PHP_FCGI_MAX_REQUESTS" => "10000" ),
                    426:         "bin-copy-environment" => (
                    427:           "PATH", "SHELL", "USER" ),
                    428:         "broken-scriptfilename" => "enable"
                    429:      ))
                    430:    )
                    431: 
                    432: Why this ? the ``cgi.fix_pathinfo = 0`` would give you a working ``PATH_INFO``
                    433: but no ``PHP_SELF``. If you enable it, it turns around. To fix the
                    434: ``PATH_INFO`` `--enable-discard-path` needs a SCRIPT_FILENAME which is against the CGI spec, a
                    435: broken-scriptfilename. With ``cgi.fix_pathinfo = 1`` in php.ini and
                    436: ``broken-scriptfilename => "enable"`` you get both.
                    437: 
                    438: 
                    439: External Spawning
                    440: -----------------
                    441: 
                    442: Spawning FastCGI processes directly in the webserver has some
                    443: disadvantages like
                    444: 
                    445: - FastCGI process can only run locally
                    446: - has the same permissions as the webserver
                    447: - has the same base-dir as the webserver
                    448: 
                    449: As soon as you are using a seperate FastCGI Server to
                    450: take off some load from the webserver you have to control
                    451: the FastCGI process by a external program like spawn-fcgi.
                    452: 
                    453: spawn-fcgi is used to start a FastCGI process in its own
                    454: environment and set the user-id, group-id and change to
                    455: another root-directory (chroot).
                    456: 
                    457: For convenience a wrapper script should be used which takes
                    458: care of all the necessary option. Such a script in included
                    459: in the lighttpd distribution and is call spawn-php.sh.
                    460: 
                    461: The script has a set of config variables you should take
                    462: a look at: ::
                    463: 
                    464:   ## ABSOLUTE path to the spawn-fcgi binary
                    465:   SPAWNFCGI="/usr/local/sbin/spawn-fcgi"
                    466: 
                    467:   ## ABSOLUTE path to the PHP binary
                    468:   FCGIPROGRAM="/usr/local/bin/php"
                    469: 
                    470:   ## bind to tcp-port on localhost
                    471:   FCGIPORT="1026"
                    472: 
                    473:   ## bind to unix domain socket
                    474:   # FCGISOCKET="/tmp/php.sock"
                    475: 
                    476:   ## number of PHP childs to spawn
                    477:   PHP_FCGI_CHILDREN=10
                    478: 
                    479:   ## number of request server by a single php-process until
                    480:   ## is will be restarted
                    481:   PHP_FCGI_MAX_REQUESTS=1000
                    482: 
                    483:   ## IP adresses where PHP should access server connections
                    484:   ## from
                    485:   FCGI_WEB_SERVER_ADDRS="127.0.0.1,192.168.0.1"
                    486: 
                    487:   # allowed environment variables sperated by spaces
                    488:   ALLOWED_ENV="ORACLE_HOME PATH USER"
                    489: 
                    490:   ## if this script is run as root switch to the following user
                    491:   USERID=wwwrun
                    492:   GROUPID=wwwrun
                    493: 
                    494: If you have set the variables to values that fit to your
                    495: setup you can start it by calling: ::
                    496: 
                    497:   $ spawn-php.sh
                    498:   spawn-fcgi.c.136: child spawned successfully: PID: 6925
                    499: 
                    500: If you get "child spawned successfully: PID:" the php
                    501: processes could be started successfully. You should see them
                    502: in your processlist: ::
                    503: 
                    504:   $ ps ax | grep php
                    505:   6925 ?        S      0:00 /usr/local/bin/php
                    506:   6928 ?        S      0:00 /usr/local/bin/php
                    507:   ...
                    508: 
                    509: The number of processes should be PHP_FCGI_CHILDREN + 1.
                    510: Here the process 6925 is the master of the slaves which
                    511: handle the work in parallel. Number of parallel workers can
                    512: be set by PHP_FCGI_CHILDREN. A worker dies automaticly of
                    513: handling PHP_FCGI_MAX_REQUESTS requests as PHP might have
                    514: memory leaks.
                    515: 
                    516: If you start the script as user root php processes will be
                    517: running as the user USERID and group GROUPID to drop the
                    518: root permissions. Otherwise the php processes will run as
                    519: the user you started script as.
                    520: 
                    521: As the script might be started from a unknown stage or even
                    522: directly from the command-line it cleans the environment
                    523: before starting the processes. ALLOWED_ENV contains all
                    524: the external environement variables that should be available
                    525: to the php-process.
                    526: 
                    527: 
                    528: Perl
                    529: ----
                    530: 
                    531: For Perl you have to install the FCGI module from CPAN.
                    532: 
                    533: Skeleton for remote authorizer
                    534: ==============================
                    535: 
                    536: The basic functionality of authorizer is as follows (see
                    537: http://www.fastcgi.com/devkit/doc/fcgi-spec.html, 6.3 for
                    538: details). ::
                    539: 
                    540:   #include <fcgi_stdio.h>
                    541:   #include <stdlib.h>
                    542:   #include <unistd.h>
                    543:   int main () {
                    544:     char* p;
                    545: 
                    546:     while (FCGI_Accept() >= 0) {
                    547:       /* wait for fastcgi authorizer request */
                    548: 
                    549:       printf("Content-type: text/html\r\n");
                    550: 
                    551:       if ((p = getenv("QUERY_STRING")) == NULL) ||
                    552:            <QUERY_STRING is unauthorized>)
                    553:            printf("Status: 403 Forbidden\r\n\r\n");
                    554: 
                    555:       else printf("\r\n");
                    556:         /* default Status is 200 - allow access */
                    557:     }
                    558: 
                    559:     return 0;
                    560:   }
                    561: 
                    562: It is possible to use any other variables provided by
                    563: FastCGI interface for authorization check. Here is only an
                    564: example.
                    565: 
                    566: 
                    567: Troubleshooting
                    568: ===============
                    569: 
                    570: fastcgi.debug should be enabled for troubleshooting.
                    571: 
                    572: If you get: ::
                    573: 
                    574:   (fcgi.c.274) connect delayed:  8
                    575:   (fcgi.c.289) connect succeeded:  8
                    576:   (fcgi.c.745) unexpected end-of-file (perhaps the fastcgi
                    577:      process died):  8
                    578: 
                    579: the fastcgi process accepted the connection but closed it
                    580: right away. This happens if FCGI_WEB_SERVER_ADDRS doesn't
                    581: include the host where you are connection from.
                    582: 
                    583: If you get ::
                    584: 
                    585:   (fcgi.c.274) connect delayed:  7
                    586:   (fcgi.c.1107) error: unexpected close of fastcgi connection
                    587:      for /peterp/seite1.php (no fastcgi process on host/port ?)
                    588:   (fcgi.c.1015) emergency exit: fastcgi: connection-fd: 5
                    589:      fcgi-fd: 7
                    590: 
                    591: the fastcgi process is not running on the host/port you are
                    592: connection to. Check your configuration.
                    593: 
                    594: If you get ::
                    595: 
                    596:   (fcgi.c.274) connect delayed:  7
                    597:   (fcgi.c.289) connect succeeded:  7
                    598: 
                    599: everything is fine. The connect() call just was delayed a
                    600: little bit and is completly normal.
                    601: 

FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>