Annotation of embedaddon/lighttpd/doc/outdated/cml.txt, revision 1.1.1.2

1.1       misho       1: =========================
                      2: CML (Cache Meta Language)
                      3: =========================
                      4: 
                      5: ---------------
                      6: Module: mod_cml
                      7: ---------------
                      8: 
                      9: :Author: Jan Kneschke
                     10: :Date: $Date: 2004/11/03 22:26:05 $
                     11: :Revision: $Revision: 1.2 $
                     12: 
                     13: :abstract:
                     14:   CML is a Meta language to describe the dependencies of a page at one side and building a page from its fragments on the other side using LUA.
                     15: 
                     16: .. meta::
                     17:   :keywords: lighttpd, cml, lua
                     18: 
                     19: .. contents:: Table of Contents
                     20: 
                     21: Description
                     22: ===========
                     23: 
                     24: CML (Cache Meta Language) wants to solves several problems:
                     25: 
                     26:  * dynamic content needs caching to perform
                     27:  * checking if the content is dirty inside of the application is usually more expensive than sending out the cached data
                     28:  * a dynamic page is usually fragmented and the fragments have different livetimes
                     29:  * the different fragements can be cached independently
                     30: 
                     31: Cache Decision
                     32: --------------
                     33: 
                     34: A simple example should show how to a content caching the very simple way in PHP.
                     35: 
                     36: jan.kneschke.de has a very simple design:
                     37: 
                     38:  * the layout is taken from a template in templates/jk.tmpl
                     39:  * the menu is generated from a menu.csv file
                     40:  * the content is coming from files on the local directory named content-1, content-2 and so on
                     41: 
                     42: The page content is static as long non of the those tree items changes. A change in the layout
                     43: is affecting all pages, a change of menu.csv too, a change of content-x file only affects the
                     44: cached page itself.
                     45: 
                     46: If we model this in PHP we get: ::
                     47: 
                     48:   <?php
                     49: 
                     50:   ## ... fetch all content-* files into $content
                     51:   $cachefile = "/cache/dir/to/cached-content";
                     52: 
                     53:   function is_cachable($content, $cachefile) {
                     54:     if (!file_exists($cachefile)) {
                     55:       return 0;
                     56:     } else {
                     57:       $cachemtime = filemtime($cachefile);
                     58:     }
                     59: 
                     60:     foreach($content as $k => $v) {
                     61:       if (isset($v["file"]) &&
                     62:           filemtime($v["file"]) > $cachemtime) {
                     63:         return 0;
                     64:       }
                     65:     }
                     66: 
                     67:     if (filemtime("/menu/menu.csv") > $cachemtime) {
                     68:       return 0;
                     69:     }
                     70:     if (filemtime("/templates/jk.tmpl") > $cachemtime) {
                     71:       return 0;
                     72:     }
                     73:   }
                     74: 
                     75:   if (is_cachable(...), $cachefile) {
                     76:     readfile($cachefile);
                     77:     exit();
                     78:   } else {
                     79:     # generate content and write it to $cachefile
                     80:   }
                     81:   ?>
                     82: 
                     83: Quite simple. No magic involved. If the one of the files is new than the cached
                     84: content, the content is dirty and has to be regenerated.
                     85: 
                     86: Now let take a look at the numbers:
                     87: 
                     88:  * 150 req/s for a Cache-Hit
                     89:  * 100 req/s for a Cache-Miss
                     90: 
                     91: As you can see the increase is not as good as it could be. The main reason as the overhead
                     92: of the PHP interpreter to start up (a byte-code cache has been used here).
                     93: 
                     94: Moving these decisions out of the PHP script into a server module will remove the need
                     95: to start PHP for a cache-hit.
                     96: 
                     97: To transform this example into a CML you need 'index.cml' in the list of indexfiles
                     98: and the following index.cml file: ::
                     99: 
                    100:   output_contenttype = "text/html"
                    101: 
                    102:   b = request["DOCUMENT_ROOT"]
                    103:   cwd = request["CWD"]
                    104: 
                    105:   output_include = { b .. "_cache.html" }
                    106: 
                    107:   trigger_handler = "index.php"
                    108: 
                    109:   if file_mtime(b .. "../lib/php/menu.csv") > file_mtime(cwd .. "_cache.html") or
                    110:      file_mtime(b .. "templates/jk.tmpl")   > file_mtime(cwd .. "_cache.html") or
                    111:      file_mtime(b .. "content.html")        > file_mtime(cwd .. "_cache.html") then
                    112:      return CACHE_MISS
                    113:   else
                    114:      return CACHE_HIT
                    115:   end
                    116: 
                    117: Numbers again:
                    118: 
                    119:  * 4900 req/s for Cache-Hit
                    120:  *  100 req/s for Cache-Miss
                    121: 
                    122: Content Assembling
                    123: ------------------
                    124: 
                    125: Sometimes the different fragment are already generated externally. You have to cat them together: ::
                    126: 
                    127:   <?php
                    128:   readfile("head.html");
                    129:   readfile("menu.html");
                    130:   readfile("spacer.html");
                    131:   readfile("db-content.html");
                    132:   readfile("spacer2.html");
                    133:   readfile("news.html");
                    134:   readfile("footer.html");
                    135:   ?>
                    136: 
                    137: We we can do the same several times faster directly in the webserver.
                    138: 
                    139: Don't forget: Webserver are built to send out static content, that is what they can do best.
                    140: 
                    141: The index.cml for this looks like: ::
                    142: 
                    143:   output_contenttype = "text/html"
                    144: 
                    145:   cwd = request["CWD"]
                    146: 
                    147:   output_include = { cwd .. "head.html",
                    148:                      cwd .. "menu.html",
                    149:                      cwd .. "spacer.html",
                    150:                      cwd .. "db-content.html",
                    151:                      cwd .. "spacer2.html",
                    152:                      cwd .. "news.html",
                    153:                      cwd .. "footer.html" }
                    154: 
                    155:   return CACHE_HIT
                    156: 
                    157: Now we get about 10000 req/s instead of 600 req/s.
                    158: 
                    159: Power Magnet
                    160: ------------
                    161: 
                    162: Next to all the features about Cache Decisions CML can do more. Starting
                    163: with lighttpd 1.4.9 a power-magnet was added which attracts each request
                    164: and allows you to manipulate the request for your needs.
                    165: 
                    166: We want to display a maintainance page by putting a file in a specified
                    167: place:
                    168: 
                    169: We enable the power magnet: ::
                    170: 
                    171:   cml.power-magnet  = "/home/www/power-magnet.cml"
                    172: 
                    173: and create /home/www/power-magnet.cml with: ::
                    174: 
                    175:   dr = request["DOCUMENT_ROOT"]
                    176: 
                    177:   if file_isreg(dr .. 'maintainance.html') then
                    178:     output_include = { 'maintainance.html' }
                    179:     return CACHE_HIT
                    180:   end
                    181: 
                    182:   return CACHE_MISS
                    183: 
                    184: For each requested file the /home/www/power-magnet.cml is executed which
                    185: checks if maintainance.html exists in the docroot and displays it
                    186: instead of handling the usual request.
                    187: 
                    188: Another example, create thumbnail for requested image and serve it instead
                    189: of sending the big image: ::
                    190: 
                    191:   ## image-url is /album/baltic_winter_2005.jpg
                    192:   ## no params -> 640x480 is served
                    193:   ## /album/baltic_winter_2005.jpg/orig for full size
                    194:   ## /album/baltic_winter_2005.jpg/thumb for thumbnail
                    195: 
                    196:   dr = request["DOCUMENT_ROOT"]
                    197:   sn = request["SCRIPT_NAME"]
                    198: 
                    199:   ## to be continued :) ...
                    200: 
                    201:   trigger_handler = '/gen_image.php'
                    202: 
                    203:   return CACHE_MISS
                    204: 
                    205: 
                    206: Installation
                    207: ============
                    208: 
1.1.1.2 ! misho     209: You need `lua <http://www.lua.org/>`_ and should install `memcached <http://www.memcached.org>`_ and have to configure lighttpd with: ::
1.1       misho     210: 
1.1.1.2 ! misho     211:   ./configure ... --with-lua --with-memcached
1.1       misho     212: 
                    213: To use the plugin you have to load it: ::
                    214: 
                    215:    server.modules = ( ..., "mod_cml", ... )
                    216: 
                    217: Options
                    218: =======
                    219: 
                    220: :cml.extension:
                    221:   the file extension that is bound to the cml-module
                    222: :cml.memcache-hosts:
                    223:   hosts for the memcache.* functions
                    224: :cml.memcache-namespace:
                    225:   (not used yet)
                    226: :cml.power-magnet:
                    227:   a cml file that is executed for each request
                    228: 
                    229: Language
                    230: ========
                    231: 
                    232: The language used for CML is provided by `LUA <http://www.lua.org/>`_.
                    233: 
                    234: Additionally to the functions provided by lua mod_cml provides: ::
                    235: 
                    236:   tables:
                    237: 
                    238:   request
                    239:     - REQUEST_URI
                    240:     - SCRIPT_NAME
                    241:     - SCRIPT_FILENAME
                    242:     - DOCUMENT_ROOT
                    243:     - PATH_INFO
                    244:     - CWD
                    245:     - BASEURI
                    246: 
                    247:   get
                    248:     - parameters from the query-string
                    249: 
                    250:   functions:
                    251:   string md5(string)
                    252:   number file_mtime(string)
                    253:   string memcache_get_string(string)
                    254:   number memcache_get_long(string)
                    255:   boolean memcache_exists(string)
                    256: 
                    257: 
                    258: What ever your script does, it has to return either CACHE_HIT or CACHE_MISS.
                    259: It case a error occures check the error-log, the user will get a error 500. If you don't like
                    260: the standard error-page use ``server.errorfile-prefix``.
                    261: 

FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>