Diff for /embedaddon/pcre/doc/html/pcrejit.html between versions 1.1 and 1.1.1.3

version 1.1, 2012/02/21 23:05:52 version 1.1.1.3, 2012/10/09 09:19:18
Line 14  man page, in case the conversion went wrong. Line 14  man page, in case the conversion went wrong.
 <br>  <br>
 <ul>  <ul>
 <li><a name="TOC1" href="#SEC1">PCRE JUST-IN-TIME COMPILER SUPPORT</a>  <li><a name="TOC1" href="#SEC1">PCRE JUST-IN-TIME COMPILER SUPPORT</a>
<li><a name="TOC2" href="#SEC2">AVAILABILITY OF JIT SUPPORT</a><li><a name="TOC2" href="#SEC2">8-BIT and 16-BIT SUPPORT</a>
<li><a name="TOC3" href="#SEC3">SIMPLE USE OF JIT</a><li><a name="TOC3" href="#SEC3">AVAILABILITY OF JIT SUPPORT</a>
<li><a name="TOC4" href="#SEC4">UNSUPPORTED OPTIONS AND PATTERN ITEMS</a><li><a name="TOC4" href="#SEC4">SIMPLE USE OF JIT</a>
<li><a name="TOC5" href="#SEC5">RETURN VALUES FROM JIT EXECUTION</a><li><a name="TOC5" href="#SEC5">UNSUPPORTED OPTIONS AND PATTERN ITEMS</a>
<li><a name="TOC6" href="#SEC6">SAVING AND RESTORING COMPILED PATTERNS</a><li><a name="TOC6" href="#SEC6">RETURN VALUES FROM JIT EXECUTION</a>
<li><a name="TOC7" href="#SEC7">CONTROLLING THE JIT STACK</a><li><a name="TOC7" href="#SEC7">SAVING AND RESTORING COMPILED PATTERNS</a>
<li><a name="TOC8" href="#SEC8">JIT STACK FAQ</a><li><a name="TOC8" href="#SEC8">CONTROLLING THE JIT STACK</a>
<li><a name="TOC9" href="#SEC9">EXAMPLE CODE</a><li><a name="TOC9" href="#SEC9">JIT STACK FAQ</a>
<li><a name="TOC10" href="#SEC10">SEE ALSO</a><li><a name="TOC10" href="#SEC10">EXAMPLE CODE</a>
<li><a name="TOC11" href="#SEC11">AUTHOR</a><li><a name="TOC11" href="#SEC11">SEE ALSO</a>
<li><a name="TOC12" href="#SEC12">REVISION</a><li><a name="TOC12" href="#SEC12">AUTHOR</a>
 <li><a name="TOC13" href="#SEC13">REVISION</a>
 </ul>  </ul>
 <br><a name="SEC1" href="#TOC1">PCRE JUST-IN-TIME COMPILER SUPPORT</a><br>  <br><a name="SEC1" href="#TOC1">PCRE JUST-IN-TIME COMPILER SUPPORT</a><br>
 <P>  <P>
 Just-in-time compiling is a heavyweight optimization that can greatly speed up  Just-in-time compiling is a heavyweight optimization that can greatly speed up
 pattern matching. However, it comes at the cost of extra processing before the  pattern matching. However, it comes at the cost of extra processing before the
 match is performed. Therefore, it is of most benefit when the same pattern is  match is performed. Therefore, it is of most benefit when the same pattern is
going to be matched many times. This does not necessarily mean many calls ofgoing to be matched many times. This does not necessarily mean many calls of a
\fPpcre_exec()\fP; if the pattern is not anchored, matching attempts may takematching function; if the pattern is not anchored, matching attempts may take
place many times at various positions in the subject, even for a single call toplace many times at various positions in the subject, even for a single call.
<b>pcre_exec()</b>. If the subject string is very long, it may still pay to useTherefore, if the subject string is very long, it may still pay to use JIT for
JIT for one-off matches.one-off matches.
 </P>  </P>
 <P>  <P>
JIT support applies only to the traditional matching function,JIT support applies only to the traditional Perl-compatible matching function.
<b>pcre_exec()</b>. It does not apply when <b>pcre_dfa_exec()</b> is being used.It does not apply when the DFA matching function is being used. The code for
The code for this support was written by Zoltan Herczeg.this support was written by Zoltan Herczeg.
 </P>  </P>
<br><a name="SEC2" href="#TOC1">AVAILABILITY OF JIT SUPPORT</a><br><br><a name="SEC2" href="#TOC1">8-BIT and 16-BIT SUPPORT</a><br>
 <P>  <P>
   JIT support is available for both the 8-bit and 16-bit PCRE libraries. To keep
   this documentation simple, only the 8-bit interface is described in what
   follows. If you are using the 16-bit library, substitute the 16-bit functions
   and 16-bit structures (for example, <i>pcre16_jit_stack</i> instead of
   <i>pcre_jit_stack</i>).
   </P>
   <br><a name="SEC3" href="#TOC1">AVAILABILITY OF JIT SUPPORT</a><br>
   <P>
 JIT support is an optional feature of PCRE. The "configure" option --enable-jit  JIT support is an optional feature of PCRE. The "configure" option --enable-jit
 (or equivalent CMake option) must be set when PCRE is built if you want to use  (or equivalent CMake option) must be set when PCRE is built if you want to use
 JIT. The support is limited to the following hardware platforms:  JIT. The support is limited to the following hardware platforms:
Line 51  JIT. The support is limited to the following hardware  Line 60  JIT. The support is limited to the following hardware 
   ARM v5, v7, and Thumb2    ARM v5, v7, and Thumb2
   Intel x86 32-bit and 64-bit    Intel x86 32-bit and 64-bit
   MIPS 32-bit    MIPS 32-bit
  Power PC 32-bit and 64-bit (experimental)  Power PC 32-bit and 64-bit
 </pre>  </pre>
The Power PC support is designated as experimental because it has not beenIf --enable-jit is set on an unsupported platform, compilation fails.
fully tested. If --enable-jit is set on an unsupported platform, compilation 
fails. 
 </P>  </P>
 <P>  <P>
 A program that is linked with PCRE 8.20 or later can tell if JIT support is  A program that is linked with PCRE 8.20 or later can tell if JIT support is
 available by calling <b>pcre_config()</b> with the PCRE_CONFIG_JIT option. The  available by calling <b>pcre_config()</b> with the PCRE_CONFIG_JIT option. The
 result is 1 when JIT is available, and 0 otherwise. However, a simple program  result is 1 when JIT is available, and 0 otherwise. However, a simple program
 does not need to check this in order to use JIT. The API is implemented in a  does not need to check this in order to use JIT. The API is implemented in a
way that falls back to the ordinary PCRE code if JIT is not available.way that falls back to the interpretive code if JIT is not available.
 </P>  </P>
 <P>  <P>
 If your program may sometimes be linked with versions of PCRE that are older  If your program may sometimes be linked with versions of PCRE that are older
Line 70  than 8.20, but you want to use JIT when it is availabl Line 77  than 8.20, but you want to use JIT when it is availabl
 the values of PCRE_MAJOR and PCRE_MINOR, or the existence of a JIT macro such  the values of PCRE_MAJOR and PCRE_MINOR, or the existence of a JIT macro such
 as PCRE_CONFIG_JIT, for compile-time control of your code.  as PCRE_CONFIG_JIT, for compile-time control of your code.
 </P>  </P>
<br><a name="SEC3" href="#TOC1">SIMPLE USE OF JIT</a><br><br><a name="SEC4" href="#TOC1">SIMPLE USE OF JIT</a><br>
 <P>  <P>
 You have to do two things to make use of the JIT support in the simplest way:  You have to do two things to make use of the JIT support in the simplest way:
 <pre>  <pre>
Line 79  You have to do two things to make use of the JIT suppo Line 86  You have to do two things to make use of the JIT suppo
       <b>pcre_exec()</b>.        <b>pcre_exec()</b>.
   
   (2) Use <b>pcre_free_study()</b> to free the <b>pcre_extra</b> block when it is    (2) Use <b>pcre_free_study()</b> to free the <b>pcre_extra</b> block when it is
      no longer needed instead of just freeing it yourself. This      no longer needed, instead of just freeing it yourself. This
       ensures that any JIT data is also freed.        ensures that any JIT data is also freed.
 </pre>  </pre>
 For a program that may be linked with pre-8.20 versions of PCRE, you can insert  For a program that may be linked with pre-8.20 versions of PCRE, you can insert
Line 97  this to free the study data: Line 104  this to free the study data:
       pcre_free(study_ptr);        pcre_free(study_ptr);
   #endif    #endif
 </pre>  </pre>
   PCRE_STUDY_JIT_COMPILE requests the JIT compiler to generate code for complete
   matches. If you want to run partial matches using the PCRE_PARTIAL_HARD or
   PCRE_PARTIAL_SOFT options of <b>pcre_exec()</b>, you should set one or both of
   the following options in addition to, or instead of, PCRE_STUDY_JIT_COMPILE
   when you call <b>pcre_study()</b>:
   <pre>
     PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE
     PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE
   </pre>
   The JIT compiler generates different optimized code for each of the three
   modes (normal, soft partial, hard partial). When <b>pcre_exec()</b> is called,
   the appropriate code is run if it is available. Otherwise, the pattern is
   matched using interpretive code.
   </P>
   <P>
 In some circumstances you may need to call additional functions. These are  In some circumstances you may need to call additional functions. These are
 described in the section entitled  described in the section entitled
 <a href="#stackcontrol">"Controlling the JIT stack"</a>  <a href="#stackcontrol">"Controlling the JIT stack"</a>
 below.  below.
 </P>  </P>
 <P>  <P>
If JIT support is not available, PCRE_STUDY_JIT_COMPILE is ignored, and no JITIf JIT support is not available, PCRE_STUDY_JIT_COMPILE etc. are ignored, and
data is set up. Otherwise, the compiled pattern is passed to the JIT compiler,no JIT data is created. Otherwise, the compiled pattern is passed to the JIT
which turns it into machine code that executes much faster than the normalcompiler, which turns it into machine code that executes much faster than the
interpretive code. When <b>pcre_exec()</b> is passed a <b>pcre_extra</b> blocknormal interpretive code. When <b>pcre_exec()</b> is passed a <b>pcre_extra</b>
containing a pointer to JIT code, it obeys that instead of the normal code. Theblock containing a pointer to JIT code of the appropriate mode (normal or
result is identical, but the code runs much faster.hard/soft partial), it obeys that code instead of running the interpreter. The
 result is identical, but the compiled JIT code runs much faster.
 </P>  </P>
 <P>  <P>
 There are some <b>pcre_exec()</b> options that are not supported for JIT  There are some <b>pcre_exec()</b> options that are not supported for JIT
 execution. There are also some pattern items that JIT cannot handle. Details  execution. There are also some pattern items that JIT cannot handle. Details
 are given below. In both cases, execution automatically falls back to the  are given below. In both cases, execution automatically falls back to the
interpretive code.interpretive code. If you want to know whether JIT was actually used for a
 particular match, you should arrange for a JIT callback function to be set up
 as described in the section entitled
 <a href="#stackcontrol">"Controlling the JIT stack"</a>
 below, even if you do not need to supply a non-default JIT stack. Such a
 callback function is called whenever JIT code is about to be obeyed. If the
 execution options are not right for JIT execution, the callback function is not
 obeyed.
 </P>  </P>
 <P>  <P>
 If the JIT compiler finds an unsupported item, no JIT data is generated. You  If the JIT compiler finds an unsupported item, no JIT data is generated. You
 can find out if JIT execution is available after studying a pattern by calling  can find out if JIT execution is available after studying a pattern by calling
 <b>pcre_fullinfo()</b> with the PCRE_INFO_JIT option. A result of 1 means that  <b>pcre_fullinfo()</b> with the PCRE_INFO_JIT option. A result of 1 means that
 JIT compilation was successful. A result of 0 means that JIT support is not  JIT compilation was successful. A result of 0 means that JIT support is not
available, or the pattern was not studied with PCRE_STUDY_JIT_COMPILE, or theavailable, or the pattern was not studied with PCRE_STUDY_JIT_COMPILE etc., or
JIT compiler was not able to handle the pattern.the JIT compiler was not able to handle the pattern.
 </P>  </P>
 <P>  <P>
 Once a pattern has been studied, with or without JIT, it can be used as many  Once a pattern has been studied, with or without JIT, it can be used as many
 times as you like for matching different subject strings.  times as you like for matching different subject strings.
 </P>  </P>
<br><a name="SEC4" href="#TOC1">UNSUPPORTED OPTIONS AND PATTERN ITEMS</a><br><br><a name="SEC5" href="#TOC1">UNSUPPORTED OPTIONS AND PATTERN ITEMS</a><br>
 <P>  <P>
 The only <b>pcre_exec()</b> options that are supported for JIT execution are  The only <b>pcre_exec()</b> options that are supported for JIT execution are
PCRE_NO_UTF8_CHECK, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, andPCRE_NO_UTF8_CHECK, PCRE_NO_UTF16_CHECK, PCRE_NOTBOL, PCRE_NOTEOL,
PCRE_NOTEMPTY_ATSTART. Note in particular that partial matching is notPCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, PCRE_PARTIAL_HARD, and PCRE_PARTIAL_SOFT.
supported. 
 </P>  </P>
 <P>  <P>
 The unsupported pattern items are:  The unsupported pattern items are:
 <pre>  <pre>
   \C             match a single byte; not supported in UTF-8 mode    \C             match a single byte; not supported in UTF-8 mode
   (?Cn)          callouts    (?Cn)          callouts
  (*COMMIT)      )  (*PRUNE)       )
  (*MARK)        )  (*SKIP)        ) backtracking control verbs
  (*PRUNE)       ) the backtracking control verbs 
  (*SKIP)        ) 
   (*THEN)        )    (*THEN)        )
 </pre>  </pre>
 Support for some of these may be added in future.  Support for some of these may be added in future.
 </P>  </P>
<br><a name="SEC5" href="#TOC1">RETURN VALUES FROM JIT EXECUTION</a><br><br><a name="SEC6" href="#TOC1">RETURN VALUES FROM JIT EXECUTION</a><br>
 <P>  <P>
 When a pattern is matched using JIT execution, the return values are the same  When a pattern is matched using JIT execution, the return values are the same
 as those given by the interpretive <b>pcre_exec()</b> code, with the addition of  as those given by the interpretive <b>pcre_exec()</b> code, with the addition of
Line 166  when JIT is not used, but the details of exactly what  Line 193  when JIT is not used, but the details of exactly what 
 same. The PCRE_ERROR_RECURSIONLIMIT error code is never returned by JIT  same. The PCRE_ERROR_RECURSIONLIMIT error code is never returned by JIT
 execution.  execution.
 </P>  </P>
<br><a name="SEC6" href="#TOC1">SAVING AND RESTORING COMPILED PATTERNS</a><br><br><a name="SEC7" href="#TOC1">SAVING AND RESTORING COMPILED PATTERNS</a><br>
 <P>  <P>
 The code that is generated by the JIT compiler is architecture-specific, and is  The code that is generated by the JIT compiler is architecture-specific, and is
 also position dependent. For those reasons it cannot be saved (in a file or  also position dependent. For those reasons it cannot be saved (in a file or
Line 179  restored pattern, and thereby recreate the JIT data, b Line 206  restored pattern, and thereby recreate the JIT data, b
 compilation uses significant resources, it is probably not worth doing this;  compilation uses significant resources, it is probably not worth doing this;
 you might as well recompile the original pattern.  you might as well recompile the original pattern.
 <a name="stackcontrol"></a></P>  <a name="stackcontrol"></a></P>
<br><a name="SEC7" href="#TOC1">CONTROLLING THE JIT STACK</a><br><br><a name="SEC8" href="#TOC1">CONTROLLING THE JIT STACK</a><br>
 <P>  <P>
 When the compiled JIT code runs, it needs a block of memory to use as a stack.  When the compiled JIT code runs, it needs a block of memory to use as a stack.
 By default, it uses 32K on the machine stack. However, some large or  By default, it uses 32K on the machine stack. However, some large or
Line 212  should use. Its arguments are as follows: Line 239  should use. Its arguments are as follows:
   void               *data    void               *data
 </pre>  </pre>
 The <i>extra</i> argument must be the result of studying a pattern with  The <i>extra</i> argument must be the result of studying a pattern with
PCRE_STUDY_JIT_COMPILE. There are three cases for the values of the other twoPCRE_STUDY_JIT_COMPILE etc. There are three cases for the values of the other
options:two options:
 <pre>  <pre>
   (1) If <i>callback</i> is NULL and <i>data</i> is NULL, an internal 32K block    (1) If <i>callback</i> is NULL and <i>data</i> is NULL, an internal 32K block
       on the machine stack is used.        on the machine stack is used.
Line 221  options: Line 248  options:
   (2) If <i>callback</i> is NULL and <i>data</i> is not NULL, <i>data</i> must be    (2) If <i>callback</i> is NULL and <i>data</i> is not NULL, <i>data</i> must be
       a valid JIT stack, the result of calling <b>pcre_jit_stack_alloc()</b>.        a valid JIT stack, the result of calling <b>pcre_jit_stack_alloc()</b>.
   
  (3) If <i>callback</i> not NULL, it must point to a function that is called  (3) If <i>callback</i> is not NULL, it must point to a function that is
      with <i>data</i> as an argument at the start of matching, in order to      called with <i>data</i> as an argument at the start of matching, in
      set up a JIT stack. If the result is NULL, the internal 32K stack      order to set up a JIT stack. If the return from the callback
      is used; otherwise the return value must be a valid JIT stack,      function is NULL, the internal 32K stack is used; otherwise the
      the result of calling <b>pcre_jit_stack_alloc()</b>.      return value must be a valid JIT stack, the result of calling
       <b>pcre_jit_stack_alloc()</b>.
 </pre>  </pre>
You may safely assign the same JIT stack to more than one pattern, as long asA callback function is obeyed whenever JIT code is about to be run; it is not
they are all matched sequentially in the same thread. In a multithreadobeyed when <b>pcre_exec()</b> is called with options that are incompatible for
application, each thread must use its own JIT stack.JIT execution. A callback function can therefore be used to determine whether a
 match operation was executed by JIT or by the interpreter.
 </P>  </P>
 <P>  <P>
Strictly speaking, even more is allowed. You can assign the same stack to anyYou may safely use the same JIT stack for more than one pattern (either by
number of patterns as long as they are not used for matching by multipleassigning directly or by callback), as long as the patterns are all matched
 sequentially in the same thread. In a multithread application, if you do not
 specify a JIT stack, or if you assign or pass back NULL from a callback, that
 is thread-safe, because each thread has its own machine stack. However, if you
 assign or pass back a non-NULL JIT stack, this must be a different stack for
 each thread so that the application is thread-safe.
 </P>
 <P>
 Strictly speaking, even more is allowed. You can assign the same non-NULL stack
 to any number of patterns as long as they are not used for matching by multiple
 threads at the same time. For example, you can assign the same stack to all  threads at the same time. For example, you can assign the same stack to all
 compiled patterns, and use a global mutex in the callback to wait until the  compiled patterns, and use a global mutex in the callback to wait until the
stack is available for use. However, this is an inefficient solution, andstack is available for use. However, this is an inefficient solution, and not
not recommended.recommended.
 </P>  </P>
 <P>  <P>
This is a suggestion for how a typical multithreaded program might operate:This is a suggestion for how a multithreaded program that needs to set up
 non-default JIT stacks might operate:
 <pre>  <pre>
   During thread initalization    During thread initalization
     thread_local_var = pcre_jit_stack_alloc(...)      thread_local_var = pcre_jit_stack_alloc(...)
Line 254  This is a suggestion for how a typical multithreaded p Line 293  This is a suggestion for how a typical multithreaded p
 All the functions described in this section do nothing if JIT is not available,  All the functions described in this section do nothing if JIT is not available,
 and <b>pcre_assign_jit_stack()</b> does nothing unless the <b>extra</b> argument  and <b>pcre_assign_jit_stack()</b> does nothing unless the <b>extra</b> argument
 is non-NULL and points to a <b>pcre_extra</b> block that is the result of a  is non-NULL and points to a <b>pcre_extra</b> block that is the result of a
successful study with PCRE_STUDY_JIT_COMPILE.successful study with PCRE_STUDY_JIT_COMPILE etc.
 <a name="stackfaq"></a></P>  <a name="stackfaq"></a></P>
<br><a name="SEC8" href="#TOC1">JIT STACK FAQ</a><br><br><a name="SEC9" href="#TOC1">JIT STACK FAQ</a><br>
 <P>  <P>
 (1) Why do we need JIT stacks?  (1) Why do we need JIT stacks?
 <br>  <br>
Line 320  pattern causes stack overflow with a stack of 1M? Is t Line 359  pattern causes stack overflow with a stack of 1M? Is t
 stack is freed?  stack is freed?
 <br>  <br>
 <br>  <br>
Especially on embedded sytems, it might be a good idea to releaseEspecially on embedded sytems, it might be a good idea to release memory
memory sometimes without freeing the stack. There is no API for this at thesometimes without freeing the stack. There is no API for this at the moment.
moment. Probably a function call which returns with the currently allocatedProbably a function call which returns with the currently allocated memory for
memory for any stack and another which allows releasing memory (shrinking theany stack and another which allows releasing memory (shrinking the stack) would
stack) would be a good idea if someone needs this.be a good idea if someone needs this.
 </P>  </P>
 <P>  <P>
 (7) This is too much of a headache. Isn't there any better solution for JIT  (7) This is too much of a headache. Isn't there any better solution for JIT
Line 334  stack handling? Line 373  stack handling?
 No, thanks to Windows. If POSIX threads were used everywhere, we could throw  No, thanks to Windows. If POSIX threads were used everywhere, we could throw
 out this complicated API.  out this complicated API.
 </P>  </P>
<br><a name="SEC9" href="#TOC1">EXAMPLE CODE</a><br><br><a name="SEC10" href="#TOC1">EXAMPLE CODE</a><br>
 <P>  <P>
 This is a single-threaded example that specifies a JIT stack without using a  This is a single-threaded example that specifies a JIT stack without using a
 callback.  callback.
Line 359  callback. Line 398  callback.
   
 </PRE>  </PRE>
 </P>  </P>
<br><a name="SEC10" href="#TOC1">SEE ALSO</a><br><br><a name="SEC11" href="#TOC1">SEE ALSO</a><br>
 <P>  <P>
 <b>pcreapi</b>(3)  <b>pcreapi</b>(3)
 </P>  </P>
<br><a name="SEC11" href="#TOC1">AUTHOR</a><br><br><a name="SEC12" href="#TOC1">AUTHOR</a><br>
 <P>  <P>
 Philip Hazel (FAQ by Zoltan Herczeg)  Philip Hazel (FAQ by Zoltan Herczeg)
 <br>  <br>
Line 372  University Computing Service Line 411  University Computing Service
 Cambridge CB2 3QH, England.  Cambridge CB2 3QH, England.
 <br>  <br>
 </P>  </P>
<br><a name="SEC12" href="#TOC1">REVISION</a><br><br><a name="SEC13" href="#TOC1">REVISION</a><br>
 <P>  <P>
Last updated: 26 November 2011Last updated: 04 May 2012
 <br>  <br>
Copyright &copy; 1997-2011 University of Cambridge.Copyright &copy; 1997-2012 University of Cambridge.
 <br>  <br>
 <p>  <p>
 Return to the <a href="index.html">PCRE index page</a>.  Return to the <a href="index.html">PCRE index page</a>.

Removed from v.1.1  
changed lines
  Added in v.1.1.1.3


FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>