version 1.1.1.1, 2012/02/21 23:05:52
|
version 1.1.1.4, 2013/07/22 08:25:57
|
Line 1
|
Line 1
|
.TH PCRESTACK 3 | .TH PCRESTACK 3 "24 June 2012" "PCRE 8.30" |
.SH NAME |
.SH NAME |
PCRE - Perl-compatible regular expressions |
PCRE - Perl-compatible regular expressions |
.SH "PCRE DISCUSSION OF STACK USAGE" |
.SH "PCRE DISCUSSION OF STACK USAGE" |
.rs |
.rs |
.sp |
.sp |
When you call \fBpcre_exec()\fP, it makes use of an internal function called | When you call \fBpcre[16|32]_exec()\fP, it makes use of an internal function |
\fBmatch()\fP. This calls itself recursively at branch points in the pattern, | called \fBmatch()\fP. This calls itself recursively at branch points in the |
in order to remember the state of the match so that it can back up and try a | pattern, in order to remember the state of the match so that it can back up and |
different alternative if the first one fails. As matching proceeds deeper and | try a different alternative if the first one fails. As matching proceeds deeper |
deeper into the tree of possibilities, the recursion depth increases. The | and deeper into the tree of possibilities, the recursion depth increases. The |
\fBmatch()\fP function is also called in other circumstances, for example, |
\fBmatch()\fP function is also called in other circumstances, for example, |
whenever a parenthesized sub-pattern is entered, and in certain cases of |
whenever a parenthesized sub-pattern is entered, and in certain cases of |
repetition. |
repetition. |
Line 19 different numbers of a's. Furthermore, in a number of
|
Line 19 different numbers of a's. Furthermore, in a number of
|
the recursive call would immediately be passed back as the result of the |
the recursive call would immediately be passed back as the result of the |
current call (a "tail recursion"), the function is just restarted instead. |
current call (a "tail recursion"), the function is just restarted instead. |
.P |
.P |
The above comments apply when \fBpcre_exec()\fP is run in its normal | The above comments apply when \fBpcre[16|32]_exec()\fP is run in its normal |
interpretive manner. If the pattern was studied with the |
interpretive manner. If the pattern was studied with the |
PCRE_STUDY_JIT_COMPILE option, and just-in-time compiling was successful, and |
PCRE_STUDY_JIT_COMPILE option, and just-in-time compiling was successful, and |
the options passed to \fBpcre_exec()\fP were not incompatible, the matching | the options passed to \fBpcre[16|32]_exec()\fP were not incompatible, the matching |
process uses the JIT-compiled code instead of the \fBmatch()\fP function. In |
process uses the JIT-compiled code instead of the \fBmatch()\fP function. In |
this case, the memory requirements are handled entirely differently. See the |
this case, the memory requirements are handled entirely differently. See the |
.\" HREF |
.\" HREF |
Line 30 this case, the memory requirements are handled entirel
|
Line 30 this case, the memory requirements are handled entirel
|
.\" |
.\" |
documentation for details. |
documentation for details. |
.P |
.P |
The \fBpcre_dfa_exec()\fP function operates in an entirely different way, and | The \fBpcre[16|32]_dfa_exec()\fP function operates in an entirely different way, |
uses recursion only when there is a regular expression recursion or subroutine | and uses recursion only when there is a regular expression recursion or |
call in the pattern. This includes the processing of assertion and "once-only" | subroutine call in the pattern. This includes the processing of assertion and |
subpatterns, which are handled like subroutine calls. Normally, these are never | "once-only" subpatterns, which are handled like subroutine calls. Normally, |
very deep, and the limit on the complexity of \fBpcre_dfa_exec()\fP is | these are never very deep, and the limit on the complexity of |
controlled by the amount of workspace it is given. However, it is possible to | \fBpcre[16|32]_dfa_exec()\fP is controlled by the amount of workspace it is given. |
write patterns with runaway infinite recursions; such patterns will cause | However, it is possible to write patterns with runaway infinite recursions; |
\fBpcre_dfa_exec()\fP to run out of stack. At present, there is no protection | such patterns will cause \fBpcre[16|32]_dfa_exec()\fP to run out of stack. At |
against this. | present, there is no protection against this. |
.P |
.P |
The comments that follow do NOT apply to \fBpcre_dfa_exec()\fP; they are | The comments that follow do NOT apply to \fBpcre[16|32]_dfa_exec()\fP; they are |
relevant only for \fBpcre_exec()\fP without the JIT optimization. | relevant only for \fBpcre[16|32]_exec()\fP without the JIT optimization. |
. |
. |
. |
. |
.SS "Reducing \fBpcre_exec()\fP's stack usage" | .SS "Reducing \fBpcre[16|32]_exec()\fP's stack usage" |
.rs |
.rs |
.sp |
.sp |
Each time that \fBmatch()\fP is actually called recursively, it uses memory |
Each time that \fBmatch()\fP is actually called recursively, it uses memory |
Line 79 subject strings is to write repeated parenthesized sub
|
Line 79 subject strings is to write repeated parenthesized sub
|
than one character whenever possible. |
than one character whenever possible. |
. |
. |
. |
. |
.SS "Compiling PCRE to use heap instead of stack for \fBpcre_exec()\fP" | .SS "Compiling PCRE to use heap instead of stack for \fBpcre[16|32]_exec()\fP" |
.rs |
.rs |
.sp |
.sp |
In environments where stack memory is constrained, you might want to compile |
In environments where stack memory is constrained, you might want to compile |
PCRE to use heap memory instead of stack for remembering back-up points when |
PCRE to use heap memory instead of stack for remembering back-up points when |
\fBpcre_exec()\fP is running. This makes it run a lot more slowly, however. | \fBpcre[16|32]_exec()\fP is running. This makes it run a lot more slowly, however. |
Details of how to do this are given in the |
Details of how to do this are given in the |
.\" HREF |
.\" HREF |
\fBpcrebuild\fP |
\fBpcrebuild\fP |
.\" |
.\" |
documentation. When built in this way, instead of using the stack, PCRE obtains |
documentation. When built in this way, instead of using the stack, PCRE obtains |
and frees memory by calling the functions that are pointed to by the |
and frees memory by calling the functions that are pointed to by the |
\fBpcre_stack_malloc\fP and \fBpcre_stack_free\fP variables. By default, these | \fBpcre[16|32]_stack_malloc\fP and \fBpcre[16|32]_stack_free\fP variables. By |
point to \fBmalloc()\fP and \fBfree()\fP, but you can replace the pointers to | default, these point to \fBmalloc()\fP and \fBfree()\fP, but you can replace |
cause PCRE to use your own functions. Since the block sizes are always the | the pointers to cause PCRE to use your own functions. Since the block sizes are |
same, and are always freed in reverse order, it may be possible to implement | always the same, and are always freed in reverse order, it may be possible to |
customized memory handlers that are more efficient than the standard functions. | implement customized memory handlers that are more efficient than the standard |
| functions. |
. |
. |
. |
. |
.SS "Limiting \fBpcre_exec()\fP's stack usage" | .SS "Limiting \fBpcre[16|32]_exec()\fP's stack usage" |
.rs |
.rs |
.sp |
.sp |
You can set limits on the number of times that \fBmatch()\fP is called, both in |
You can set limits on the number of times that \fBmatch()\fP is called, both in |
total and recursively. If a limit is exceeded, \fBpcre_exec()\fP returns an | total and recursively. If a limit is exceeded, \fBpcre[16|32]_exec()\fP returns an |
error code. Setting suitable limits should prevent it from running out of |
error code. Setting suitable limits should prevent it from running out of |
stack. The default values of the limits are very large, and unlikely ever to |
stack. The default values of the limits are very large, and unlikely ever to |
operate. They can be changed when PCRE is built, and they can also be set when |
operate. They can be changed when PCRE is built, and they can also be set when |
\fBpcre_exec()\fP is called. For details of these interfaces, see the | \fBpcre[16|32]_exec()\fP is called. For details of these interfaces, see the |
.\" HREF |
.\" HREF |
\fBpcrebuild\fP |
\fBpcrebuild\fP |
.\" |
.\" |
documentation and the |
documentation and the |
.\" HTML <a href="pcreapi.html#extradata"> |
.\" HTML <a href="pcreapi.html#extradata"> |
.\" </a> |
.\" </a> |
section on extra data for \fBpcre_exec()\fP | section on extra data for \fBpcre[16|32]_exec()\fP |
.\" |
.\" |
in the |
in the |
.\" HREF |
.\" HREF |
Line 122 in the
|
Line 123 in the
|
documentation. |
documentation. |
.P |
.P |
As a very rough rule of thumb, you should reckon on about 500 bytes per |
As a very rough rule of thumb, you should reckon on about 500 bytes per |
recursion. Thus, if you want to limit your stack usage to 8Mb, you | recursion. Thus, if you want to limit your stack usage to 8Mb, you should set |
should set the limit at 16000 recursions. A 64Mb stack, on the other hand, can | the limit at 16000 recursions. A 64Mb stack, on the other hand, can support |
support around 128000 recursions. | around 128000 recursions. |
.P |
.P |
In Unix-like environments, the \fBpcretest\fP test program has a command line |
In Unix-like environments, the \fBpcretest\fP test program has a command line |
option (\fB-S\fP) that can be used to increase the size of its stack. As long |
option (\fB-S\fP) that can be used to increase the size of its stack. As long |
as the stack is large enough, another option (\fB-M\fP) can be used to find the |
as the stack is large enough, another option (\fB-M\fP) can be used to find the |
smallest limits that allow a particular pattern to match a given subject |
smallest limits that allow a particular pattern to match a given subject |
string. This is done by calling \fBpcre_exec()\fP repeatedly with different | string. This is done by calling \fBpcre[16|32]_exec()\fP repeatedly with different |
limits. |
limits. |
. |
. |
. |
. |
|
.SS "Obtaining an estimate of stack usage" |
|
.rs |
|
.sp |
|
The actual amount of stack used per recursion can vary quite a lot, depending |
|
on the compiler that was used to build PCRE and the optimization or debugging |
|
options that were set for it. The rule of thumb value of 500 bytes mentioned |
|
above may be larger or smaller than what is actually needed. A better |
|
approximation can be obtained by running this command: |
|
.sp |
|
pcretest -m -C |
|
.sp |
|
The \fB-C\fP option causes \fBpcretest\fP to output information about the |
|
options with which PCRE was compiled. When \fB-m\fP is also given (before |
|
\fB-C\fP), information about stack use is given in a line like this: |
|
.sp |
|
Match recursion uses stack: approximate frame size = 640 bytes |
|
.sp |
|
The value is approximate because some recursions need a bit more (up to perhaps |
|
16 more bytes). |
|
.P |
|
If the above command is given when PCRE is compiled to use the heap instead of |
|
the stack for recursion, the value that is output is the size of each block |
|
that is obtained from the heap. |
|
. |
|
. |
.SS "Changing stack size in Unix-like systems" |
.SS "Changing stack size in Unix-like systems" |
.rs |
.rs |
.sp |
.sp |
Line 155 limit on stack size by code such as this:
|
Line 181 limit on stack size by code such as this:
|
.sp |
.sp |
This reads the current limits (soft and hard) using \fBgetrlimit()\fP, then |
This reads the current limits (soft and hard) using \fBgetrlimit()\fP, then |
attempts to increase the soft limit to 100Mb using \fBsetrlimit()\fP. You must |
attempts to increase the soft limit to 100Mb using \fBsetrlimit()\fP. You must |
do this before calling \fBpcre_exec()\fP. | do this before calling \fBpcre[16|32]_exec()\fP. |
. |
. |
. |
. |
.SS "Changing stack size in Mac OS X" |
.SS "Changing stack size in Mac OS X" |
Line 184 Cambridge CB2 3QH, England.
|
Line 210 Cambridge CB2 3QH, England.
|
.rs |
.rs |
.sp |
.sp |
.nf |
.nf |
Last updated: 26 August 2011 | Last updated: 24 June 2012 |
Copyright (c) 1997-2011 University of Cambridge. | Copyright (c) 1997-2012 University of Cambridge. |
.fi |
.fi |