embedaddon/pcre/doc/pcrejit.3 - diff

Return to pcrejit.3 CVS log

Up to [ELWIX - Embedded LightWeight unIX -] / embedaddon / pcre / doc

Diff for /embedaddon/pcre/doc/pcrejit.3 between versions 1.1.1.1 and 1.1.1.4

version 1.1.1.1, 2012/02/21 23:05:51	version 1.1.1.4, 2013/07/22 08:25:57
Line 1	Line 1
.TH PCREJIT 3	.TH PCREJIT 3 "17 March 2013" "PCRE 8.33"
.SH NAME	.SH NAME
PCRE - Perl-compatible regular expressions	PCRE - Perl-compatible regular expressions
.SH "PCRE JUST-IN-TIME COMPILER SUPPORT"	.SH "PCRE JUST-IN-TIME COMPILER SUPPORT"
Line 7 PCRE - Perl-compatible regular expressions	Line 7 PCRE - Perl-compatible regular expressions
Just-in-time compiling is a heavyweight optimization that can greatly speed up	Just-in-time compiling is a heavyweight optimization that can greatly speed up
pattern matching. However, it comes at the cost of extra processing before the	pattern matching. However, it comes at the cost of extra processing before the
match is performed. Therefore, it is of most benefit when the same pattern is	match is performed. Therefore, it is of most benefit when the same pattern is
going to be matched many times. This does not necessarily mean many calls of	going to be matched many times. This does not necessarily mean many calls of a
\fPpcre_exec()\fP; if the pattern is not anchored, matching attempts may take	matching function; if the pattern is not anchored, matching attempts may take
place many times at various positions in the subject, even for a single call to	place many times at various positions in the subject, even for a single call.
\fBpcre_exec()\fP. If the subject string is very long, it may still pay to use	Therefore, if the subject string is very long, it may still pay to use JIT for
JIT for one-off matches.	one-off matches.
.P	.P
JIT support applies only to the traditional matching function,	JIT support applies only to the traditional Perl-compatible matching function.
\fBpcre_exec()\fP. It does not apply when \fBpcre_dfa_exec()\fP is being used.	It does not apply when the DFA matching function is being used. The code for
The code for this support was written by Zoltan Herczeg.	this support was written by Zoltan Herczeg.
.	.
.	.
	.SH "8-BIT, 16-BIT AND 32-BIT SUPPORT"
	.rs
	.sp
	JIT support is available for all of the 8-bit, 16-bit and 32-bit PCRE
	libraries. To keep this documentation simple, only the 8-bit interface is
	described in what follows. If you are using the 16-bit library, substitute the
	16-bit functions and 16-bit structures (for example, \fIpcre16_jit_stack\fP
	instead of \fIpcre_jit_stack\fP). If you are using the 32-bit library,
	substitute the 32-bit functions and 32-bit structures (for example,
	\fIpcre32_jit_stack\fP instead of \fIpcre_jit_stack\fP).
	.
	.
.SH "AVAILABILITY OF JIT SUPPORT"	.SH "AVAILABILITY OF JIT SUPPORT"
.rs	.rs
.sp	.sp
Line 28 JIT. The support is limited to the following hardware	Line 40 JIT. The support is limited to the following hardware
ARM v5, v7, and Thumb2	ARM v5, v7, and Thumb2
Intel x86 32-bit and 64-bit	Intel x86 32-bit and 64-bit
MIPS 32-bit	MIPS 32-bit
Power PC 32-bit and 64-bit (experimental)	Power PC 32-bit and 64-bit
	SPARC 32-bit (experimental)
.sp	.sp
The Power PC support is designated as experimental because it has not been	If --enable-jit is set on an unsupported platform, compilation fails.
fully tested. If --enable-jit is set on an unsupported platform, compilation
fails.
.P	.P
A program that is linked with PCRE 8.20 or later can tell if JIT support is	A program that is linked with PCRE 8.20 or later can tell if JIT support is
available by calling \fBpcre_config()\fP with the PCRE_CONFIG_JIT option. The	available by calling \fBpcre_config()\fP with the PCRE_CONFIG_JIT option. The
result is 1 when JIT is available, and 0 otherwise. However, a simple program	result is 1 when JIT is available, and 0 otherwise. However, a simple program
does not need to check this in order to use JIT. The API is implemented in a	does not need to check this in order to use JIT. The normal API is implemented
way that falls back to the ordinary PCRE code if JIT is not available.	in a way that falls back to the interpretive code if JIT is not available. For
	programs that need the best possible performance, there is also a "fast path"
	API that is JIT-specific.
.P	.P
If your program may sometimes be linked with versions of PCRE that are older	If your program may sometimes be linked with versions of PCRE that are older
than 8.20, but you want to use JIT when it is available, you can test	than 8.20, but you want to use JIT when it is available, you can test
Line 56 You have to do two things to make use of the JIT suppo	Line 69 You have to do two things to make use of the JIT suppo
\fBpcre_exec()\fP.	\fBpcre_exec()\fP.
.sp	.sp
(2) Use \fBpcre_free_study()\fP to free the \fBpcre_extra\fP block when it is	(2) Use \fBpcre_free_study()\fP to free the \fBpcre_extra\fP block when it is
no longer needed instead of just freeing it yourself. This	no longer needed, instead of just freeing it yourself. This ensures that
ensures that any JIT data is also freed.	any JIT data is also freed.
.sp	.sp
For a program that may be linked with pre-8.20 versions of PCRE, you can insert	For a program that may be linked with pre-8.20 versions of PCRE, you can insert
.sp	.sp
Line 74 this to free the study data:	Line 87 this to free the study data:
pcre_free(study_ptr);	pcre_free(study_ptr);
#endif	#endif
.sp	.sp
	PCRE_STUDY_JIT_COMPILE requests the JIT compiler to generate code for complete
	matches. If you want to run partial matches using the PCRE_PARTIAL_HARD or
	PCRE_PARTIAL_SOFT options of \fBpcre_exec()\fP, you should set one or both of
	the following options in addition to, or instead of, PCRE_STUDY_JIT_COMPILE
	when you call \fBpcre_study()\fP:
	.sp
	PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE
	PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE
	.sp
	The JIT compiler generates different optimized code for each of the three
	modes (normal, soft partial, hard partial). When \fBpcre_exec()\fP is called,
	the appropriate code is run if it is available. Otherwise, the pattern is
	matched using interpretive code.
	.P
In some circumstances you may need to call additional functions. These are	In some circumstances you may need to call additional functions. These are
described in the section entitled	described in the section entitled
.\" HTML <a href="#stackcontrol">	.\" HTML <a href="#stackcontrol">
Line 82 described in the section entitled	Line 109 described in the section entitled
.\"	.\"
below.	below.
.P	.P
If JIT support is not available, PCRE_STUDY_JIT_COMPILE is ignored, and no JIT	If JIT support is not available, PCRE_STUDY_JIT_COMPILE etc. are ignored, and
data is set up. Otherwise, the compiled pattern is passed to the JIT compiler,	no JIT data is created. Otherwise, the compiled pattern is passed to the JIT
which turns it into machine code that executes much faster than the normal	compiler, which turns it into machine code that executes much faster than the
interpretive code. When \fBpcre_exec()\fP is passed a \fBpcre_extra\fP block	normal interpretive code. When \fBpcre_exec()\fP is passed a \fBpcre_extra\fP
containing a pointer to JIT code, it obeys that instead of the normal code. The	block containing a pointer to JIT code of the appropriate mode (normal or
result is identical, but the code runs much faster.	hard/soft partial), it obeys that code instead of running the interpreter. The
	result is identical, but the compiled JIT code runs much faster.
.P	.P
There are some \fBpcre_exec()\fP options that are not supported for JIT	There are some \fBpcre_exec()\fP options that are not supported for JIT
execution. There are also some pattern items that JIT cannot handle. Details	execution. There are also some pattern items that JIT cannot handle. Details
are given below. In both cases, execution automatically falls back to the	are given below. In both cases, execution automatically falls back to the
interpretive code.	interpretive code. If you want to know whether JIT was actually used for a
	particular match, you should arrange for a JIT callback function to be set up
	as described in the section entitled
	.\" HTML <a href="#stackcontrol">
	.\" </a>
	"Controlling the JIT stack"
	.\"
	below, even if you do not need to supply a non-default JIT stack. Such a
	callback function is called whenever JIT code is about to be obeyed. If the
	execution options are not right for JIT execution, the callback function is not
	obeyed.
.P	.P
If the JIT compiler finds an unsupported item, no JIT data is generated. You	If the JIT compiler finds an unsupported item, no JIT data is generated. You
can find out if JIT execution is available after studying a pattern by calling	can find out if JIT execution is available after studying a pattern by calling
\fBpcre_fullinfo()\fP with the PCRE_INFO_JIT option. A result of 1 means that	\fBpcre_fullinfo()\fP with the PCRE_INFO_JIT option. A result of 1 means that
JIT compilation was successful. A result of 0 means that JIT support is not	JIT compilation was successful. A result of 0 means that JIT support is not
available, or the pattern was not studied with PCRE_STUDY_JIT_COMPILE, or the	available, or the pattern was not studied with PCRE_STUDY_JIT_COMPILE etc., or
JIT compiler was not able to handle the pattern.	the JIT compiler was not able to handle the pattern.
.P	.P
Once a pattern has been studied, with or without JIT, it can be used as many	Once a pattern has been studied, with or without JIT, it can be used as many
times as you like for matching different subject strings.	times as you like for matching different subject strings.
Line 109 times as you like for matching different subject strin	Line 147 times as you like for matching different subject strin
.rs	.rs
.sp	.sp
The only \fBpcre_exec()\fP options that are supported for JIT execution are	The only \fBpcre_exec()\fP options that are supported for JIT execution are
PCRE_NO_UTF8_CHECK, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, and	PCRE_NO_UTF8_CHECK, PCRE_NO_UTF16_CHECK, PCRE_NO_UTF32_CHECK, PCRE_NOTBOL,
PCRE_NOTEMPTY_ATSTART. Note in particular that partial matching is not	PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, PCRE_PARTIAL_HARD, and
supported.	PCRE_PARTIAL_SOFT.
.P	.P
The unsupported pattern items are:	The only unsupported pattern items are \eC (match a single data unit) when
.sp	running in a UTF mode, and a callout immediately before an assertion condition
\eC match a single byte; not supported in UTF-8 mode	in a conditional group.
(?Cn) callouts
(*COMMIT) )
(*MARK) )
(*PRUNE) ) the backtracking control verbs
(*SKIP) )
(*THEN) )
.sp
Support for some of these may be added in future.
.	.
.	.
.SH "RETURN VALUES FROM JIT EXECUTION"	.SH "RETURN VALUES FROM JIT EXECUTION"
Line 200 should use. Its arguments are as follows:	Line 230 should use. Its arguments are as follows:
void *data	void *data
.sp	.sp
The \fIextra\fP argument must be the result of studying a pattern with	The \fIextra\fP argument must be the result of studying a pattern with
PCRE_STUDY_JIT_COMPILE. There are three cases for the values of the other two	PCRE_STUDY_JIT_COMPILE etc. There are three cases for the values of the other
options:	two options:
.sp	.sp
(1) If \fIcallback\fP is NULL and \fIdata\fP is NULL, an internal 32K block	(1) If \fIcallback\fP is NULL and \fIdata\fP is NULL, an internal 32K block
on the machine stack is used.	on the machine stack is used.
Line 209 options:	Line 239 options:
(2) If \fIcallback\fP is NULL and \fIdata\fP is not NULL, \fIdata\fP must be	(2) If \fIcallback\fP is NULL and \fIdata\fP is not NULL, \fIdata\fP must be
a valid JIT stack, the result of calling \fBpcre_jit_stack_alloc()\fP.	a valid JIT stack, the result of calling \fBpcre_jit_stack_alloc()\fP.
.sp	.sp
(3) If \fIcallback\fP not NULL, it must point to a function that is called	(3) If \fIcallback\fP is not NULL, it must point to a function that is
with \fIdata\fP as an argument at the start of matching, in order to	called with \fIdata\fP as an argument at the start of matching, in
set up a JIT stack. If the result is NULL, the internal 32K stack	order to set up a JIT stack. If the return from the callback
is used; otherwise the return value must be a valid JIT stack,	function is NULL, the internal 32K stack is used; otherwise the
the result of calling \fBpcre_jit_stack_alloc()\fP.	return value must be a valid JIT stack, the result of calling
	\fBpcre_jit_stack_alloc()\fP.
.sp	.sp
You may safely assign the same JIT stack to more than one pattern, as long as	A callback function is obeyed whenever JIT code is about to be run; it is not
they are all matched sequentially in the same thread. In a multithread	obeyed when \fBpcre_exec()\fP is called with options that are incompatible for
application, each thread must use its own JIT stack.	JIT execution. A callback function can therefore be used to determine whether a
	match operation was executed by JIT or by the interpreter.
.P	.P
Strictly speaking, even more is allowed. You can assign the same stack to any	You may safely use the same JIT stack for more than one pattern (either by
number of patterns as long as they are not used for matching by multiple	assigning directly or by callback), as long as the patterns are all matched
	sequentially in the same thread. In a multithread application, if you do not
	specify a JIT stack, or if you assign or pass back NULL from a callback, that
	is thread-safe, because each thread has its own machine stack. However, if you
	assign or pass back a non-NULL JIT stack, this must be a different stack for
	each thread so that the application is thread-safe.
	.P
	Strictly speaking, even more is allowed. You can assign the same non-NULL stack
	to any number of patterns as long as they are not used for matching by multiple
threads at the same time. For example, you can assign the same stack to all	threads at the same time. For example, you can assign the same stack to all
compiled patterns, and use a global mutex in the callback to wait until the	compiled patterns, and use a global mutex in the callback to wait until the
stack is available for use. However, this is an inefficient solution, and	stack is available for use. However, this is an inefficient solution, and not
not recommended.	recommended.
.P	.P
This is a suggestion for how a typical multithreaded program might operate:	This is a suggestion for how a multithreaded program that needs to set up
	non-default JIT stacks might operate:
.sp	.sp
During thread initalization	During thread initalization
thread_local_var = pcre_jit_stack_alloc(...)	thread_local_var = pcre_jit_stack_alloc(...)
Line 240 This is a suggestion for how a typical multithreaded p	Line 281 This is a suggestion for how a typical multithreaded p
All the functions described in this section do nothing if JIT is not available,	All the functions described in this section do nothing if JIT is not available,
and \fBpcre_assign_jit_stack()\fP does nothing unless the \fBextra\fP argument	and \fBpcre_assign_jit_stack()\fP does nothing unless the \fBextra\fP argument
is non-NULL and points to a \fBpcre_extra\fP block that is the result of a	is non-NULL and points to a \fBpcre_extra\fP block that is the result of a
successful study with PCRE_STUDY_JIT_COMPILE.	successful study with PCRE_STUDY_JIT_COMPILE etc.
.	.
.	.
.\" HTML <a name="stackfaq"></a>	.\" HTML <a name="stackfaq"></a>
Line 291 replacement.	Line 332 replacement.
.sp	.sp
No, because this is too costly in terms of resources. However, you could	No, because this is too costly in terms of resources. However, you could
implement some clever idea which release the stack if it is not used in let's	implement some clever idea which release the stack if it is not used in let's
say two minutes. The JIT callback can help to achive this without keeping a	say two minutes. The JIT callback can help to achieve this without keeping a
list of the currently JIT studied patterns.	list of the currently JIT studied patterns.
.P	.P
(6) OK, the stack is for long term memory allocation. But what happens if a	(6) OK, the stack is for long term memory allocation. But what happens if a
pattern causes stack overflow with a stack of 1M? Is that 1M kept until the	pattern causes stack overflow with a stack of 1M? Is that 1M kept until the
stack is freed?	stack is freed?
.sp	.sp
Especially on embedded sytems, it might be a good idea to release	Especially on embedded sytems, it might be a good idea to release memory
memory sometimes without freeing the stack. There is no API for this at the	sometimes without freeing the stack. There is no API for this at the moment.
moment. Probably a function call which returns with the currently allocated	Probably a function call which returns with the currently allocated memory for
memory for any stack and another which allows releasing memory (shrinking the	any stack and another which allows releasing memory (shrinking the stack) would
stack) would be a good idea if someone needs this.	be a good idea if someone needs this.
.P	.P
(7) This is too much of a headache. Isn't there any better solution for JIT	(7) This is too much of a headache. Isn't there any better solution for JIT
stack handling?	stack handling?
Line 337 callback.	Line 378 callback.
.sp	.sp
.	.
.	.
	.SH "JIT FAST PATH API"
	.rs
	.sp
	Because the API described above falls back to interpreted execution when JIT is
	not available, it is convenient for programs that are written for general use
	in many environments. However, calling JIT via \fBpcre_exec()\fP does have a
	performance impact. Programs that are written for use where JIT is known to be
	available, and which need the best possible performance, can instead use a
	"fast path" API to call JIT execution directly instead of calling
	\fBpcre_exec()\fP (obviously only for patterns that have been successfully
	studied by JIT).
	.P
	The fast path function is called \fBpcre_jit_exec()\fP, and it takes exactly
	the same arguments as \fBpcre_exec()\fP, plus one additional argument that
	must point to a JIT stack. The JIT stack arrangements described above do not
	apply. The return values are the same as for \fBpcre_exec()\fP.
	.P
	When you call \fBpcre_exec()\fP, as well as testing for invalid options, a
	number of other sanity checks are performed on the arguments. For example, if
	the subject pointer is NULL, or its length is negative, an immediate error is
	given. Also, unless PCRE_NO_UTF[8\|16\|32] is set, a UTF subject string is tested
	for validity. In the interests of speed, these checks do not happen on the JIT
	fast path, and if invalid data is passed, the result is undefined.
	.P
	Bypassing the sanity checks and the \fBpcre_exec()\fP wrapping can give
	speedups of more than 10%.
	.
	.
.SH "SEE ALSO"	.SH "SEE ALSO"
.rs	.rs
.sp	.sp
Line 357 Cambridge CB2 3QH, England.	Line 426 Cambridge CB2 3QH, England.
.rs	.rs
.sp	.sp
.nf	.nf
Last updated: 26 November 2011	Last updated: 17 March 2013
Copyright (c) 1997-2011 University of Cambridge.	Copyright (c) 1997-2013 University of Cambridge.
.fi	.fi

FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>

Removed from v.1.1.1.1
changed lines
	Added in v.1.1.1.4