|
version 1.1.1.1, 2012/02/21 23:05:51
|
version 1.1.1.5, 2014/06/15 19:46:04
|
|
Line 49 complexity in Perl regular expressions, I couldn't do
|
Line 49 complexity in Perl regular expressions, I couldn't do
|
| first pass through the pattern is helpful for other reasons. |
first pass through the pattern is helpful for other reasons. |
| |
|
| |
|
| |
Support for 16-bit and 32-bit data strings |
| |
------------------------------------------- |
| |
|
| |
From release 8.30, PCRE supports 16-bit as well as 8-bit data strings; and from |
| |
release 8.32, PCRE supports 32-bit data strings. The library can be compiled |
| |
in any combination of 8-bit, 16-bit or 32-bit modes, creating up to three |
| |
different libraries. In the description that follows, the word "short" is used |
| |
for a 16-bit data quantity, and the word "unit" is used for a quantity that is |
| |
a byte in 8-bit mode, a short in 16-bit mode and a 32-bit word in 32-bit mode. |
| |
However, so as not to over-complicate the text, the names of PCRE functions are |
| |
given in 8-bit form only. |
| |
|
| |
|
| Computing the memory requirement: how it was |
Computing the memory requirement: how it was |
| -------------------------------------------- |
-------------------------------------------- |
| |
|
|
Line 81 runs more slowly than before (30% or more, depending o
|
Line 94 runs more slowly than before (30% or more, depending o
|
| is doing a full analysis of the pattern. My hope was that this would not be a |
is doing a full analysis of the pattern. My hope was that this would not be a |
| big issue, and in the event, nobody has commented on it. |
big issue, and in the event, nobody has commented on it. |
| |
|
| |
At release 8.34, a limit on the nesting depth of parentheses was re-introduced |
| |
(default 250, settable at build time) so as to put a limit on the amount of |
| |
system stack used by pcre_compile(). This is a safety feature for environments |
| |
with small stacks where the patterns are provided by users. |
| |
|
| |
|
| Traditional matching function |
Traditional matching function |
| ----------------------------- |
----------------------------- |
| |
|
|
Line 107 facilities are available, and those that are do not al
|
Line 125 facilities are available, and those that are do not al
|
| same way. See the user documentation for details. |
same way. See the user documentation for details. |
| |
|
| The algorithm that is used for pcre_dfa_exec() is not a traditional FSM, |
The algorithm that is used for pcre_dfa_exec() is not a traditional FSM, |
| because it may have a number of states active at one time. More work would be | because it may have a number of states active at one time. More work would be |
| needed at compile time to produce a traditional FSM where only one state is | needed at compile time to produce a traditional FSM where only one state is |
| ever active at once. I believe some other regex matchers work this way. | ever active at once. I believe some other regex matchers work this way. JIT |
| | support is not available for this kind of matching. |
| |
|
| |
|
| Changeable options |
Changeable options |
| ------------------ |
------------------ |
| |
|
| The /i, /m, or /s options (PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL) may | The /i, /m, or /s options (PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and some |
| change in the middle of patterns. From PCRE 8.13, their processing is handled | others) may change in the middle of patterns. From PCRE 8.13, their processing |
| entirely at compile time by generating different opcodes for the different | is handled entirely at compile time by generating different opcodes for the |
| settings. The runtime functions do not need to keep track of an options state | different settings. The runtime functions do not need to keep track of an |
| any more. | options state any more. |
| |
|
| |
|
| Format of compiled patterns |
Format of compiled patterns |
| --------------------------- |
--------------------------- |
| |
|
| The compiled form of a pattern is a vector of bytes, containing items of | The compiled form of a pattern is a vector of unsigned units (bytes in 8-bit |
| variable length. The first byte in an item is an opcode, and the length of the | mode, shorts in 16-bit mode, 32-bit words in 32-bit mode), containing items of |
| item is either implicit in the opcode or contained in the data bytes that | variable length. The first unit in an item contains an opcode, and the length |
| follow it. | of the item is either implicit in the opcode or contained in the data that |
| | follows it. |
| |
|
| In many cases below LINK_SIZE data values are specified for offsets within the | In many cases listed below, LINK_SIZE data values are specified for offsets |
| compiled pattern. The default value for LINK_SIZE is 2, but PCRE can be | within the compiled pattern. LINK_SIZE always specifies a number of bytes. The |
| compiled to use 3-byte or 4-byte values for these offsets (impairing the | default value for LINK_SIZE is 2, but PCRE can be compiled to use 3-byte or |
| performance). This is necessary only when patterns whose compiled length is | 4-byte values for these offsets, although this impairs the performance. (3-byte |
| greater than 64K are going to be processed. In this description, we assume the | LINK_SIZE values are available only in 8-bit mode.) Specifing a LINK_SIZE |
| "normal" compilation options. Data values that are counts (e.g. for | larger than 2 is necessary only when patterns whose compiled length is greater |
| quantifiers) are always just two bytes long. | than 64K are going to be processed. In this description, we assume the "normal" |
| | compilation options. Data values that are counts (e.g. quantifiers) are two |
| | bytes long in 8-bit mode (most significant byte first), or one unit in 16-bit |
| | and 32-bit modes. |
| |
|
| |
|
| Opcodes with no following data |
Opcodes with no following data |
| ------------------------------ |
------------------------------ |
| |
|
| These items are all just one byte long | These items are all just one unit long |
| |
|
| OP_END end of pattern |
OP_END end of pattern |
| OP_ANY match any one character other than newline |
OP_ANY match any one character other than newline |
| OP_ALLANY match any one character, including newline |
OP_ALLANY match any one character, including newline |
| OP_ANYBYTE match any single byte, even in UTF-8 mode | OP_ANYBYTE match any single unit, even in UTF-8/16 mode |
| OP_SOD match start of data: \A |
OP_SOD match start of data: \A |
| OP_SOM, start of match (subject + offset): \G |
OP_SOM, start of match (subject + offset): \G |
| OP_SET_SOM, set start of match (\K) |
OP_SET_SOM, set start of match (\K) |
|
Line 164 These items are all just one byte long
|
Line 188 These items are all just one byte long
|
| OP_VSPACE \v |
OP_VSPACE \v |
| OP_NOT_WORDCHAR \W |
OP_NOT_WORDCHAR \W |
| OP_WORDCHAR \w |
OP_WORDCHAR \w |
| OP_EODN match end of data or \n at end: \Z | OP_EODN match end of data or newline at end: \Z |
| OP_EOD match end of data: \z |
OP_EOD match end of data: \z |
| OP_DOLL $ (end of data, or before final newline) |
OP_DOLL $ (end of data, or before final newline) |
| OP_DOLLM $ multiline mode (end of data or before newline) |
OP_DOLLM $ multiline mode (end of data or before newline) |
| OP_EXTUNI match an extended Unicode character | OP_EXTUNI match an extended Unicode grapheme cluster |
| OP_ANYNL match any Unicode newline sequence |
OP_ANYNL match any Unicode newline sequence |
| |
|
| |
OP_ASSERT_ACCEPT ) |
| OP_ACCEPT ) These are Perl 5.10's "backtracking control |
OP_ACCEPT ) These are Perl 5.10's "backtracking control |
| OP_COMMIT ) verbs". If OP_ACCEPT is inside capturing |
OP_COMMIT ) verbs". If OP_ACCEPT is inside capturing |
| OP_FAIL ) parentheses, it may be preceded by one or more |
OP_FAIL ) parentheses, it may be preceded by one or more |
| OP_PRUNE ) OP_CLOSE, followed by a 2-byte number, | OP_PRUNE ) OP_CLOSE, each followed by a count that |
| OP_SKIP ) indicating which parentheses must be closed. | OP_SKIP ) indicates which parentheses must be closed. |
| | OP_THEN ) |
| |
|
| |
OP_ASSERT_ACCEPT is used when (*ACCEPT) is encountered within an assertion. |
| |
This ends the assertion, not the entire pattern match. |
| |
|
| |
|
| Backtracking control verbs with (optional) data | Backtracking control verbs with optional data |
| ----------------------------------------------- | --------------------------------------------- |
| |
|
| (*THEN) without an argument generates the opcode OP_THEN and no following data. |
(*THEN) without an argument generates the opcode OP_THEN and no following data. |
| OP_MARK is followed by the mark name, preceded by a one-byte length, and | OP_MARK is followed by the mark name, preceded by a one-unit length, and |
| followed by a binary zero. For (*PRUNE), (*SKIP), and (*THEN) with arguments, |
followed by a binary zero. For (*PRUNE), (*SKIP), and (*THEN) with arguments, |
| the opcodes OP_PRUNE_ARG, OP_SKIP_ARG, and OP_THEN_ARG are used, with the name |
the opcodes OP_PRUNE_ARG, OP_SKIP_ARG, and OP_THEN_ARG are used, with the name |
| following in the same format. | following in the same format as OP_MARK. |
| |
|
| |
|
| Matching literal characters |
Matching literal characters |
| --------------------------- |
--------------------------- |
| |
|
| The OP_CHAR opcode is followed by a single character that is to be matched |
The OP_CHAR opcode is followed by a single character that is to be matched |
| casefully. For caseless matching, OP_CHARI is used. In UTF-8 mode, the | casefully. For caseless matching, OP_CHARI is used. In UTF-8 or UTF-16 modes, |
| character may be more than one byte long. (Earlier versions of PCRE used | the character may be more than one unit long. In UTF-32 mode, characters |
| multi-character strings, but this was changed to allow some new features to be | are always exactly one unit long. |
| added.) | |
| |
|
| |
If there is only one character in a character class, OP_CHAR or OP_CHARI is |
| |
used for a positive class, and OP_NOT or OP_NOTI for a negative one (that is, |
| |
for something like [^a]). |
| |
|
| |
|
| Repeating single characters |
Repeating single characters |
| --------------------------- |
--------------------------- |
| |
|
| The common repeats (*, +, ?) when applied to a single character use the | The common repeats (*, +, ?), when applied to a single character, use the |
| following opcodes, which come in caseful and caseless versions: |
following opcodes, which come in caseful and caseless versions: |
| |
|
| Caseful Caseless |
Caseful Caseless |
|
Line 215 following opcodes, which come in caseful and caseless
|
Line 247 following opcodes, which come in caseful and caseless
|
| OP_MINQUERY OP_MINQUERYI |
OP_MINQUERY OP_MINQUERYI |
| OP_POSQUERY OP_POSQUERYI |
OP_POSQUERY OP_POSQUERYI |
| |
|
| In ASCII mode, these are two-byte items; in UTF-8 mode, the length is variable. | Each opcode is followed by the character that is to be repeated. In ASCII mode, |
| Those with "MIN" in their name are the minimizing versions. Those with "POS" in | these are two-unit items; in UTF-8 or UTF-16 modes, the length is variable; in |
| their names are possessive versions. Each is followed by the character that is | UTF-32 mode these are one-unit items. Those with "MIN" in their names are the |
| to be repeated. Other repeats make use of these opcodes: | minimizing versions. Those with "POS" in their names are possessive versions. |
| | Other repeats make use of these opcodes: |
| |
|
| Caseful Caseless |
Caseful Caseless |
| OP_UPTO OP_UPTOI |
OP_UPTO OP_UPTOI |
|
Line 226 to be repeated. Other repeats make use of these opcode
|
Line 259 to be repeated. Other repeats make use of these opcode
|
| OP_POSUPTO OP_POSUPTOI |
OP_POSUPTO OP_POSUPTOI |
| OP_EXACT OP_EXACTI |
OP_EXACT OP_EXACTI |
| |
|
| Each of these is followed by a two-byte count (most significant first) and the | Each of these is followed by a count and then the repeated character. OP_UPTO |
| repeated character. OP_UPTO matches from 0 to the given number. A repeat with a | matches from 0 to the given number. A repeat with a non-zero minimum and a |
| non-zero minimum and a fixed maximum is coded as an OP_EXACT followed by an | fixed maximum is coded as an OP_EXACT followed by an OP_UPTO (or OP_MINUPTO or |
| OP_UPTO (or OP_MINUPTO or OPT_POSUPTO). | OPT_POSUPTO). |
| |
|
| |
Another set of matching repeating opcodes (called OP_NOTSTAR, OP_NOTSTARI, |
| |
etc.) are used for repeated, negated, single-character classes such as [^a]*. |
| |
The normal single-character opcodes (OP_STAR, etc.) are used for repeated |
| |
positive single-character classes. |
| |
|
| |
|
| Repeating character types |
Repeating character types |
| ------------------------- |
------------------------- |
| |
|
| Repeats of things like \d are done exactly as for single characters, except |
Repeats of things like \d are done exactly as for single characters, except |
| that instead of a character, the opcode for the type is stored in the data |
that instead of a character, the opcode for the type is stored in the data |
| byte. The opcodes are: | unit. The opcodes are: |
| |
|
| OP_TYPESTAR |
OP_TYPESTAR |
| OP_TYPEMINSTAR |
OP_TYPEMINSTAR |
|
Line 259 Match by Unicode property
|
Line 297 Match by Unicode property
|
| |
|
| OP_PROP and OP_NOTPROP are used for positive and negative matches of a |
OP_PROP and OP_NOTPROP are used for positive and negative matches of a |
| character by testing its Unicode property (the \p and \P escape sequences). |
character by testing its Unicode property (the \p and \P escape sequences). |
| Each is followed by two bytes that encode the desired property as a type and a | Each is followed by two units that encode the desired property as a type and a |
| value. | value. The types are a set of #defines of the form PT_xxx, and the values are |
| | enumerations of the form ucp_xx, defined in the ucp.h source file. The value is |
| | relevant only for PT_GC (General Category), PT_PC (Particular Category), and |
| | PT_SC (Script). |
| |
|
| Repeats of these items use the OP_TYPESTAR etc. set of opcodes, followed by | Repeats of these items use the OP_TYPESTAR etc. set of opcodes, followed by |
| three bytes: OP_PROP or OP_NOTPROP and then the desired property type and | three units: OP_PROP or OP_NOTPROP, and then the desired property type and |
| value. |
value. |
| |
|
| |
|
| Character classes |
Character classes |
| ----------------- |
----------------- |
| |
|
| If there is only one character, OP_CHAR or OP_CHARI is used for a positive | If there is only one character in a class, OP_CHAR or OP_CHARI is used for a |
| class, and OP_NOT or OP_NOTI for a negative one (that is, for something like | positive class, and OP_NOT or OP_NOTI for a negative one (that is, for |
| [^a]). However, in UTF-8 mode, the use of OP_NOT[I] applies only to characters | something like [^a]). |
| with values < 128, because OP_NOT[I] is confined to single bytes. | |
| |
|
| Another set of 13 repeating opcodes (called OP_NOTSTAR etc.) are used for a | A set of repeating opcodes (called OP_NOTSTAR etc.) are used for repeated, |
| repeated, negated, single-character class. The normal single-character opcodes | negated, single-character classes. The normal single-character opcodes |
| (OP_STAR, etc.) are used for a repeated positive single-character class. | (OP_STAR, etc.) are used for repeated positive single-character classes. |
| |
|
| When there is more than one character in a class and all the characters are | When there is more than one character in a class, and all the code points are |
| less than 256, OP_CLASS is used for a positive class, and OP_NCLASS for a |
less than 256, OP_CLASS is used for a positive class, and OP_NCLASS for a |
| negative one. In either case, the opcode is followed by a 32-byte bit map | negative one. In either case, the opcode is followed by a 32-byte (16-short, |
| containing a 1 bit for every character that is acceptable. The bits are counted | 8-word) bit map containing a 1 bit for every character that is acceptable. The |
| from the least significant end of each byte. In caseless mode, bits for both | bits are counted from the least significant end of each unit. In caseless mode, |
| cases are set. | bits for both cases are set. |
| |
|
| The reason for having both OP_CLASS and OP_NCLASS is so that, in UTF-8 mode, | The reason for having both OP_CLASS and OP_NCLASS is so that, in UTF-8/16/32 |
| subject characters with values greater than 256 can be handled correctly. For | mode, subject characters with values greater than 255 can be handled correctly. |
| OP_CLASS they do not match, whereas for OP_NCLASS they do. | For OP_CLASS they do not match, whereas for OP_NCLASS they do. |
| |
|
| For classes containing characters with values > 255, OP_XCLASS is used. It | For classes containing characters with values greater than 255 or that contain |
| optionally uses a bit map (if any characters lie within it), followed by a list | \p or \P, OP_XCLASS is used. It optionally uses a bit map if any code points |
| of pairs (for a range) and single characters. In caseless mode, both cases are | are less than 256, followed by a list of pairs (for a range) and single |
| explicitly listed. There is a flag character than indicates whether it is a | characters. In caseless mode, both cases are explicitly listed. |
| positive or a negative class. | |
| |
|
| |
OP_XCLASS is followed by a unit containing flag bits: XCL_NOT indicates that |
| |
this is a negative class, and XCL_MAP indicates that a bit map is present. |
| |
There follows the bit map, if XCL_MAP is set, and then a sequence of items |
| |
coded as follows: |
| |
|
| |
XCL_END marks the end of the list |
| |
XCL_SINGLE one character follows |
| |
XCL_RANGE two characters follow |
| |
XCL_PROP a Unicode property (type, value) follows |
| |
XCL_NOTPROP a Unicode property (type, value) follows |
| |
|
| |
If a range starts with a code point less than 256 and ends with one greater |
| |
than 256, an XCL_RANGE item is used, without setting any bits in the bit map. |
| |
This means that if no other items in the class set bits in the map, a map is |
| |
not needed. |
| |
|
| |
|
| Back references |
Back references |
| --------------- |
--------------- |
| |
|
| OP_REF (caseful) or OP_REFI (caseless) is followed by two bytes containing the | OP_REF (caseful) or OP_REFI (caseless) is followed by a count containing the |
| reference number. | reference number if the reference is to a unique capturing group (either by |
| | number or by name). When named groups are used, there may be more than one |
| | group with the same name. In this case, a reference by name generates OP_DNREF |
| | or OP_DNREFI. These are followed by two counts: the index (not the byte offset) |
| | in the group name table of the first entry for the requred name, followed by |
| | the number of groups with the same name. |
| |
|
| |
|
| Repeating character classes and back references |
Repeating character classes and back references |
| ----------------------------------------------- |
----------------------------------------------- |
| |
|
| Single-character classes are handled specially (see above). This section |
Single-character classes are handled specially (see above). This section |
| applies to OP_CLASS and OP_REF[I]. In both cases, the repeat information | applies to other classes and also to back references. In both cases, the repeat |
| follows the base item. The matching code looks at the following opcode to see | information follows the base item. The matching code looks at the following |
| if it is one of | opcode to see if it is one of |
| |
|
| OP_CRSTAR |
OP_CRSTAR |
| OP_CRMINSTAR |
OP_CRMINSTAR |
| |
OP_CRPOSSTAR |
| OP_CRPLUS |
OP_CRPLUS |
| OP_CRMINPLUS |
OP_CRMINPLUS |
| |
OP_CRPOSPLUS |
| OP_CRQUERY |
OP_CRQUERY |
| OP_CRMINQUERY |
OP_CRMINQUERY |
| |
OP_CRPOSQUERY |
| OP_CRRANGE |
OP_CRRANGE |
| OP_CRMINRANGE |
OP_CRMINRANGE |
| |
OP_CRPOSRANGE |
| |
|
| All but the last two are just single-byte items. The others are followed by | All but the last three are single-unit items, with no data. The others are |
| four bytes of data, comprising the minimum and maximum repeat counts. There are | followed by the minimum and maximum repeat counts. |
| no special possessive opcodes for these repeats; a possessive repeat is | |
| compiled into an atomic group. | |
| |
|
| |
|
| Brackets and alternation |
Brackets and alternation |
| ------------------------ |
------------------------ |
| |
|
| A pair of non-capturing (round) brackets is wrapped round each expression at | A pair of non-capturing round brackets is wrapped round each expression at |
| compile time, so alternation always happens in the context of brackets. |
compile time, so alternation always happens in the context of brackets. |
| |
|
| [Note for North Americans: "bracket" to some English speakers, including |
[Note for North Americans: "bracket" to some English speakers, including |
| myself, can be round, square, curly, or pointy. Hence this usage.] | myself, can be round, square, curly, or pointy. Hence this usage rather than |
| | "parentheses".] |
| |
|
| Non-capturing brackets use the opcode OP_BRA. Originally PCRE was limited to 99 |
Non-capturing brackets use the opcode OP_BRA. Originally PCRE was limited to 99 |
| capturing brackets and it used a different opcode for each one. From release |
capturing brackets and it used a different opcode for each one. From release |
|
Line 346 A bracket opcode is followed by LINK_SIZE bytes which
|
Line 409 A bracket opcode is followed by LINK_SIZE bytes which
|
| next alternative OP_ALT or, if there aren't any branches, to the matching |
next alternative OP_ALT or, if there aren't any branches, to the matching |
| OP_KET opcode. Each OP_ALT is followed by LINK_SIZE bytes giving the offset to |
OP_KET opcode. Each OP_ALT is followed by LINK_SIZE bytes giving the offset to |
| the next one, or to the OP_KET opcode. For capturing brackets, the bracket |
the next one, or to the OP_KET opcode. For capturing brackets, the bracket |
| number immediately follows the offset, always as a 2-byte item. | number is a count that immediately follows the offset. |
| |
|
| OP_KET is used for subpatterns that do not repeat indefinitely, while | OP_KET is used for subpatterns that do not repeat indefinitely, and OP_KETRMIN |
| OP_KETRMIN and OP_KETRMAX are used for indefinite repetitions, minimally or | and OP_KETRMAX are used for indefinite repetitions, minimally or maximally |
| maximally respectively (see below for possessive repetitions). All three are | respectively (see below for possessive repetitions). All three are followed by |
| followed by LINK_SIZE bytes giving (as a positive number) the offset back to | LINK_SIZE bytes giving (as a positive number) the offset back to the matching |
| the matching bracket opcode. | bracket opcode. |
| |
|
| If a subpattern is quantified such that it is permitted to match zero times, it |
If a subpattern is quantified such that it is permitted to match zero times, it |
| is preceded by one of OP_BRAZERO, OP_BRAMINZERO, or OP_SKIPZERO. These are |
is preceded by one of OP_BRAZERO, OP_BRAMINZERO, or OP_SKIPZERO. These are |
| single-byte opcodes that tell the matcher that skipping the following | single-unit opcodes that tell the matcher that skipping the following |
| subpattern entirely is a valid branch. In the case of the first two, not |
subpattern entirely is a valid branch. In the case of the first two, not |
| skipping the pattern is also valid (greedy and non-greedy). The third is used |
skipping the pattern is also valid (greedy and non-greedy). The third is used |
| when a pattern has the quantifier {0,0}. It cannot be entirely discarded, | when a pattern has the quantifier {0,0}. It cannot be entirely discarded, |
| because it may be called as a subroutine from elsewhere in the regex. |
because it may be called as a subroutine from elsewhere in the regex. |
| |
|
| A subpattern with an indefinite maximum repetition is replicated in the |
A subpattern with an indefinite maximum repetition is replicated in the |
|
Line 379 final replication is changed to OP_SBRA or OP_SCBRA. T
|
Line 442 final replication is changed to OP_SBRA or OP_SCBRA. T
|
| that it needs to check for matching an empty string when it hits OP_KETRMIN or |
that it needs to check for matching an empty string when it hits OP_KETRMIN or |
| OP_KETRMAX, and if so, to break the loop. |
OP_KETRMAX, and if so, to break the loop. |
| |
|
| |
|
| Possessive brackets |
Possessive brackets |
| ------------------- |
------------------- |
| |
|
|
Line 389 of OP_SCBRA. The end of such a group is marked by OP_K
|
Line 453 of OP_SCBRA. The end of such a group is marked by OP_K
|
| repetition is zero, the group is preceded by OP_BRAPOSZERO. |
repetition is zero, the group is preceded by OP_BRAPOSZERO. |
| |
|
| |
|
| |
Once-only (atomic) groups |
| |
------------------------- |
| |
|
| |
These are just like other subpatterns, but they start with the opcode |
| |
OP_ONCE or OP_ONCE_NC. The former is used when there are no capturing brackets |
| |
within the atomic group; the latter when there are. The distinction is needed |
| |
for when there is a backtrack to before the group - any captures within the |
| |
group must be reset, so it is necessary to retain backtracking points inside |
| |
the group even after it is complete in order to do this. When there are no |
| |
captures in an atomic group, all the backtracking can be discarded when it is |
| |
complete. This is more efficient, and also uses less stack. |
| |
|
| |
The check for matching an empty string in an unbounded repeat is handled |
| |
entirely at runtime, so there are just these two opcodes for atomic groups. |
| |
|
| |
|
| Assertions |
Assertions |
| ---------- |
---------- |
| |
|
| Forward assertions are just like other subpatterns, but starting with one of | Forward assertions are also just like other subpatterns, but starting with one |
| the opcodes OP_ASSERT or OP_ASSERT_NOT. Backward assertions use the opcodes | of the opcodes OP_ASSERT or OP_ASSERT_NOT. Backward assertions use the opcodes |
| OP_ASSERTBACK and OP_ASSERTBACK_NOT, and the first opcode inside the assertion |
OP_ASSERTBACK and OP_ASSERTBACK_NOT, and the first opcode inside the assertion |
| is OP_REVERSE, followed by a two byte count of the number of characters to move | is OP_REVERSE, followed by a count of the number of characters to move back the |
| back the pointer in the subject string. When operating in UTF-8 mode, the count | pointer in the subject string. In ASCII mode, the count is a number of units, |
| is a character count rather than a byte count. A separate count is present in | but in UTF-8/16 mode each character may occupy more than one unit; in UTF-32 |
| | mode each character occupies exactly one unit. A separate count is present in |
| each alternative of a lookbehind assertion, allowing them to have different |
each alternative of a lookbehind assertion, allowing them to have different |
| fixed lengths. |
fixed lengths. |
| |
|
| |
|
| Once-only (atomic) subpatterns |
|
| ------------------------------ |
|
| |
|
| These are also just like other subpatterns, but they start with the opcode |
|
| OP_ONCE. The check for matching an empty string in an unbounded repeat is |
|
| handled entirely at runtime, so there is just this one opcode. |
|
| |
|
| |
|
| Conditional subpatterns |
Conditional subpatterns |
| ----------------------- |
----------------------- |
| |
|
| These are like other subpatterns, but they start with the opcode OP_COND, or |
These are like other subpatterns, but they start with the opcode OP_COND, or |
| OP_SCOND for one that might match an empty string in an unbounded repeat. If |
OP_SCOND for one that might match an empty string in an unbounded repeat. If |
| the condition is a back reference, this is stored at the start of the |
the condition is a back reference, this is stored at the start of the |
| subpattern using the opcode OP_CREF followed by two bytes containing the | subpattern using the opcode OP_CREF followed by a count containing the |
| reference number. OP_NCREF is used instead if the reference was generated by | reference number, provided that the reference is to a unique capturing group. |
| name (so that the runtime code knows to check for duplicate names). | If the reference was by name and there is more than one group with that name, |
| | OP_DNCREF is used instead. It is followed by two counts: the index in the group |
| | names table, and the number of groups with the same name. |
| |
|
| If the condition is "in recursion" (coded as "(?(R)"), or "in recursion of |
If the condition is "in recursion" (coded as "(?(R)"), or "in recursion of |
| group x" (coded as "(?(Rx)"), the group number is stored at the start of the |
group x" (coded as "(?(Rx)"), the group number is stored at the start of the |
| subpattern using the opcode OP_RREF or OP_NRREF (cf OP_NCREF), and a value of | subpattern using the opcode OP_RREF (with a value of zero for "the whole |
| zero for "the whole pattern". For a DEFINE condition, just the single byte | pattern") or OP_DNRREF (with data as for OP_DNCREF). For a DEFINE condition, |
| OP_DEF is used (it has no associated data). Otherwise, a conditional subpattern | just the single unit OP_DEF is used (it has no associated data). Otherwise, a |
| always starts with one of the assertions. | conditional subpattern always starts with one of the assertions. |
| |
|
| |
|
| Recursion |
Recursion |
| --------- |
--------- |
| |
|
| Recursion either matches the current regex, or some subexpression. The opcode |
Recursion either matches the current regex, or some subexpression. The opcode |
| OP_RECURSE is followed by an value which is the offset to the starting bracket | OP_RECURSE is followed by aLINK_SIZE value that is the offset to the starting |
| from the start of the whole pattern. From release 6.5, OP_RECURSE is | bracket from the start of the whole pattern. From release 6.5, OP_RECURSE is |
| automatically wrapped inside OP_ONCE brackets (because otherwise some patterns | automatically wrapped inside OP_ONCE brackets, because otherwise some patterns |
| broke it). OP_RECURSE is also used for "subroutine" calls, even though they | broke it. OP_RECURSE is also used for "subroutine" calls, even though they are |
| are not strictly a recursion. | not strictly a recursion. |
| |
|
| |
|
| Callout |
Callout |
| ------- |
------- |
| |
|
| OP_CALLOUT is followed by one byte of data that holds a callout number in the | OP_CALLOUT is followed by one unit of data that holds a callout number in the |
| range 0 to 254 for manual callouts, or 255 for an automatic callout. In both |
range 0 to 254 for manual callouts, or 255 for an automatic callout. In both |
| cases there follows a two-byte value giving the offset in the pattern to the | cases there follows a count giving the offset in the pattern string to the |
| start of the following item, and another two-byte item giving the length of the | start of the following item, and another count giving the length of this item. |
| next item. | These values make is possible for pcretest to output useful tracing information |
| | using automatic callouts. |
| |
|
| |
|
| Philip Hazel |
Philip Hazel |
| October 2011 | November 2013 |
![[BACK]](/icons/cvsweb/back.gif){kind=icon}
Return to HACKING CVS log ![[TXT]](/icons/cvsweb/text.gif){kind=icon}
![[DIR]](/icons/cvsweb/dir.gif){kind=icon}
Up to [ELWIX - Embedded LightWeight unIX -] / embedaddon / pcre