version 1.1.1.2, 2012/02/21 23:50:25
|
version 1.1.1.4, 2013/07/22 08:25:57
|
Line 14 man page, in case the conversion went wrong.
|
Line 14 man page, in case the conversion went wrong.
|
<br> |
<br> |
<ul> |
<ul> |
<li><a name="TOC1" href="#SEC1">PARTIAL MATCHING IN PCRE</a> |
<li><a name="TOC1" href="#SEC1">PARTIAL MATCHING IN PCRE</a> |
<li><a name="TOC2" href="#SEC2">PARTIAL MATCHING USING pcre_exec() OR pcre16_exec()</a> | <li><a name="TOC2" href="#SEC2">PARTIAL MATCHING USING pcre_exec() OR pcre[16|32]_exec()</a> |
<li><a name="TOC3" href="#SEC3">PARTIAL MATCHING USING pcre_dfa_exec() OR pcre16_dfa_exec()</a> | <li><a name="TOC3" href="#SEC3">PARTIAL MATCHING USING pcre_dfa_exec() OR pcre[16|32]_dfa_exec()</a> |
<li><a name="TOC4" href="#SEC4">PARTIAL MATCHING AND WORD BOUNDARIES</a> |
<li><a name="TOC4" href="#SEC4">PARTIAL MATCHING AND WORD BOUNDARIES</a> |
<li><a name="TOC5" href="#SEC5">FORMERLY RESTRICTED PATTERNS</a> |
<li><a name="TOC5" href="#SEC5">FORMERLY RESTRICTED PATTERNS</a> |
<li><a name="TOC6" href="#SEC6">EXAMPLE OF PARTIAL MATCHING USING PCRETEST</a> |
<li><a name="TOC6" href="#SEC6">EXAMPLE OF PARTIAL MATCHING USING PCRETEST</a> |
<li><a name="TOC7" href="#SEC7">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre16_dfa_exec()</a> | <li><a name="TOC7" href="#SEC7">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre[16|32]_dfa_exec()</a> |
<li><a name="TOC8" href="#SEC8">MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre16_exec()</a> | <li><a name="TOC8" href="#SEC8">MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre[16|32]_exec()</a> |
<li><a name="TOC9" href="#SEC9">ISSUES WITH MULTI-SEGMENT MATCHING</a> |
<li><a name="TOC9" href="#SEC9">ISSUES WITH MULTI-SEGMENT MATCHING</a> |
<li><a name="TOC10" href="#SEC10">AUTHOR</a> |
<li><a name="TOC10" href="#SEC10">AUTHOR</a> |
<li><a name="TOC11" href="#SEC11">REVISION</a> |
<li><a name="TOC11" href="#SEC11">REVISION</a> |
Line 58 the details differ between the two types of matching f
|
Line 58 the details differ between the two types of matching f
|
are set, PCRE_PARTIAL_HARD takes precedence. |
are set, PCRE_PARTIAL_HARD takes precedence. |
</P> |
</P> |
<P> |
<P> |
Setting a partial matching option disables the use of any just-in-time code | If you want to use partial matching with just-in-time optimized code, you must |
that was set up by studying the compiled pattern with the | call <b>pcre_study()</b>, <b>pcre16_study()</b> or <b>pcre32_study()</b> with one |
PCRE_STUDY_JIT_COMPILE option. It also disables two of PCRE's standard | or both of these options: |
| <pre> |
| PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE |
| PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE |
| </pre> |
| PCRE_STUDY_JIT_COMPILE should also be set if you are going to run non-partial |
| matches on the same pattern. If the appropriate JIT study mode has not been set |
| for a match, the interpretive matching code is used. |
| </P> |
| <P> |
| Setting a partial matching option disables two of PCRE's standard |
optimizations. PCRE remembers the last literal data unit in a pattern, and |
optimizations. PCRE remembers the last literal data unit in a pattern, and |
abandons matching immediately if it is not present in the subject string. This |
abandons matching immediately if it is not present in the subject string. This |
optimization cannot be used for a subject string that might match only |
optimization cannot be used for a subject string that might match only |
Line 68 partially. If the pattern was studied, PCRE knows the
|
Line 78 partially. If the pattern was studied, PCRE knows the
|
matching string, and does not bother to run the matching function on shorter |
matching string, and does not bother to run the matching function on shorter |
strings. This optimization is also disabled for partial matching. |
strings. This optimization is also disabled for partial matching. |
</P> |
</P> |
<br><a name="SEC2" href="#TOC1">PARTIAL MATCHING USING pcre_exec() OR pcre16_exec()</a><br> | <br><a name="SEC2" href="#TOC1">PARTIAL MATCHING USING pcre_exec() OR pcre[16|32]_exec()</a><br> |
<P> |
<P> |
A partial match occurs during a call to <b>pcre_exec()</b> or |
A partial match occurs during a call to <b>pcre_exec()</b> or |
<b>pcre16_exec()</b> when the end of the subject string is reached successfully, | <b>pcre[16|32]_exec()</b> when the end of the subject string is reached |
but matching cannot continue because more characters are needed. However, at | successfully, but matching cannot continue because more characters are needed. |
least one character in the subject must have been inspected. This character | However, at least one character in the subject must have been inspected. This |
need not form part of the final matched string; lookbehind assertions and the | character need not form part of the final matched string; lookbehind assertions |
\K escape sequence provide ways of inspecting characters before the start of a | and the \K escape sequence provide ways of inspecting characters before the |
matched substring. The requirement for inspecting at least one character exists | start of a matched substring. The requirement for inspecting at least one |
because an empty string can always be matched; without such a restriction there | character exists because an empty string can always be matched; without such a |
would always be a partial match of an empty string at the end of the subject. | restriction there would always be a partial match of an empty string at the end |
| of the subject. |
</P> |
</P> |
<P> |
<P> |
If there are at least two slots in the offsets vector when a partial match is |
If there are at least two slots in the offsets vector when a partial match is |
returned, the first slot is set to the offset of the earliest character that |
returned, the first slot is set to the offset of the earliest character that |
was inspected. For convenience, the second offset points to the end of the |
was inspected. For convenience, the second offset points to the end of the |
subject so that a substring can easily be identified. | subject so that a substring can easily be identified. If there are at least |
| three slots in the offsets vector, the third slot is set to the offset of the |
| character where matching started. |
</P> |
</P> |
<P> |
<P> |
For the majority of patterns, the first offset identifies the start of the | For the majority of patterns, the contents of the first and third slots will be |
partially matched string. However, for patterns that contain lookbehind | the same. However, for patterns that contain lookbehind assertions, or begin |
assertions, or \K, or begin with \b or \B, earlier characters have been | with \b or \B, characters before the one where matching started may have been |
inspected while carrying out the match. For example: | inspected while carrying out the match. For example, consider this pattern: |
<pre> |
<pre> |
/(?<=abc)123/ |
/(?<=abc)123/ |
</pre> |
</pre> |
This pattern matches "123", but only if it is preceded by "abc". If the subject |
This pattern matches "123", but only if it is preceded by "abc". If the subject |
string is "xyzabc12", the offsets after a partial match are for the substring | string is "xyzabc12", the first two offsets after a partial match are for the |
"abc12", because all these characters are needed if another match is tried | substring "abc12", because all these characters were inspected. However, the |
with extra characters added to the subject. | third offset is set to 6, because that is the offset where matching began. |
</P> |
</P> |
<P> |
<P> |
What happens when a partial match is identified depends on which of the two |
What happens when a partial match is identified depends on which of the two |
partial matching options are set. |
partial matching options are set. |
</P> |
</P> |
<br><b> |
<br><b> |
PCRE_PARTIAL_SOFT WITH pcre_exec() OR pcre16_exec() | PCRE_PARTIAL_SOFT WITH pcre_exec() OR pcre[16|32]_exec() |
</b><br> |
</b><br> |
<P> |
<P> |
If PCRE_PARTIAL_SOFT is set when <b>pcre_exec()</b> or <b>pcre16_exec()</b> | If PCRE_PARTIAL_SOFT is set when <b>pcre_exec()</b> or <b>pcre[16|32]_exec()</b> |
identifies a partial match, the partial match is remembered, but matching |
identifies a partial match, the partial match is remembered, but matching |
continues as normal, and other alternatives in the pattern are tried. If no |
continues as normal, and other alternatives in the pattern are tried. If no |
complete match can be found, PCRE_ERROR_PARTIAL is returned instead of |
complete match can be found, PCRE_ERROR_PARTIAL is returned instead of |
Line 134 example, there are two partial matches, because "dog"
|
Line 147 example, there are two partial matches, because "dog"
|
matches the second alternative.) |
matches the second alternative.) |
</P> |
</P> |
<br><b> |
<br><b> |
PCRE_PARTIAL_HARD WITH pcre_exec() OR pcre16_exec() | PCRE_PARTIAL_HARD WITH pcre_exec() OR pcre[16|32]_exec() |
</b><br> |
</b><br> |
<P> |
<P> |
If PCRE_PARTIAL_HARD is set for <b>pcre_exec()</b> or <b>pcre16_exec()</b>, | If PCRE_PARTIAL_HARD is set for <b>pcre_exec()</b> or <b>pcre[16|32]_exec()</b>, |
PCRE_ERROR_PARTIAL is returned as soon as a partial match is found, without |
PCRE_ERROR_PARTIAL is returned as soon as a partial match is found, without |
continuing to search for possible complete matches. This option is "hard" |
continuing to search for possible complete matches. This option is "hard" |
because it prefers an earlier partial match over a later complete match. For |
because it prefers an earlier partial match over a later complete match. For |
Line 182 to follow this explanation by thinking of the two patt
|
Line 195 to follow this explanation by thinking of the two patt
|
The second pattern will never match "dogsbody", because it will always find the |
The second pattern will never match "dogsbody", because it will always find the |
shorter match first. |
shorter match first. |
</P> |
</P> |
<br><a name="SEC3" href="#TOC1">PARTIAL MATCHING USING pcre_dfa_exec() OR pcre16_dfa_exec()</a><br> | <br><a name="SEC3" href="#TOC1">PARTIAL MATCHING USING pcre_dfa_exec() OR pcre[16|32]_dfa_exec()</a><br> |
<P> |
<P> |
The DFA functions move along the subject string character by character, without |
The DFA functions move along the subject string character by character, without |
backtracking, searching for all possible matches simultaneously. If the end of |
backtracking, searching for all possible matches simultaneously. If the end of |
Line 270 if DFA matching is used.
|
Line 283 if DFA matching is used.
|
If the escape sequence \P is present more than once in a <b>pcretest</b> data |
If the escape sequence \P is present more than once in a <b>pcretest</b> data |
line, the PCRE_PARTIAL_HARD option is set for the match. |
line, the PCRE_PARTIAL_HARD option is set for the match. |
</P> |
</P> |
<br><a name="SEC7" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre16_dfa_exec()</a><br> | <br><a name="SEC7" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre[16|32]_dfa_exec()</a><br> |
<P> |
<P> |
When a partial match has been found using a DFA matching function, it is |
When a partial match has been found using a DFA matching function, it is |
possible to continue the match by providing additional subject data and calling |
possible to continue the match by providing additional subject data and calling |
Line 298 PCRE_DFA_RESTART to continue partial matching over mul
|
Line 311 PCRE_DFA_RESTART to continue partial matching over mul
|
facility can be used to pass very long subject strings to the DFA matching |
facility can be used to pass very long subject strings to the DFA matching |
functions. |
functions. |
</P> |
</P> |
<br><a name="SEC8" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre16_exec()</a><br> | <br><a name="SEC8" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre[16|32]_exec()</a><br> |
<P> |
<P> |
From release 8.00, the standard matching functions can also be used to do |
From release 8.00, the standard matching functions can also be used to do |
multi-segment matching. Unlike the DFA functions, it is not possible to |
multi-segment matching. Unlike the DFA functions, it is not possible to |
Line 317 treat the end of a segment as the end of the subject w
|
Line 330 treat the end of a segment as the end of the subject w
|
</pre> |
</pre> |
At this stage, an application could discard the text preceding "23ja", add on |
At this stage, an application could discard the text preceding "23ja", add on |
text from the next segment, and call the matching function again. Unlike the |
text from the next segment, and call the matching function again. Unlike the |
DFA matching functions the entire matching string must always be available, and | DFA matching functions, the entire matching string must always be available, |
the complete matching process occurs for each call, so more memory and more | and the complete matching process occurs for each call, so more memory and more |
processing time is needed. |
processing time is needed. |
</P> |
</P> |
<P> |
<P> |
<b>Note:</b> If the pattern contains lookbehind assertions, or \K, or starts |
<b>Note:</b> If the pattern contains lookbehind assertions, or \K, or starts |
with \b or \B, the string that is returned for a partial match includes |
with \b or \B, the string that is returned for a partial match includes |
characters that precede the partially matched string itself, because these must | characters that precede the start of what would be returned for a complete |
be retained when adding on more characters for a subsequent matching attempt. | match, because it contains all the characters that were inspected during the |
| partial match. |
</P> |
</P> |
<br><a name="SEC9" href="#TOC1">ISSUES WITH MULTI-SEGMENT MATCHING</a><br> |
<br><a name="SEC9" href="#TOC1">ISSUES WITH MULTI-SEGMENT MATCHING</a><br> |
<P> |
<P> |
Line 340 doing multi-segment matching you should be using PCRE_
|
Line 354 doing multi-segment matching you should be using PCRE_
|
includes the effect of PCRE_NOTEOL. |
includes the effect of PCRE_NOTEOL. |
</P> |
</P> |
<P> |
<P> |
2. Lookbehind assertions at the start of a pattern are catered for in the | 2. Lookbehind assertions that have already been obeyed are catered for in the |
offsets that are returned for a partial match. However, in theory, a lookbehind | offsets that are returned for a partial match. However a lookbehind assertion |
assertion later in the pattern could require even earlier characters to be | later in the pattern could require even earlier characters to be inspected. You |
inspected, and it might not have been reached when a partial match occurs. This | can handle this case by using the PCRE_INFO_MAXLOOKBEHIND option of the |
is probably an extremely unlikely case; you could guard against it to a certain | <b>pcre_fullinfo()</b> or <b>pcre[16|32]_fullinfo()</b> functions to obtain the |
extent by always including extra characters at the start. | length of the longest lookbehind in the pattern. This length is given in |
| characters, not bytes. If you always retain at least that many characters |
| before the partially matched string, all should be well. (Of course, near the |
| start of the subject, fewer characters may be present; in that case all |
| characters should be retained.) |
</P> |
</P> |
<P> |
<P> |
3. Matching a subject string that is split into multiple segments may not | From release 8.33, there is a more accurate way of deciding which characters to |
| retain. Instead of subtracting the length of the longest lookbehind from the |
| earliest inspected character (<i>offsets[0]</i>), the match start position |
| (<i>offsets[2]</i>) should be used, and the next match attempt started at the |
| <i>offsets[2]</i> character by setting the <i>startoffset</i> argument of |
| <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b>. |
| </P> |
| <P> |
| For example, if the pattern "(?<=123)abc" is partially |
| matched against the string "xx123a", the three offset values returned are 2, 6, |
| and 5. This indicates that the matching process that gave a partial match |
| started at offset 5, but the characters "123a" were all inspected. The maximum |
| lookbehind for that pattern is 3, so taking that away from 5 shows that we need |
| only keep "123a", and the next match attempt can be started at offset 3 (that |
| is, at "a") when further characters have been added. When the match start is |
| not the earliest inspected character, <b>pcretest</b> shows it explicitly: |
| <pre> |
| re> "(?<=123)abc" |
| data> xx123a\P\P |
| Partial match at offset 5: 123a |
| </PRE> |
| </P> |
| <P> |
| 3. Because a partial match must always contain at least one character, what |
| might be considered a partial match of an empty string actually gives a "no |
| match" result. For example: |
| <pre> |
| re> /c(?<=abc)x/ |
| data> ab\P |
| No match |
| </pre> |
| If the next segment begins "cx", a match should be found, but this will only |
| happen if characters from the previous segment are retained. For this reason, a |
| "no match" result should be interpreted as "partial match of an empty string" |
| when the pattern contains lookbehinds. |
| </P> |
| <P> |
| 4. Matching a subject string that is split into multiple segments may not |
always produce exactly the same result as matching over one single long string, |
always produce exactly the same result as matching over one single long string, |
especially when PCRE_PARTIAL_SOFT is used. The section "Partial Matching and |
especially when PCRE_PARTIAL_SOFT is used. The section "Partial Matching and |
Word Boundaries" above describes an issue that arises if the pattern ends with |
Word Boundaries" above describes an issue that arises if the pattern ends with |
Line 390 multi-segment data. The example above then behaves dif
|
Line 445 multi-segment data. The example above then behaves dif
|
data> gsb\R\P\P\D |
data> gsb\R\P\P\D |
Partial match: gsb |
Partial match: gsb |
</pre> |
</pre> |
4. Patterns that contain alternatives at the top level which do not all start | 5. Patterns that contain alternatives at the top level which do not all start |
with the same pattern item may not work as expected when PCRE_DFA_RESTART is |
with the same pattern item may not work as expected when PCRE_DFA_RESTART is |
used. For example, consider this pattern: |
used. For example, consider this pattern: |
<pre> |
<pre> |
Line 435 Cambridge CB2 3QH, England.
|
Line 490 Cambridge CB2 3QH, England.
|
</P> |
</P> |
<br><a name="SEC11" href="#TOC1">REVISION</a><br> |
<br><a name="SEC11" href="#TOC1">REVISION</a><br> |
<P> |
<P> |
Last updated: 21 January 2012 | Last updated: 20 February 2013 |
<br> |
<br> |
Copyright © 1997-2012 University of Cambridge. | Copyright © 1997-2013 University of Cambridge. |
<br> |
<br> |
<p> |
<p> |
Return to the <a href="index.html">PCRE index page</a>. |
Return to the <a href="index.html">PCRE index page</a>. |