|
version 1.1.1.1, 2012/02/21 23:05:52
|
version 1.1.1.4, 2013/07/22 08:25:57
|
|
Line 14 man page, in case the conversion went wrong.
|
Line 14 man page, in case the conversion went wrong.
|
| <br> |
<br> |
| <ul> |
<ul> |
| <li><a name="TOC1" href="#SEC1">PARTIAL MATCHING IN PCRE</a> |
<li><a name="TOC1" href="#SEC1">PARTIAL MATCHING IN PCRE</a> |
| <li><a name="TOC2" href="#SEC2">PARTIAL MATCHING USING pcre_exec()</a> | <li><a name="TOC2" href="#SEC2">PARTIAL MATCHING USING pcre_exec() OR pcre[16|32]_exec()</a> |
| <li><a name="TOC3" href="#SEC3">PARTIAL MATCHING USING pcre_dfa_exec()</a> | <li><a name="TOC3" href="#SEC3">PARTIAL MATCHING USING pcre_dfa_exec() OR pcre[16|32]_dfa_exec()</a> |
| <li><a name="TOC4" href="#SEC4">PARTIAL MATCHING AND WORD BOUNDARIES</a> |
<li><a name="TOC4" href="#SEC4">PARTIAL MATCHING AND WORD BOUNDARIES</a> |
| <li><a name="TOC5" href="#SEC5">FORMERLY RESTRICTED PATTERNS</a> |
<li><a name="TOC5" href="#SEC5">FORMERLY RESTRICTED PATTERNS</a> |
| <li><a name="TOC6" href="#SEC6">EXAMPLE OF PARTIAL MATCHING USING PCRETEST</a> |
<li><a name="TOC6" href="#SEC6">EXAMPLE OF PARTIAL MATCHING USING PCRETEST</a> |
| <li><a name="TOC7" href="#SEC7">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec()</a> | <li><a name="TOC7" href="#SEC7">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre[16|32]_dfa_exec()</a> |
| <li><a name="TOC8" href="#SEC8">MULTI-SEGMENT MATCHING WITH pcre_exec()</a> | <li><a name="TOC8" href="#SEC8">MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre[16|32]_exec()</a> |
| <li><a name="TOC9" href="#SEC9">ISSUES WITH MULTI-SEGMENT MATCHING</a> |
<li><a name="TOC9" href="#SEC9">ISSUES WITH MULTI-SEGMENT MATCHING</a> |
| <li><a name="TOC10" href="#SEC10">AUTHOR</a> |
<li><a name="TOC10" href="#SEC10">AUTHOR</a> |
| <li><a name="TOC11" href="#SEC11">REVISION</a> |
<li><a name="TOC11" href="#SEC11">REVISION</a> |
| </ul> |
</ul> |
| <br><a name="SEC1" href="#TOC1">PARTIAL MATCHING IN PCRE</a><br> |
<br><a name="SEC1" href="#TOC1">PARTIAL MATCHING IN PCRE</a><br> |
| <P> |
<P> |
| In normal use of PCRE, if the subject string that is passed to | In normal use of PCRE, if the subject string that is passed to a matching |
| <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> matches as far as it goes, but is | function matches as far as it goes, but is too short to match the entire |
| too short to match the entire pattern, PCRE_ERROR_NOMATCH is returned. There | pattern, PCRE_ERROR_NOMATCH is returned. There are circumstances where it might |
| are circumstances where it might be helpful to distinguish this case from other | be helpful to distinguish this case from other cases in which there is no |
| cases in which there is no match. | match. |
| </P> |
</P> |
| <P> |
<P> |
| Consider, for example, an application where a human is required to type in data |
Consider, for example, an application where a human is required to type in data |
|
Line 50 long and is not all available at once.
|
Line 50 long and is not all available at once.
|
| </P> |
</P> |
| <P> |
<P> |
| PCRE supports partial matching by means of the PCRE_PARTIAL_SOFT and |
PCRE supports partial matching by means of the PCRE_PARTIAL_SOFT and |
| PCRE_PARTIAL_HARD options, which can be set when calling <b>pcre_exec()</b> or | PCRE_PARTIAL_HARD options, which can be set when calling any of the matching |
| <b>pcre_dfa_exec()</b>. For backwards compatibility, PCRE_PARTIAL is a synonym | functions. For backwards compatibility, PCRE_PARTIAL is a synonym for |
| for PCRE_PARTIAL_SOFT. The essential difference between the two options is | PCRE_PARTIAL_SOFT. The essential difference between the two options is whether |
| whether or not a partial match is preferred to an alternative complete match, | or not a partial match is preferred to an alternative complete match, though |
| though the details differ between the two matching functions. If both options | the details differ between the two types of matching function. If both options |
| are set, PCRE_PARTIAL_HARD takes precedence. |
are set, PCRE_PARTIAL_HARD takes precedence. |
| </P> |
</P> |
| <P> |
<P> |
| Setting a partial matching option for <b>pcre_exec()</b> disables the use of any | If you want to use partial matching with just-in-time optimized code, you must |
| just-in-time code that was set up by calling <b>pcre_study()</b> with the | call <b>pcre_study()</b>, <b>pcre16_study()</b> or <b>pcre32_study()</b> with one |
| PCRE_STUDY_JIT_COMPILE option. It also disables two of PCRE's standard | or both of these options: |
| optimizations. PCRE remembers the last literal byte in a pattern, and abandons | <pre> |
| matching immediately if such a byte is not present in the subject string. This | PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE |
| | PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE |
| | </pre> |
| | PCRE_STUDY_JIT_COMPILE should also be set if you are going to run non-partial |
| | matches on the same pattern. If the appropriate JIT study mode has not been set |
| | for a match, the interpretive matching code is used. |
| | </P> |
| | <P> |
| | Setting a partial matching option disables two of PCRE's standard |
| | optimizations. PCRE remembers the last literal data unit in a pattern, and |
| | abandons matching immediately if it is not present in the subject string. This |
| optimization cannot be used for a subject string that might match only |
optimization cannot be used for a subject string that might match only |
| partially. If the pattern was studied, PCRE knows the minimum length of a |
partially. If the pattern was studied, PCRE knows the minimum length of a |
| matching string, and does not bother to run the matching function on shorter |
matching string, and does not bother to run the matching function on shorter |
| strings. This optimization is also disabled for partial matching. |
strings. This optimization is also disabled for partial matching. |
| </P> |
</P> |
| <br><a name="SEC2" href="#TOC1">PARTIAL MATCHING USING pcre_exec()</a><br> | <br><a name="SEC2" href="#TOC1">PARTIAL MATCHING USING pcre_exec() OR pcre[16|32]_exec()</a><br> |
| <P> |
<P> |
| A partial match occurs during a call to <b>pcre_exec()</b> when the end of the | A partial match occurs during a call to <b>pcre_exec()</b> or |
| subject string is reached successfully, but matching cannot continue because | <b>pcre[16|32]_exec()</b> when the end of the subject string is reached |
| more characters are needed. However, at least one character in the subject must | successfully, but matching cannot continue because more characters are needed. |
| have been inspected. This character need not form part of the final matched | However, at least one character in the subject must have been inspected. This |
| string; lookbehind assertions and the \K escape sequence provide ways of | character need not form part of the final matched string; lookbehind assertions |
| inspecting characters before the start of a matched substring. The requirement | and the \K escape sequence provide ways of inspecting characters before the |
| for inspecting at least one character exists because an empty string can always | start of a matched substring. The requirement for inspecting at least one |
| be matched; without such a restriction there would always be a partial match of | character exists because an empty string can always be matched; without such a |
| an empty string at the end of the subject. | restriction there would always be a partial match of an empty string at the end |
| | of the subject. |
| </P> |
</P> |
| <P> |
<P> |
| If there are at least two slots in the offsets vector when <b>pcre_exec()</b> | If there are at least two slots in the offsets vector when a partial match is |
| returns with a partial match, the first slot is set to the offset of the | returned, the first slot is set to the offset of the earliest character that |
| earliest character that was inspected when the partial match was found. For | was inspected. For convenience, the second offset points to the end of the |
| convenience, the second offset points to the end of the subject so that a | subject so that a substring can easily be identified. If there are at least |
| substring can easily be identified. | three slots in the offsets vector, the third slot is set to the offset of the |
| | character where matching started. |
| </P> |
</P> |
| <P> |
<P> |
| For the majority of patterns, the first offset identifies the start of the | For the majority of patterns, the contents of the first and third slots will be |
| partially matched string. However, for patterns that contain lookbehind | the same. However, for patterns that contain lookbehind assertions, or begin |
| assertions, or \K, or begin with \b or \B, earlier characters have been | with \b or \B, characters before the one where matching started may have been |
| inspected while carrying out the match. For example: | inspected while carrying out the match. For example, consider this pattern: |
| <pre> |
<pre> |
| /(?<=abc)123/ |
/(?<=abc)123/ |
| </pre> |
</pre> |
| This pattern matches "123", but only if it is preceded by "abc". If the subject |
This pattern matches "123", but only if it is preceded by "abc". If the subject |
| string is "xyzabc12", the offsets after a partial match are for the substring | string is "xyzabc12", the first two offsets after a partial match are for the |
| "abc12", because all these characters are needed if another match is tried | substring "abc12", because all these characters were inspected. However, the |
| with extra characters added to the subject. | third offset is set to 6, because that is the offset where matching began. |
| </P> |
</P> |
| <P> |
<P> |
| What happens when a partial match is identified depends on which of the two |
What happens when a partial match is identified depends on which of the two |
| partial matching options are set. |
partial matching options are set. |
| </P> |
</P> |
| <br><b> |
<br><b> |
| PCRE_PARTIAL_SOFT with pcre_exec() | PCRE_PARTIAL_SOFT WITH pcre_exec() OR pcre[16|32]_exec() |
| </b><br> |
</b><br> |
| <P> |
<P> |
| If PCRE_PARTIAL_SOFT is set when <b>pcre_exec()</b> identifies a partial match, | If PCRE_PARTIAL_SOFT is set when <b>pcre_exec()</b> or <b>pcre[16|32]_exec()</b> |
| the partial match is remembered, but matching continues as normal, and other | identifies a partial match, the partial match is remembered, but matching |
| alternatives in the pattern are tried. If no complete match can be found, | continues as normal, and other alternatives in the pattern are tried. If no |
| <b>pcre_exec()</b> returns PCRE_ERROR_PARTIAL instead of PCRE_ERROR_NOMATCH. | complete match can be found, PCRE_ERROR_PARTIAL is returned instead of |
| | PCRE_ERROR_NOMATCH. |
| </P> |
</P> |
| <P> |
<P> |
| This option is "soft" because it prefers a complete match over a partial match. |
This option is "soft" because it prefers a complete match over a partial match. |
|
Line 134 example, there are two partial matches, because "dog"
|
Line 147 example, there are two partial matches, because "dog"
|
| matches the second alternative.) |
matches the second alternative.) |
| </P> |
</P> |
| <br><b> |
<br><b> |
| PCRE_PARTIAL_HARD with pcre_exec() | PCRE_PARTIAL_HARD WITH pcre_exec() OR pcre[16|32]_exec() |
| </b><br> |
</b><br> |
| <P> |
<P> |
| If PCRE_PARTIAL_HARD is set for <b>pcre_exec()</b>, it returns | If PCRE_PARTIAL_HARD is set for <b>pcre_exec()</b> or <b>pcre[16|32]_exec()</b>, |
| PCRE_ERROR_PARTIAL as soon as a partial match is found, without continuing to | PCRE_ERROR_PARTIAL is returned as soon as a partial match is found, without |
| search for possible complete matches. This option is "hard" because it prefers | continuing to search for possible complete matches. This option is "hard" |
| an earlier partial match over a later complete match. For this reason, the | because it prefers an earlier partial match over a later complete match. For |
| assumption is made that the end of the supplied subject string may not be the | this reason, the assumption is made that the end of the supplied subject string |
| true end of the available data, and so, if \z, \Z, \b, \B, or $ are | may not be the true end of the available data, and so, if \z, \Z, \b, \B, |
| encountered at the end of the subject, the result is PCRE_ERROR_PARTIAL. | or $ are encountered at the end of the subject, the result is |
| | PCRE_ERROR_PARTIAL, provided that at least one character in the subject has |
| | been inspected. |
| </P> |
</P> |
| <P> |
<P> |
| Setting PCRE_PARTIAL_HARD also affects the way <b>pcre_exec()</b> checks UTF-8 | Setting PCRE_PARTIAL_HARD also affects the way UTF-8 and UTF-16 |
| subject strings for validity. Normally, an invalid UTF-8 sequence causes the | subject strings are checked for validity. Normally, an invalid sequence |
| error PCRE_ERROR_BADUTF8. However, in the special case of a truncated UTF-8 | causes the error PCRE_ERROR_BADUTF8 or PCRE_ERROR_BADUTF16. However, in the |
| character at the end of the subject, PCRE_ERROR_SHORTUTF8 is returned when | special case of a truncated character at the end of the subject, |
| | PCRE_ERROR_SHORTUTF8 or PCRE_ERROR_SHORTUTF16 is returned when |
| PCRE_PARTIAL_HARD is set. |
PCRE_PARTIAL_HARD is set. |
| </P> |
</P> |
| <br><b> |
<br><b> |
|
Line 169 if the pattern is made ungreedy the result is differen
|
Line 185 if the pattern is made ungreedy the result is differen
|
| <pre> |
<pre> |
| /dog(sbody)??/ |
/dog(sbody)??/ |
| </pre> |
</pre> |
| In this case the result is always a complete match because <b>pcre_exec()</b> | In this case the result is always a complete match because that is found first, |
| finds that first, and it never continues after finding a match. It might be | and matching never continues after finding a complete match. It might be easier |
| easier to follow this explanation by thinking of the two patterns like this: | to follow this explanation by thinking of the two patterns like this: |
| <pre> |
<pre> |
| /dog(sbody)?/ is the same as /dogsbody|dog/ |
/dog(sbody)?/ is the same as /dogsbody|dog/ |
| /dog(sbody)??/ is the same as /dog|dogsbody/ |
/dog(sbody)??/ is the same as /dog|dogsbody/ |
| </pre> |
</pre> |
| The second pattern will never match "dogsbody" when <b>pcre_exec()</b> is | The second pattern will never match "dogsbody", because it will always find the |
| used, because it will always find the shorter match first. | shorter match first. |
| </P> |
</P> |
| <br><a name="SEC3" href="#TOC1">PARTIAL MATCHING USING pcre_dfa_exec()</a><br> | <br><a name="SEC3" href="#TOC1">PARTIAL MATCHING USING pcre_dfa_exec() OR pcre[16|32]_dfa_exec()</a><br> |
| <P> |
<P> |
| The <b>pcre_dfa_exec()</b> function moves along the subject string character by | The DFA functions move along the subject string character by character, without |
| character, without backtracking, searching for all possible matches | backtracking, searching for all possible matches simultaneously. If the end of |
| simultaneously. If the end of the subject is reached before the end of the | the subject is reached before the end of the pattern, there is the possibility |
| pattern, there is the possibility of a partial match, again provided that at | of a partial match, again provided that at least one character has been |
| least one character has been inspected. | inspected. |
| </P> |
</P> |
| <P> |
<P> |
| When PCRE_PARTIAL_SOFT is set, PCRE_ERROR_PARTIAL is returned only if there |
When PCRE_PARTIAL_SOFT is set, PCRE_ERROR_PARTIAL is returned only if there |
|
Line 196 partial match was found is set as the first matching s
|
Line 212 partial match was found is set as the first matching s
|
| at least two slots in the offsets vector. |
at least two slots in the offsets vector. |
| </P> |
</P> |
| <P> |
<P> |
| Because <b>pcre_dfa_exec()</b> always searches for all possible matches, and | Because the DFA functions always search for all possible matches, and there is |
| there is no difference between greedy and ungreedy repetition, its behaviour is | no difference between greedy and ungreedy repetition, their behaviour is |
| different from <b>pcre_exec</b> when PCRE_PARTIAL_HARD is set. Consider the | different from the standard functions when PCRE_PARTIAL_HARD is set. Consider |
| string "dog" matched against the ungreedy pattern shown above: | the string "dog" matched against the ungreedy pattern shown above: |
| <pre> |
<pre> |
| /dog(sbody)??/ |
/dog(sbody)??/ |
| </pre> |
</pre> |
| Whereas <b>pcre_exec()</b> stops as soon as it finds the complete match for | Whereas the standard functions stop as soon as they find the complete match for |
| "dog", <b>pcre_dfa_exec()</b> also finds the partial match for "dogsbody", and | "dog", the DFA functions also find the partial match for "dogsbody", and so |
| so returns that when PCRE_PARTIAL_HARD is set. | return that when PCRE_PARTIAL_HARD is set. |
| </P> |
</P> |
| <br><a name="SEC4" href="#TOC1">PARTIAL MATCHING AND WORD BOUNDARIES</a><br> |
<br><a name="SEC4" href="#TOC1">PARTIAL MATCHING AND WORD BOUNDARIES</a><br> |
| <P> |
<P> |
|
Line 217 results. Consider this pattern:
|
Line 233 results. Consider this pattern:
|
| </pre> |
</pre> |
| This matches "cat", provided there is a word boundary at either end. If the |
This matches "cat", provided there is a word boundary at either end. If the |
| subject string is "the cat", the comparison of the final "t" with a following |
subject string is "the cat", the comparison of the final "t" with a following |
| character cannot take place, so a partial match is found. However, | character cannot take place, so a partial match is found. However, normal |
| <b>pcre_exec()</b> carries on with normal matching, which matches \b at the end | matching carries on, and \b matches at the end of the subject when the last |
| of the subject when the last character is a letter, thus finding a complete | character is a letter, so a complete match is found. The result, therefore, is |
| match. The result, therefore, is <i>not</i> PCRE_ERROR_PARTIAL. The same thing | <i>not</i> PCRE_ERROR_PARTIAL. Using PCRE_PARTIAL_HARD in this case does yield |
| happens with <b>pcre_dfa_exec()</b>, because it also finds the complete match. | PCRE_ERROR_PARTIAL, because then the partial match takes precedence. |
| </P> |
</P> |
| <P> |
|
| Using PCRE_PARTIAL_HARD in this case does yield PCRE_ERROR_PARTIAL, because |
|
| then the partial match takes precedence. |
|
| </P> |
|
| <br><a name="SEC5" href="#TOC1">FORMERLY RESTRICTED PATTERNS</a><br> |
<br><a name="SEC5" href="#TOC1">FORMERLY RESTRICTED PATTERNS</a><br> |
| <P> |
<P> |
| For releases of PCRE prior to 8.00, because of the way certain internal |
For releases of PCRE prior to 8.00, because of the way certain internal |
| optimizations were implemented in the <b>pcre_exec()</b> function, the |
optimizations were implemented in the <b>pcre_exec()</b> function, the |
| PCRE_PARTIAL option (predecessor of PCRE_PARTIAL_SOFT) could not be used with |
PCRE_PARTIAL option (predecessor of PCRE_PARTIAL_SOFT) could not be used with |
| all patterns. From release 8.00 onwards, the restrictions no longer apply, and |
all patterns. From release 8.00 onwards, the restrictions no longer apply, and |
| partial matching with <b>pcre_exec()</b> can be requested for any pattern. | partial matching with can be requested for any pattern. |
| </P> |
</P> |
| <P> |
<P> |
| Items that were formerly restricted were repeated single characters and |
Items that were formerly restricted were repeated single characters and |
|
Line 265 that uses the date example quoted above:
|
Line 277 that uses the date example quoted above:
|
| The first data string is matched completely, so <b>pcretest</b> shows the |
The first data string is matched completely, so <b>pcretest</b> shows the |
| matched substrings. The remaining four strings do not match the complete |
matched substrings. The remaining four strings do not match the complete |
| pattern, but the first two are partial matches. Similar output is obtained |
pattern, but the first two are partial matches. Similar output is obtained |
| when <b>pcre_dfa_exec()</b> is used. | if DFA matching is used. |
| </P> |
</P> |
| <P> |
<P> |
| If the escape sequence \P is present more than once in a <b>pcretest</b> data |
If the escape sequence \P is present more than once in a <b>pcretest</b> data |
| line, the PCRE_PARTIAL_HARD option is set for the match. |
line, the PCRE_PARTIAL_HARD option is set for the match. |
| </P> |
</P> |
| <br><a name="SEC7" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec()</a><br> | <br><a name="SEC7" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre[16|32]_dfa_exec()</a><br> |
| <P> |
<P> |
| When a partial match has been found using <b>pcre_dfa_exec()</b>, it is possible | When a partial match has been found using a DFA matching function, it is |
| to continue the match by providing additional subject data and calling | possible to continue the match by providing additional subject data and calling |
| <b>pcre_dfa_exec()</b> again with the same compiled regular expression, this | the function again with the same compiled regular expression, this time setting |
| time setting the PCRE_DFA_RESTART option. You must pass the same working | the PCRE_DFA_RESTART option. You must pass the same working space as before, |
| space as before, because this is where details of the previous partial match | because this is where details of the previous partial match are stored. Here is |
| are stored. Here is an example using <b>pcretest</b>, using the \R escape | an example using <b>pcretest</b>, using the \R escape sequence to set the |
| sequence to set the PCRE_DFA_RESTART option (\D specifies the use of | PCRE_DFA_RESTART option (\D specifies the use of the DFA matching function): |
| <b>pcre_dfa_exec()</b>): | |
| <pre> |
<pre> |
| re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ |
re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ |
| data> 23ja\P\D |
data> 23ja\P\D |
|
Line 297 program to do that if it needs to.
|
Line 308 program to do that if it needs to.
|
| <P> |
<P> |
| You can set the PCRE_PARTIAL_SOFT or PCRE_PARTIAL_HARD options with |
You can set the PCRE_PARTIAL_SOFT or PCRE_PARTIAL_HARD options with |
| PCRE_DFA_RESTART to continue partial matching over multiple segments. This |
PCRE_DFA_RESTART to continue partial matching over multiple segments. This |
| facility can be used to pass very long subject strings to | facility can be used to pass very long subject strings to the DFA matching |
| <b>pcre_dfa_exec()</b>. | functions. |
| </P> |
</P> |
| <br><a name="SEC8" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_exec()</a><br> | <br><a name="SEC8" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre[16|32]_exec()</a><br> |
| <P> |
<P> |
| From release 8.00, <b>pcre_exec()</b> can also be used to do multi-segment | From release 8.00, the standard matching functions can also be used to do |
| matching. Unlike <b>pcre_dfa_exec()</b>, it is not possible to restart the | multi-segment matching. Unlike the DFA functions, it is not possible to |
| previous match with a new segment of data. Instead, new data must be added to | restart the previous match with a new segment of data. Instead, new data must |
| the previous subject string, and the entire match re-run, starting from the | be added to the previous subject string, and the entire match re-run, starting |
| point where the partial match occurred. Earlier data can be discarded. It is | from the point where the partial match occurred. Earlier data can be discarded. |
| best to use PCRE_PARTIAL_HARD in this situation, because it does not treat the | </P> |
| end of a segment as the end of the subject when matching \z, \Z, \b, \B, | <P> |
| and $. Consider an unanchored pattern that matches dates: | It is best to use PCRE_PARTIAL_HARD in this situation, because it does not |
| | treat the end of a segment as the end of the subject when matching \z, \Z, |
| | \b, \B, and $. Consider an unanchored pattern that matches dates: |
| <pre> |
<pre> |
| re> /\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d/ |
re> /\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d/ |
| data> The date is 23ja\P\P |
data> The date is 23ja\P\P |
| Partial match: 23ja |
Partial match: 23ja |
| </pre> |
</pre> |
| At this stage, an application could discard the text preceding "23ja", add on |
At this stage, an application could discard the text preceding "23ja", add on |
| text from the next segment, and call <b>pcre_exec()</b> again. Unlike | text from the next segment, and call the matching function again. Unlike the |
| <b>pcre_dfa_exec()</b>, the entire matching string must always be available, and | DFA matching functions, the entire matching string must always be available, |
| the complete matching process occurs for each call, so more memory and more | and the complete matching process occurs for each call, so more memory and more |
| processing time is needed. |
processing time is needed. |
| </P> |
</P> |
| <P> |
<P> |
| <b>Note:</b> If the pattern contains lookbehind assertions, or \K, or starts |
<b>Note:</b> If the pattern contains lookbehind assertions, or \K, or starts |
| with \b or \B, the string that is returned for a partial match will include | with \b or \B, the string that is returned for a partial match includes |
| characters that precede the partially matched string itself, because these must | characters that precede the start of what would be returned for a complete |
| be retained when adding on more characters for a subsequent matching attempt. | match, because it contains all the characters that were inspected during the |
| | partial match. |
| </P> |
</P> |
| <br><a name="SEC9" href="#TOC1">ISSUES WITH MULTI-SEGMENT MATCHING</a><br> |
<br><a name="SEC9" href="#TOC1">ISSUES WITH MULTI-SEGMENT MATCHING</a><br> |
| <P> |
<P> |
|
Line 340 doing multi-segment matching you should be using PCRE_
|
Line 354 doing multi-segment matching you should be using PCRE_
|
| includes the effect of PCRE_NOTEOL. |
includes the effect of PCRE_NOTEOL. |
| </P> |
</P> |
| <P> |
<P> |
| 2. Lookbehind assertions at the start of a pattern are catered for in the | 2. Lookbehind assertions that have already been obeyed are catered for in the |
| offsets that are returned for a partial match. However, in theory, a lookbehind | offsets that are returned for a partial match. However a lookbehind assertion |
| assertion later in the pattern could require even earlier characters to be | later in the pattern could require even earlier characters to be inspected. You |
| inspected, and it might not have been reached when a partial match occurs. This | can handle this case by using the PCRE_INFO_MAXLOOKBEHIND option of the |
| is probably an extremely unlikely case; you could guard against it to a certain | <b>pcre_fullinfo()</b> or <b>pcre[16|32]_fullinfo()</b> functions to obtain the |
| extent by always including extra characters at the start. | length of the longest lookbehind in the pattern. This length is given in |
| | characters, not bytes. If you always retain at least that many characters |
| | before the partially matched string, all should be well. (Of course, near the |
| | start of the subject, fewer characters may be present; in that case all |
| | characters should be retained.) |
| </P> |
</P> |
| <P> |
<P> |
| 3. Matching a subject string that is split into multiple segments may not | From release 8.33, there is a more accurate way of deciding which characters to |
| | retain. Instead of subtracting the length of the longest lookbehind from the |
| | earliest inspected character (<i>offsets[0]</i>), the match start position |
| | (<i>offsets[2]</i>) should be used, and the next match attempt started at the |
| | <i>offsets[2]</i> character by setting the <i>startoffset</i> argument of |
| | <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b>. |
| | </P> |
| | <P> |
| | For example, if the pattern "(?<=123)abc" is partially |
| | matched against the string "xx123a", the three offset values returned are 2, 6, |
| | and 5. This indicates that the matching process that gave a partial match |
| | started at offset 5, but the characters "123a" were all inspected. The maximum |
| | lookbehind for that pattern is 3, so taking that away from 5 shows that we need |
| | only keep "123a", and the next match attempt can be started at offset 3 (that |
| | is, at "a") when further characters have been added. When the match start is |
| | not the earliest inspected character, <b>pcretest</b> shows it explicitly: |
| | <pre> |
| | re> "(?<=123)abc" |
| | data> xx123a\P\P |
| | Partial match at offset 5: 123a |
| | </PRE> |
| | </P> |
| | <P> |
| | 3. Because a partial match must always contain at least one character, what |
| | might be considered a partial match of an empty string actually gives a "no |
| | match" result. For example: |
| | <pre> |
| | re> /c(?<=abc)x/ |
| | data> ab\P |
| | No match |
| | </pre> |
| | If the next segment begins "cx", a match should be found, but this will only |
| | happen if characters from the previous segment are retained. For this reason, a |
| | "no match" result should be interpreted as "partial match of an empty string" |
| | when the pattern contains lookbehinds. |
| | </P> |
| | <P> |
| | 4. Matching a subject string that is split into multiple segments may not |
| always produce exactly the same result as matching over one single long string, |
always produce exactly the same result as matching over one single long string, |
| especially when PCRE_PARTIAL_SOFT is used. The section "Partial Matching and |
especially when PCRE_PARTIAL_SOFT is used. The section "Partial Matching and |
| Word Boundaries" above describes an issue that arises if the pattern ends with |
Word Boundaries" above describes an issue that arises if the pattern ends with |
|
Line 369 longer possible. Consider again this <b>pcretest</b> e
|
Line 424 longer possible. Consider again this <b>pcretest</b> e
|
| 0: dogsbody |
0: dogsbody |
| 1: dog |
1: dog |
| </pre> |
</pre> |
| The first data line passes the string "dogsb" to <b>pcre_exec()</b>, setting the | The first data line passes the string "dogsb" to a standard matching function, |
| PCRE_PARTIAL_SOFT option. Although the string is a partial match for | setting the PCRE_PARTIAL_SOFT option. Although the string is a partial match |
| "dogsbody", the result is not PCRE_ERROR_PARTIAL, because the shorter string | for "dogsbody", the result is not PCRE_ERROR_PARTIAL, because the shorter |
| "dog" is a complete match. Similarly, when the subject is presented to | string "dog" is a complete match. Similarly, when the subject is presented to |
| <b>pcre_dfa_exec()</b> in several parts ("do" and "gsb" being the first two) the | a DFA matching function in several parts ("do" and "gsb" being the first two) |
| match stops when "dog" has been found, and it is not possible to continue. On | the match stops when "dog" has been found, and it is not possible to continue. |
| the other hand, if "dogsbody" is presented as a single string, | On the other hand, if "dogsbody" is presented as a single string, a DFA |
| <b>pcre_dfa_exec()</b> finds both matches. | matching function finds both matches. |
| </P> |
</P> |
| <P> |
<P> |
| Because of these problems, it is best to use PCRE_PARTIAL_HARD when matching |
Because of these problems, it is best to use PCRE_PARTIAL_HARD when matching |
|
Line 390 multi-segment data. The example above then behaves dif
|
Line 445 multi-segment data. The example above then behaves dif
|
| data> gsb\R\P\P\D |
data> gsb\R\P\P\D |
| Partial match: gsb |
Partial match: gsb |
| </pre> |
</pre> |
| 4. Patterns that contain alternatives at the top level which do not all | 5. Patterns that contain alternatives at the top level which do not all start |
| start with the same pattern item may not work as expected when | with the same pattern item may not work as expected when PCRE_DFA_RESTART is |
| PCRE_DFA_RESTART is used with <b>pcre_dfa_exec()</b>. For example, consider this | used. For example, consider this pattern: |
| pattern: | |
| <pre> |
<pre> |
| 1234|3789 |
1234|3789 |
| </pre> |
</pre> |
|
Line 409 patterns or patterns such as:
|
Line 463 patterns or patterns such as:
|
| 1234|ABCD |
1234|ABCD |
| </pre> |
</pre> |
| where no string can be a partial match for both alternatives. This is not a |
where no string can be a partial match for both alternatives. This is not a |
| problem if <b>pcre_exec()</b> is used, because the entire match has to be rerun | problem if a standard matching function is used, because the entire match has |
| each time: | to be rerun each time: |
| <pre> |
<pre> |
| re> /1234|3789/ |
re> /1234|3789/ |
| data> ABC123\P\P |
data> ABC123\P\P |
|
Line 419 each time:
|
Line 473 each time:
|
| 0: 3789 |
0: 3789 |
| </pre> |
</pre> |
| Of course, instead of using PCRE_DFA_RESTART, the same technique of re-running |
Of course, instead of using PCRE_DFA_RESTART, the same technique of re-running |
| the entire match can also be used with <b>pcre_dfa_exec()</b>. Another | the entire match can also be used with the DFA matching functions. Another |
| possibility is to work with two buffers. If a partial match at offset <i>n</i> |
possibility is to work with two buffers. If a partial match at offset <i>n</i> |
| in the first buffer is followed by "no match" when PCRE_DFA_RESTART is used on |
in the first buffer is followed by "no match" when PCRE_DFA_RESTART is used on |
| the second buffer, you can then try a new match starting at offset <i>n+1</i> in |
the second buffer, you can then try a new match starting at offset <i>n+1</i> in |
|
Line 436 Cambridge CB2 3QH, England.
|
Line 490 Cambridge CB2 3QH, England.
|
| </P> |
</P> |
| <br><a name="SEC11" href="#TOC1">REVISION</a><br> |
<br><a name="SEC11" href="#TOC1">REVISION</a><br> |
| <P> |
<P> |
| Last updated: 26 August 2011 | Last updated: 20 February 2013 |
| <br> |
<br> |
| Copyright © 1997-2011 University of Cambridge. | Copyright © 1997-2013 University of Cambridge. |
| <br> |
<br> |
| <p> |
<p> |
| Return to the <a href="index.html">PCRE index page</a>. |
Return to the <a href="index.html">PCRE index page</a>. |
![[BACK]](/icons/cvsweb/back.gif){kind=icon}
Return to pcrepartial.html CVS log ![[TXT]](/icons/cvsweb/text.gif){kind=icon}
![[DIR]](/icons/cvsweb/dir.gif){kind=icon}
Up to [ELWIX - Embedded LightWeight unIX -] / embedaddon / pcre / doc / html