If you want to use partial matching with just-in-time optimized code, you must -call pcre_study() or pcre16_study() with one or both of these -options: +call pcre_study(), pcre16_study() or pcre32_study() with one +or both of these options:
PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE @@ -78,46 +78,49 @@ partially. If the pattern was studied, PCRE knows the matching string, and does not bother to run the matching function on shorter strings. This optimization is also disabled for partial matching. -
PARTIAL MATCHING USING pcre_exec() OR pcre16_exec()
+
PARTIAL MATCHING USING pcre_exec() OR pcre[16|32]_exec()
A partial match occurs during a call to pcre_exec() or -pcre16_exec() when the end of the subject string is reached successfully, -but matching cannot continue because more characters are needed. However, at -least one character in the subject must have been inspected. This character -need not form part of the final matched string; lookbehind assertions and the -\K escape sequence provide ways of inspecting characters before the start of a -matched substring. The requirement for inspecting at least one character exists -because an empty string can always be matched; without such a restriction there -would always be a partial match of an empty string at the end of the subject. +pcre[16|32]_exec() when the end of the subject string is reached +successfully, but matching cannot continue because more characters are needed. +However, at least one character in the subject must have been inspected. This +character need not form part of the final matched string; lookbehind assertions +and the \K escape sequence provide ways of inspecting characters before the +start of a matched substring. The requirement for inspecting at least one +character exists because an empty string can always be matched; without such a +restriction there would always be a partial match of an empty string at the end +of the subject.
If there are at least two slots in the offsets vector when a partial match is returned, the first slot is set to the offset of the earliest character that was inspected. For convenience, the second offset points to the end of the -subject so that a substring can easily be identified. +subject so that a substring can easily be identified. If there are at least +three slots in the offsets vector, the third slot is set to the offset of the +character where matching started.
-For the majority of patterns, the first offset identifies the start of the -partially matched string. However, for patterns that contain lookbehind -assertions, or \K, or begin with \b or \B, earlier characters have been -inspected while carrying out the match. For example: +For the majority of patterns, the contents of the first and third slots will be +the same. However, for patterns that contain lookbehind assertions, or begin +with \b or \B, characters before the one where matching started may have been +inspected while carrying out the match. For example, consider this pattern:
/(?<=abc)123/This pattern matches "123", but only if it is preceded by "abc". If the subject -string is "xyzabc12", the offsets after a partial match are for the substring -"abc12", because all these characters are needed if another match is tried -with extra characters added to the subject. +string is "xyzabc12", the first two offsets after a partial match are for the +substring "abc12", because all these characters were inspected. However, the +third offset is set to 6, because that is the offset where matching began.What happens when a partial match is identified depends on which of the two partial matching options are set.
-PCRE_PARTIAL_SOFT WITH pcre_exec() OR pcre16_exec() +PCRE_PARTIAL_SOFT WITH pcre_exec() OR pcre[16|32]_exec()
-If PCRE_PARTIAL_SOFT is set when pcre_exec() or pcre16_exec() +If PCRE_PARTIAL_SOFT is set when pcre_exec() or pcre[16|32]_exec() identifies a partial match, the partial match is remembered, but matching continues as normal, and other alternatives in the pattern are tried. If no complete match can be found, PCRE_ERROR_PARTIAL is returned instead of @@ -144,10 +147,10 @@ example, there are two partial matches, because "dog" matches the second alternative.)
-PCRE_PARTIAL_HARD WITH pcre_exec() OR pcre16_exec() +PCRE_PARTIAL_HARD WITH pcre_exec() OR pcre[16|32]_exec()
-If PCRE_PARTIAL_HARD is set for pcre_exec() or pcre16_exec(), +If PCRE_PARTIAL_HARD is set for pcre_exec() or pcre[16|32]_exec(), PCRE_ERROR_PARTIAL is returned as soon as a partial match is found, without continuing to search for possible complete matches. This option is "hard" because it prefers an earlier partial match over a later complete match. For @@ -192,7 +195,7 @@ to follow this explanation by thinking of the two patt The second pattern will never match "dogsbody", because it will always find the shorter match first.
-
PARTIAL MATCHING USING pcre_dfa_exec() OR pcre16_dfa_exec()
+
PARTIAL MATCHING USING pcre_dfa_exec() OR pcre[16|32]_dfa_exec()
The DFA functions move along the subject string character by character, without backtracking, searching for all possible matches simultaneously. If the end of @@ -280,7 +283,7 @@ if DFA matching is used. If the escape sequence \P is present more than once in a pcretest data line, the PCRE_PARTIAL_HARD option is set for the match.
-
MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre16_dfa_exec()
+
MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre[16|32]_dfa_exec()
When a partial match has been found using a DFA matching function, it is possible to continue the match by providing additional subject data and calling @@ -303,12 +306,22 @@ not retain the previously partially-matched string. It program to do that if it needs to.
+That means that, for an unanchored pattern, if a continued match fails, it is +not possible to try again at a new starting point. All this facility is capable +of doing is continuing with the previous match attempt. In the previous +example, if the second set of data is "ug23" the result is no match, even +though there would be a match for "aug23" if the entire string were given at +once. Depending on the application, this may or may not be what you want. +The only way to allow for starting again at the next character is to retain the +matched part of the subject and try a new complete match. +
+You can set the PCRE_PARTIAL_SOFT or PCRE_PARTIAL_HARD options with PCRE_DFA_RESTART to continue partial matching over multiple segments. This facility can be used to pass very long subject strings to the DFA matching functions.
-
MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre16_exec()
+
MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre[16|32]_exec()
From release 8.00, the standard matching functions can also be used to do multi-segment matching. Unlike the DFA functions, it is not possible to @@ -334,10 +347,9 @@ processing time is needed.
Note: If the pattern contains lookbehind assertions, or \K, or starts with \b or \B, the string that is returned for a partial match includes -characters that precede the partially matched string itself, because these must -be retained when adding on more characters for a subsequent matching attempt. -However, in some cases you may need to retain even earlier characters, as -discussed in the next section. +characters that precede the start of what would be returned for a complete +match, because it contains all the characters that were inspected during the +partial match.
ISSUES WITH MULTI-SEGMENT MATCHING
@@ -356,14 +368,37 @@ includes the effect of PCRE_NOTEOL. offsets that are returned for a partial match. However a lookbehind assertion later in the pattern could require even earlier characters to be inspected. You can handle this case by using the PCRE_INFO_MAXLOOKBEHIND option of the -pcre_fullinfo() or pcre16_fullinfo() functions to obtain the length -of the largest lookbehind in the pattern. This length is given in characters, -not bytes. If you always retain at least that many characters before the -partially matched string, all should be well. (Of course, near the start of the -subject, fewer characters may be present; in that case all characters should be -retained.) +pcre_fullinfo() or pcre[16|32]_fullinfo() functions to obtain the +length of the longest lookbehind in the pattern. This length is given in +characters, not bytes. If you always retain at least that many characters +before the partially matched string, all should be well. (Of course, near the +start of the subject, fewer characters may be present; in that case all +characters should be retained.)
+From release 8.33, there is a more accurate way of deciding which characters to +retain. Instead of subtracting the length of the longest lookbehind from the +earliest inspected character (offsets[0]), the match start position +(offsets[2]) should be used, and the next match attempt started at the +offsets[2] character by setting the startoffset argument of +pcre_exec() or pcre_dfa_exec(). +
++For example, if the pattern "(?<=123)abc" is partially +matched against the string "xx123a", the three offset values returned are 2, 6, +and 5. This indicates that the matching process that gave a partial match +started at offset 5, but the characters "123a" were all inspected. The maximum +lookbehind for that pattern is 3, so taking that away from 5 shows that we need +only keep "123a", and the next match attempt can be started at offset 3 (that +is, at "a") when further characters have been added. When the match start is +not the earliest inspected character, pcretest shows it explicitly: +
+ re> "(?<=123)abc" + data> xx123a\P\P + Partial match at offset 5: 123a ++ +3. Because a partial match must always contain at least one character, what might be considered a partial match of an empty string actually gives a "no match" result. For example: @@ -465,9 +500,9 @@ Cambridge CB2 3QH, England.
REVISION
-Last updated: 24 February 2012 +Last updated: 02 July 2013
-Copyright © 1997-2012 University of Cambridge. +Copyright © 1997-2013 University of Cambridge.
Return to the PCRE index page.