version 1.1.1.1, 2012/02/21 23:05:51
|
version 1.1.1.2, 2012/02/21 23:50:25
|
Line 34 The contents of this README file are:
|
Line 34 The contents of this README file are:
|
The PCRE APIs |
The PCRE APIs |
------------- |
------------- |
|
|
PCRE is written in C, and it has its own API. The distribution also includes a | PCRE is written in C, and it has its own API. There are two sets of functions, |
set of C++ wrapper functions (see the pcrecpp man page for details), courtesy | one for the 8-bit library, which processes strings of bytes, and one for the |
of Google Inc. | 16-bit library, which processes strings of 16-bit values. The distribution also |
| includes a set of C++ wrapper functions (see the pcrecpp man page for details), |
| courtesy of Google Inc., which can be used to call the 8-bit PCRE library from |
| C++. |
|
|
In addition, there is a set of C wrapper functions that are based on the POSIX | In addition, there is a set of C wrapper functions (again, just for the 8-bit |
regular expression API (see the pcreposix man page). These end up in the | library) that are based on the POSIX regular expression API (see the pcreposix |
library called libpcreposix. Note that this just provides a POSIX calling | man page). These end up in the library called libpcreposix. Note that this just |
interface to PCRE; the regular expressions themselves still follow Perl syntax | provides a POSIX calling interface to PCRE; the regular expressions themselves |
and semantics. The POSIX API is restricted, and does not give full access to | still follow Perl syntax and semantics. The POSIX API is restricted, and does |
all of PCRE's facilities. | not give full access to all of PCRE's facilities. |
|
|
The header file for the POSIX-style functions is called pcreposix.h. The |
The header file for the POSIX-style functions is called pcreposix.h. The |
official POSIX name is regex.h, but I did not want to risk possible problems |
official POSIX name is regex.h, but I did not want to risk possible problems |
Line 143 the usual methods of changing standard defaults are av
|
Line 146 the usual methods of changing standard defaults are av
|
|
|
CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local |
CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local |
|
|
specifies that the C compiler should be run with the flags '-O2 -Wall' instead | This command specifies that the C compiler should be run with the flags '-O2 |
of the default, and that "make install" should install PCRE under /opt/local | -Wall' instead of the default, and that "make install" should install PCRE |
instead of the default /usr/local. | under /opt/local instead of the default /usr/local. |
|
|
If you want to build in a different directory, just run "configure" with that |
If you want to build in a different directory, just run "configure" with that |
directory as current. For example, suppose you have unpacked the PCRE source |
directory as current. For example, suppose you have unpacked the PCRE source |
Line 169 library. They are also documented in the pcrebuild man
|
Line 172 library. They are also documented in the pcrebuild man
|
|
|
(See also "Shared libraries on Unix-like systems" below.) |
(See also "Shared libraries on Unix-like systems" below.) |
|
|
. If you want to suppress the building of the C++ wrapper library, you can add | . By default, only the 8-bit library is built. If you add --enable-pcre16 to |
--disable-cpp to the "configure" command. Otherwise, when "configure" is run, | the "configure" command, the 16-bit library is also built. If you want only |
it will try to find a C++ compiler and C++ header files, and if it succeeds, | the 16-bit library, use "./configure --enable-pcre16 --disable-pcre8". |
it will try to build the C++ wrapper. | |
|
|
|
. If you are building the 8-bit library and want to suppress the building of |
|
the C++ wrapper library, you can add --disable-cpp to the "configure" |
|
command. Otherwise, when "configure" is run without --disable-pcre8, it will |
|
try to find a C++ compiler and C++ header files, and if it succeeds, it will |
|
try to build the C++ wrapper. |
|
|
. If you want to include support for just-in-time compiling, which can give |
. If you want to include support for just-in-time compiling, which can give |
large performance improvements on certain platforms, add --enable-jit to the |
large performance improvements on certain platforms, add --enable-jit to the |
"configure" command. This support is available only for certain hardware |
"configure" command. This support is available only for certain hardware |
Line 184 library. They are also documented in the pcrebuild man
|
Line 192 library. They are also documented in the pcrebuild man
|
you add --disable-pcregrep-jit to the "configure" command. |
you add --disable-pcregrep-jit to the "configure" command. |
|
|
. If you want to make use of the support for UTF-8 Unicode character strings in |
. If you want to make use of the support for UTF-8 Unicode character strings in |
PCRE, you must add --enable-utf8 to the "configure" command. Without it, the | the 8-bit library, or UTF-16 Unicode character strings in the 16-bit library, |
code for handling UTF-8 is not included in the library. Even when included, | you must add --enable-utf to the "configure" command. Without it, the code |
it still has to be enabled by an option at run time. When PCRE is compiled | for handling UTF-8 and UTF-16 is not included in the relevant library. Even |
with this option, its input can only either be ASCII or UTF-8, even when | when --enable-utf is included, the use of a UTF encoding still has to be |
running on EBCDIC platforms. It is not possible to use both --enable-utf8 and | enabled by an option at run time. When PCRE is compiled with this option, its |
--enable-ebcdic at the same time. | input can only either be ASCII or UTF-8/16, even when running on EBCDIC |
| platforms. It is not possible to use both --enable-utf and --enable-ebcdic at |
| the same time. |
|
|
. If, in addition to support for UTF-8 character strings, you want to include | . There are no separate options for enabling UTF-8 and UTF-16 independently |
support for the \P, \p, and \X sequences that recognize Unicode character | because that would allow ridiculous settings such as requesting UTF-16 |
properties, you must add --enable-unicode-properties to the "configure" | support while building only the 8-bit library. However, the option |
command. This adds about 30K to the size of the library (in the form of a | --enable-utf8 is retained for backwards compatibility with earlier releases |
property table); only the basic two-letter properties such as Lu are | that did not support 16-bit character strings. It is synonymous with |
supported. | --enable-utf. It is not possible to configure one library with UTF support |
| and the other without in the same configuration. |
|
|
|
. If, in addition to support for UTF-8/16 character strings, you want to |
|
include support for the \P, \p, and \X sequences that recognize Unicode |
|
character properties, you must add --enable-unicode-properties to the |
|
"configure" command. This adds about 30K to the size of the library (in the |
|
form of a property table); only the basic two-letter properties such as Lu |
|
are supported. |
|
|
. You can build PCRE to recognize either CR or LF or the sequence CRLF or any |
. You can build PCRE to recognize either CR or LF or the sequence CRLF or any |
of the preceding, or any of the Unicode newline sequences as indicating the |
of the preceding, or any of the Unicode newline sequences as indicating the |
end of a line. Whatever you specify at build time is the default; the caller |
end of a line. Whatever you specify at build time is the default; the caller |
Line 249 library. They are also documented in the pcrebuild man
|
Line 267 library. They are also documented in the pcrebuild man
|
sizes in the pcrestack man page. |
sizes in the pcrestack man page. |
|
|
. The default maximum compiled pattern size is around 64K. You can increase |
. The default maximum compiled pattern size is around 64K. You can increase |
this by adding --with-link-size=3 to the "configure" command. You can | this by adding --with-link-size=3 to the "configure" command. In the 8-bit |
increase it even more by setting --with-link-size=4, but this is unlikely | library, PCRE then uses three bytes instead of two for offsets to different |
ever to be necessary. Increasing the internal link size will reduce | parts of the compiled pattern. In the 16-bit library, --with-link-size=3 is |
performance. | the same as --with-link-size=4, which (in both libraries) uses four-byte |
| offsets. Increasing the internal link size reduces performance. |
|
|
. You can build PCRE so that its internal match() function that is called from |
. You can build PCRE so that its internal match() function that is called from |
pcre_exec() does not call itself recursively. Instead, it uses memory blocks |
pcre_exec() does not call itself recursively. Instead, it uses memory blocks |
Line 287 library. They are also documented in the pcrebuild man
|
Line 306 library. They are also documented in the pcrebuild man
|
|
|
This automatically implies --enable-rebuild-chartables (see above). However, |
This automatically implies --enable-rebuild-chartables (see above). However, |
when PCRE is built this way, it always operates in EBCDIC. It cannot support |
when PCRE is built this way, it always operates in EBCDIC. It cannot support |
both EBCDIC and UTF-8. | both EBCDIC and UTF-8/16. |
|
|
. It is possible to compile pcregrep to use libz and/or libbz2, in order to | . The pcregrep program currently supports only 8-bit data files, and so |
read .gz and .bz2 files (respectively), by specifying one or both of | requires the 8-bit PCRE library. It is possible to compile pcregrep to use |
| libz and/or libbz2, in order to read .gz and .bz2 files (respectively), by |
| specifying one or both of |
|
|
--enable-pcregrep-libz |
--enable-pcregrep-libz |
--enable-pcregrep-libbz2 |
--enable-pcregrep-libbz2 |
Line 333 The "configure" script builds the following files for
|
Line 354 The "configure" script builds the following files for
|
. pcre-config script that shows the building settings such as CFLAGS |
. pcre-config script that shows the building settings such as CFLAGS |
that were set for "configure" |
that were set for "configure" |
. libpcre.pc ) data for the pkg-config command |
. libpcre.pc ) data for the pkg-config command |
|
. libpcre16.pc ) |
. libpcreposix.pc ) |
. libpcreposix.pc ) |
. libtool script that builds shared and/or static libraries |
. libtool script that builds shared and/or static libraries |
. RunTest script for running tests on the basic C library |
. RunTest script for running tests on the basic C library |
Line 343 names config.h.generic and pcre.h.generic. These are p
|
Line 365 names config.h.generic and pcre.h.generic. These are p
|
have to built PCRE without using "configure" or CMake. If you use "configure" |
have to built PCRE without using "configure" or CMake. If you use "configure" |
or CMake, the .generic versions are not used. |
or CMake, the .generic versions are not used. |
|
|
If a C++ compiler is found, the following files are also built: | When building the 8-bit library, if a C++ compiler is found, the following |
| files are also built: |
|
|
. libpcrecpp.pc data for the pkg-config command |
. libpcrecpp.pc data for the pkg-config command |
. pcrecpparg.h header file for calling PCRE via the C++ wrapper |
. pcrecpparg.h header file for calling PCRE via the C++ wrapper |
Line 353 The "configure" script also creates config.status, whi
|
Line 376 The "configure" script also creates config.status, whi
|
script that can be run to recreate the configuration, and config.log, which |
script that can be run to recreate the configuration, and config.log, which |
contains compiler output from tests that "configure" runs. |
contains compiler output from tests that "configure" runs. |
|
|
Once "configure" has run, you can run "make". It builds two libraries, called | Once "configure" has run, you can run "make". This builds either or both of the |
libpcre and libpcreposix, a test program called pcretest, and the pcregrep | libraries libpcre and libpcre16, and a test program called pcretest. If you |
command. If a C++ compiler was found on your system, and you did not disable it | enabled JIT support with --enable-jit, a test program called pcre_jit_test is |
with --disable-cpp, "make" also builds the C++ wrapper library, which is called | built as well. |
libpcrecpp, and some test programs called pcrecpp_unittest, | |
pcre_scanner_unittest, and pcre_stringpiece_unittest. If you enabled JIT | |
support with --enable-jit, a test program called pcre_jit_test is also built. | |
|
|
|
If the 8-bit library is built, libpcreposix and the pcregrep command are also |
|
built, and if a C++ compiler was found on your system, and you did not disable |
|
it with --disable-cpp, "make" builds the C++ wrapper library, which is called |
|
libpcrecpp, as well as some test programs called pcrecpp_unittest, |
|
pcre_scanner_unittest, and pcre_stringpiece_unittest. |
|
|
The command "make check" runs all the appropriate tests. Details of the PCRE |
The command "make check" runs all the appropriate tests. Details of the PCRE |
tests are given below in a separate section of this document. |
tests are given below in a separate section of this document. |
|
|
Line 370 system. The following are installed (file names are al
|
Line 396 system. The following are installed (file names are al
|
|
|
Commands (bin): |
Commands (bin): |
pcretest |
pcretest |
pcregrep | pcregrep (if 8-bit support is enabled) |
pcre-config |
pcre-config |
|
|
Libraries (lib): |
Libraries (lib): |
libpcre | libpcre16 (if 16-bit support is enabled) |
libpcreposix | libpcre (if 8-bit support is enabled) |
libpcrecpp (if C++ support is enabled) | libpcreposix (if 8-bit support is enabled) |
| libpcrecpp (if 8-bit and C++ support is enabled) |
|
|
Configuration information (lib/pkgconfig): |
Configuration information (lib/pkgconfig): |
|
libpcre16.pc |
libpcre.pc |
libpcre.pc |
libpcreposix.pc |
libpcreposix.pc |
libpcrecpp.pc (if C++ support is enabled) |
libpcrecpp.pc (if C++ support is enabled) |
Line 558 The RunTest script runs the pcretest test program (whi
|
Line 586 The RunTest script runs the pcretest test program (whi
|
own man page) on each of the relevant testinput files in the testdata |
own man page) on each of the relevant testinput files in the testdata |
directory, and compares the output with the contents of the corresponding |
directory, and compares the output with the contents of the corresponding |
testoutput files. Some tests are relevant only when certain build-time options |
testoutput files. Some tests are relevant only when certain build-time options |
were selected. For example, the tests for UTF-8 support are run only if | were selected. For example, the tests for UTF-8/16 support are run only if |
--enable-utf8 was used. RunTest outputs a comment when it skips a test. | --enable-utf was used. RunTest outputs a comment when it skips a test. |
|
|
Many of the tests that are not skipped are run up to three times. The second |
Many of the tests that are not skipped are run up to three times. The second |
run forces pcre_study() to be called for all patterns except for a few in some |
run forces pcre_study() to be called for all patterns except for a few in some |
Line 567 tests that are marked "never study" (see the pcretest
|
Line 595 tests that are marked "never study" (see the pcretest
|
done). If JIT support is available, the non-DFA tests are run a third time, |
done). If JIT support is available, the non-DFA tests are run a third time, |
this time with a forced pcre_study() with the PCRE_STUDY_JIT_COMPILE option. |
this time with a forced pcre_study() with the PCRE_STUDY_JIT_COMPILE option. |
|
|
RunTest uses a file called testtry to hold the main output from pcretest | When both 8-bit and 16-bit support is enabled, the entire set of tests is run |
(testsavedregex is also used as a working file). To run pcretest on just one of | twice, once for each library. If you want to run just one set of tests, call |
the test files, give its number as an argument to RunTest, for example: | RunTest with either the -8 or -16 option. |
|
|
RunTest 2 | RunTest uses a file called testtry to hold the main output from pcretest. |
| Other files whose names begin with "test" are used as working files in some |
| tests. To run pcretest on just one or more specific test files, give their |
| numbers as arguments to RunTest, for example: |
|
|
|
RunTest 2 7 11 |
|
|
The first test file can be fed directly into the perltest.pl script to check |
The first test file can be fed directly into the perltest.pl script to check |
that Perl gives the same results. The only difference you should see is in the |
that Perl gives the same results. The only difference you should see is in the |
first few lines, where the Perl version is given instead of the PCRE version. |
first few lines, where the Perl version is given instead of the PCRE version. |
|
|
The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(), | The second set of tests check pcre_fullinfo(), pcre_study(), |
pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error |
pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error |
detection, and run-time flags that are specific to PCRE, as well as the POSIX |
detection, and run-time flags that are specific to PCRE, as well as the POSIX |
wrapper API. It also uses the debugging flags to check some of the internals of |
wrapper API. It also uses the debugging flags to check some of the internals of |
Line 612 RunTest.bat. The version of RunTest.bat included with
|
Line 645 RunTest.bat. The version of RunTest.bat included with
|
Windows versions of test 2. More info on using RunTest.bat is included in the |
Windows versions of test 2. More info on using RunTest.bat is included in the |
document entitled NON-UNIX-USE.] |
document entitled NON-UNIX-USE.] |
|
|
The fourth test checks the UTF-8 support. This file can be also fed directly to | The fourth and fifth tests check the UTF-8/16 support and error handling and |
the perltest.pl script, provided you are running Perl 5.8 or higher. | internal UTF features of PCRE that are not relevant to Perl, respectively. The |
| sixth and seventh tests do the same for Unicode character properties support. |
|
|
The fifth test checks error handling with UTF-8 encoding, and internal UTF-8 | The eighth, ninth, and tenth tests check the pcre_dfa_exec() alternative |
features of PCRE that are not relevant to Perl. | matching function, in non-UTF-8/16 mode, UTF-8/16 mode, and UTF-8/16 mode with |
| Unicode property support, respectively. |
|
|
The sixth test (which is Perl-5.10 compatible) checks the support for Unicode | The eleventh test checks some internal offsets and code size features; it is |
character properties. This file can be also fed directly to the perltest.pl | run only when the default "link size" of 2 is set (in other cases the sizes |
script, provided you are running Perl 5.10 or higher. | |
| |
The seventh, eighth, and ninth tests check the pcre_dfa_exec() alternative | |
matching function, in non-UTF-8 mode, UTF-8 mode, and UTF-8 mode with Unicode | |
property support, respectively. | |
| |
The tenth test checks some internal offsets and code size features; it is run | |
only when the default "link size" of 2 is set (in other cases the sizes | |
change) and when Unicode property support is enabled. |
change) and when Unicode property support is enabled. |
|
|
The eleventh and twelfth tests check out features that are new in Perl 5.10, | The twelfth test is run only when JIT support is available, and the thirteenth |
without and with UTF-8 support, respectively. This file can be also fed | test is run only when JIT support is not available. They test some JIT-specific |
directly to the perltest.pl script, provided you are running Perl 5.10 or | features such as information output from pcretest about JIT compilation. |
higher. | |
|
|
The thirteenth test checks a number internals and non-Perl features concerned | The fourteenth, fifteenth, and sixteenth tests are run only in 8-bit mode, and |
with Unicode property support. | the seventeenth, eighteenth, and nineteenth tests are run only in 16-bit mode. |
| These are tests that generate different output in the two modes. They are for |
| general cases, UTF-8/16 support, and Unicode property support, respectively. |
|
|
The fourteenth test is run only when JIT support is available, and the | The twentieth test is run only in 16-bit mode. It tests some specific 16-bit |
fifteenth test is run only when JIT support is not available. They test some | features of the DFA matching engine. |
JIT-specific features such as information output from pcretest about JIT | |
compilation. | |
|
|
|
The twenty-first and twenty-second tests are run only in 16-bit mode, when the |
|
link size is set to 2. They test reloading pre-compiled patterns. |
|
|
|
|
Character tables |
Character tables |
---------------- |
---------------- |
|
|
Line 701 will cause PCRE to malfunction.
|
Line 730 will cause PCRE to malfunction.
|
File manifest |
File manifest |
------------- |
------------- |
|
|
The distribution should contain the following files: | The distribution should contain the files listed below. Where a file name is |
| given as pcre[16]_xxx it means that there are two files, one with the name |
| pcre_xxx and the other with the name pcre16_xxx. |
|
|
(A) Source files of the PCRE library functions and their headers: |
(A) Source files of the PCRE library functions and their headers: |
|
|
Line 710 The distribution should contain the following files:
|
Line 741 The distribution should contain the following files:
|
|
|
pcre_chartables.c.dist a default set of character tables that assume ASCII |
pcre_chartables.c.dist a default set of character tables that assume ASCII |
coding; used, unless --enable-rebuild-chartables is |
coding; used, unless --enable-rebuild-chartables is |
specified, by copying to pcre_chartables.c | specified, by copying to pcre[16]_chartables.c |
|
|
pcreposix.c ) |
pcreposix.c ) |
pcre_compile.c ) | pcre[16]_byte_order.c ) |
pcre_config.c ) | pcre[16]_compile.c ) |
pcre_dfa_exec.c ) | pcre[16]_config.c ) |
pcre_exec.c ) | pcre[16]_dfa_exec.c ) |
pcre_fullinfo.c ) | pcre[16]_exec.c ) |
pcre_get.c ) sources for the functions in the library, | pcre[16]_fullinfo.c ) |
pcre_globals.c ) and some internal functions that they use | pcre[16]_get.c ) sources for the functions in the library, |
pcre_info.c ) | pcre[16]_globals.c ) and some internal functions that they use |
pcre_jit_compile.c ) | pcre[16]_jit_compile.c ) |
pcre_maketables.c ) | pcre[16]_maketables.c ) |
pcre_newline.c ) | pcre[16]_newline.c ) |
| pcre[16]_refcount.c ) |
| pcre[16]_string_utils.c ) |
| pcre[16]_study.c ) |
| pcre[16]_tables.c ) |
| pcre[16]_ucd.c ) |
| pcre[16]_version.c ) |
| pcre[16]_xclass.c ) |
pcre_ord2utf8.c ) |
pcre_ord2utf8.c ) |
pcre_refcount.c ) |
|
pcre_study.c ) |
|
pcre_tables.c ) |
|
pcre_try_flipped.c ) |
|
pcre_ucd.c ) |
|
pcre_valid_utf8.c ) |
pcre_valid_utf8.c ) |
pcre_version.c ) | pcre16_ord2utf16.c ) |
pcre_xclass.c ) | pcre16_utf16_utils.c ) |
pcre_printint.src ) debugging function that is #included in pcretest, | pcre16_valid_utf16.c ) |
| |
| pcre[16]_printint.c ) debugging function that is used by pcretest, |
) and can also be #included in pcre_compile() |
) and can also be #included in pcre_compile() |
|
|
pcre.h.in template for pcre.h when built by "configure" |
pcre.h.in template for pcre.h when built by "configure" |
pcreposix.h header for the external POSIX wrapper API |
pcreposix.h header for the external POSIX wrapper API |
pcre_internal.h header for internal use |
pcre_internal.h header for internal use |
Line 796 The distribution should contain the following files:
|
Line 832 The distribution should contain the following files:
|
doc/pcretest.txt plain text documentation of test program |
doc/pcretest.txt plain text documentation of test program |
doc/perltest.txt plain text documentation of Perl test program |
doc/perltest.txt plain text documentation of Perl test program |
install-sh a shell script for installing files |
install-sh a shell script for installing files |
|
libpcre16.pc.in template for libpcre16.pc for pkg-config |
libpcre.pc.in template for libpcre.pc for pkg-config |
libpcre.pc.in template for libpcre.pc for pkg-config |
libpcreposix.pc.in template for libpcreposix.pc for pkg-config |
libpcreposix.pc.in template for libpcreposix.pc for pkg-config |
libpcrecpp.pc.in template for libpcrecpp.pc for pkg-config |
libpcrecpp.pc.in template for libpcrecpp.pc for pkg-config |
Line 812 The distribution should contain the following files:
|
Line 849 The distribution should contain the following files:
|
testdata/testinput* test data for main library tests |
testdata/testinput* test data for main library tests |
testdata/testoutput* expected test results |
testdata/testoutput* expected test results |
testdata/grep* input and output for pcregrep tests |
testdata/grep* input and output for pcregrep tests |
|
testdata/* other supporting test files |
|
|
(D) Auxiliary files for cmake support |
(D) Auxiliary files for cmake support |
|
|
Line 842 The distribution should contain the following files:
|
Line 880 The distribution should contain the following files:
|
Philip Hazel |
Philip Hazel |
Email local part: ph10 |
Email local part: ph10 |
Email domain: cam.ac.uk |
Email domain: cam.ac.uk |
Last updated: 06 September 2011 | Last updated: 20 January 2012 |