1: README file for PCRE (Perl-compatible regular expression library)
2: -----------------------------------------------------------------
3:
4: The latest release of PCRE is always available in three alternative formats
5: from:
6:
7: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz
8: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.bz2
9: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.zip
10:
11: There is a mailing list for discussion about the development of PCRE at
12:
13: pcre-dev@exim.org
14:
15: Please read the NEWS file if you are upgrading from a previous release.
16: The contents of this README file are:
17:
18: The PCRE APIs
19: Documentation for PCRE
20: Contributions by users of PCRE
21: Building PCRE on non-Unix systems
22: Building PCRE on Unix-like systems
23: Retrieving configuration information on Unix-like systems
24: Shared libraries on Unix-like systems
25: Cross-compiling on Unix-like systems
26: Using HP's ANSI C++ compiler (aCC)
27: Using PCRE from MySQL
28: Making new tarballs
29: Testing PCRE
30: Character tables
31: File manifest
32:
33:
34: The PCRE APIs
35: -------------
36:
37: PCRE is written in C, and it has its own API. The distribution also includes a
38: set of C++ wrapper functions (see the pcrecpp man page for details), courtesy
39: of Google Inc.
40:
41: In addition, there is a set of C wrapper functions that are based on the POSIX
42: regular expression API (see the pcreposix man page). These end up in the
43: library called libpcreposix. Note that this just provides a POSIX calling
44: interface to PCRE; the regular expressions themselves still follow Perl syntax
45: and semantics. The POSIX API is restricted, and does not give full access to
46: all of PCRE's facilities.
47:
48: The header file for the POSIX-style functions is called pcreposix.h. The
49: official POSIX name is regex.h, but I did not want to risk possible problems
50: with existing files of that name by distributing it that way. To use PCRE with
51: an existing program that uses the POSIX API, pcreposix.h will have to be
52: renamed or pointed at by a link.
53:
54: If you are using the POSIX interface to PCRE and there is already a POSIX regex
55: library installed on your system, as well as worrying about the regex.h header
56: file (as mentioned above), you must also take care when linking programs to
57: ensure that they link with PCRE's libpcreposix library. Otherwise they may pick
58: up the POSIX functions of the same name from the other library.
59:
60: One way of avoiding this confusion is to compile PCRE with the addition of
61: -Dregcomp=PCREregcomp (and similarly for the other POSIX functions) to the
62: compiler flags (CFLAGS if you are using "configure" -- see below). This has the
63: effect of renaming the functions so that the names no longer clash. Of course,
64: you have to do the same thing for your applications, or write them using the
65: new names.
66:
67:
68: Documentation for PCRE
69: ----------------------
70:
71: If you install PCRE in the normal way on a Unix-like system, you will end up
72: with a set of man pages whose names all start with "pcre". The one that is just
73: called "pcre" lists all the others. In addition to these man pages, the PCRE
74: documentation is supplied in two other forms:
75:
76: 1. There are files called doc/pcre.txt, doc/pcregrep.txt, and
77: doc/pcretest.txt in the source distribution. The first of these is a
78: concatenation of the text forms of all the section 3 man pages except
79: those that summarize individual functions. The other two are the text
80: forms of the section 1 man pages for the pcregrep and pcretest commands.
81: These text forms are provided for ease of scanning with text editors or
82: similar tools. They are installed in <prefix>/share/doc/pcre, where
83: <prefix> is the installation prefix (defaulting to /usr/local).
84:
85: 2. A set of files containing all the documentation in HTML form, hyperlinked
86: in various ways, and rooted in a file called index.html, is distributed in
87: doc/html and installed in <prefix>/share/doc/pcre/html.
88:
89: Users of PCRE have contributed files containing the documentation for various
90: releases in CHM format. These can be found in the Contrib directory of the FTP
91: site (see next section).
92:
93:
94: Contributions by users of PCRE
95: ------------------------------
96:
97: You can find contributions from PCRE users in the directory
98:
99: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib
100:
101: There is a README file giving brief descriptions of what they are. Some are
102: complete in themselves; others are pointers to URLs containing relevant files.
103: Some of this material is likely to be well out-of-date. Several of the earlier
104: contributions provided support for compiling PCRE on various flavours of
105: Windows (I myself do not use Windows). Nowadays there is more Windows support
106: in the standard distribution, so these contibutions have been archived.
107:
108:
109: Building PCRE on non-Unix systems
110: ---------------------------------
111:
112: For a non-Unix system, please read the comments in the file NON-UNIX-USE,
113: though if your system supports the use of "configure" and "make" you may be
114: able to build PCRE in the same way as for Unix-like systems. PCRE can also be
115: configured in many platform environments using the GUI facility provided by
116: CMake's cmake-gui command. This creates Makefiles, solution files, etc.
117:
118: PCRE has been compiled on many different operating systems. It should be
119: straightforward to build PCRE on any system that has a Standard C compiler and
120: library, because it uses only Standard C functions.
121:
122:
123: Building PCRE on Unix-like systems
124: ----------------------------------
125:
126: If you are using HP's ANSI C++ compiler (aCC), please see the special note
127: in the section entitled "Using HP's ANSI C++ compiler (aCC)" below.
128:
129: The following instructions assume the use of the widely used "configure, make,
130: make install" process. There is also support for CMake in the PCRE
131: distribution; there are some comments about using CMake in the NON-UNIX-USE
132: file, though it can also be used in Unix-like systems.
133:
134: To build PCRE on a Unix-like system, first run the "configure" command from the
135: PCRE distribution directory, with your current directory set to the directory
136: where you want the files to be created. This command is a standard GNU
137: "autoconf" configuration script, for which generic instructions are supplied in
138: the file INSTALL.
139:
140: Most commonly, people build PCRE within its own distribution directory, and in
141: this case, on many systems, just running "./configure" is sufficient. However,
142: the usual methods of changing standard defaults are available. For example:
143:
144: CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local
145:
146: specifies that the C compiler should be run with the flags '-O2 -Wall' instead
147: of the default, and that "make install" should install PCRE under /opt/local
148: instead of the default /usr/local.
149:
150: If you want to build in a different directory, just run "configure" with that
151: directory as current. For example, suppose you have unpacked the PCRE source
152: into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx:
153:
154: cd /build/pcre/pcre-xxx
155: /source/pcre/pcre-xxx/configure
156:
157: PCRE is written in C and is normally compiled as a C library. However, it is
158: possible to build it as a C++ library, though the provided building apparatus
159: does not have any features to support this.
160:
161: There are some optional features that can be included or omitted from the PCRE
162: library. They are also documented in the pcrebuild man page.
163:
164: . By default, both shared and static libraries are built. You can change this
165: by adding one of these options to the "configure" command:
166:
167: --disable-shared
168: --disable-static
169:
170: (See also "Shared libraries on Unix-like systems" below.)
171:
172: . If you want to suppress the building of the C++ wrapper library, you can add
173: --disable-cpp to the "configure" command. Otherwise, when "configure" is run,
174: it will try to find a C++ compiler and C++ header files, and if it succeeds,
175: it will try to build the C++ wrapper.
176:
177: . If you want to include support for just-in-time compiling, which can give
178: large performance improvements on certain platforms, add --enable-jit to the
179: "configure" command. This support is available only for certain hardware
180: architectures. If you try to enable it on an unsupported architecture, there
181: will be a compile time error.
182:
183: . When JIT support is enabled, pcregrep automatically makes use of it, unless
184: you add --disable-pcregrep-jit to the "configure" command.
185:
186: . If you want to make use of the support for UTF-8 Unicode character strings in
187: PCRE, you must add --enable-utf8 to the "configure" command. Without it, the
188: code for handling UTF-8 is not included in the library. Even when included,
189: it still has to be enabled by an option at run time. When PCRE is compiled
190: with this option, its input can only either be ASCII or UTF-8, even when
191: running on EBCDIC platforms. It is not possible to use both --enable-utf8 and
192: --enable-ebcdic at the same time.
193:
194: . If, in addition to support for UTF-8 character strings, you want to include
195: support for the \P, \p, and \X sequences that recognize Unicode character
196: properties, you must add --enable-unicode-properties to the "configure"
197: command. This adds about 30K to the size of the library (in the form of a
198: property table); only the basic two-letter properties such as Lu are
199: supported.
200:
201: . You can build PCRE to recognize either CR or LF or the sequence CRLF or any
202: of the preceding, or any of the Unicode newline sequences as indicating the
203: end of a line. Whatever you specify at build time is the default; the caller
204: of PCRE can change the selection at run time. The default newline indicator
205: is a single LF character (the Unix standard). You can specify the default
206: newline indicator by adding --enable-newline-is-cr or --enable-newline-is-lf
207: or --enable-newline-is-crlf or --enable-newline-is-anycrlf or
208: --enable-newline-is-any to the "configure" command, respectively.
209:
210: If you specify --enable-newline-is-cr or --enable-newline-is-crlf, some of
211: the standard tests will fail, because the lines in the test files end with
212: LF. Even if the files are edited to change the line endings, there are likely
213: to be some failures. With --enable-newline-is-anycrlf or
214: --enable-newline-is-any, many tests should succeed, but there may be some
215: failures.
216:
217: . By default, the sequence \R in a pattern matches any Unicode line ending
218: sequence. This is independent of the option specifying what PCRE considers to
219: be the end of a line (see above). However, the caller of PCRE can restrict \R
220: to match only CR, LF, or CRLF. You can make this the default by adding
221: --enable-bsr-anycrlf to the "configure" command (bsr = "backslash R").
222:
223: . When called via the POSIX interface, PCRE uses malloc() to get additional
224: storage for processing capturing parentheses if there are more than 10 of
225: them in a pattern. You can increase this threshold by setting, for example,
226:
227: --with-posix-malloc-threshold=20
228:
229: on the "configure" command.
230:
231: . PCRE has a counter that can be set to limit the amount of resources it uses.
232: If the limit is exceeded during a match, the match fails. The default is ten
233: million. You can change the default by setting, for example,
234:
235: --with-match-limit=500000
236:
237: on the "configure" command. This is just the default; individual calls to
238: pcre_exec() can supply their own value. There is more discussion on the
239: pcreapi man page.
240:
241: . There is a separate counter that limits the depth of recursive function calls
242: during a matching process. This also has a default of ten million, which is
243: essentially "unlimited". You can change the default by setting, for example,
244:
245: --with-match-limit-recursion=500000
246:
247: Recursive function calls use up the runtime stack; running out of stack can
248: cause programs to crash in strange ways. There is a discussion about stack
249: sizes in the pcrestack man page.
250:
251: . The default maximum compiled pattern size is around 64K. You can increase
252: this by adding --with-link-size=3 to the "configure" command. You can
253: increase it even more by setting --with-link-size=4, but this is unlikely
254: ever to be necessary. Increasing the internal link size will reduce
255: performance.
256:
257: . You can build PCRE so that its internal match() function that is called from
258: pcre_exec() does not call itself recursively. Instead, it uses memory blocks
259: obtained from the heap via the special functions pcre_stack_malloc() and
260: pcre_stack_free() to save data that would otherwise be saved on the stack. To
261: build PCRE like this, use
262:
263: --disable-stack-for-recursion
264:
265: on the "configure" command. PCRE runs more slowly in this mode, but it may be
266: necessary in environments with limited stack sizes. This applies only to the
267: normal execution of the pcre_exec() function; if JIT support is being
268: successfully used, it is not relevant. Equally, it does not apply to
269: pcre_dfa_exec(), which does not use deeply nested recursion. There is a
270: discussion about stack sizes in the pcrestack man page.
271:
272: . For speed, PCRE uses four tables for manipulating and identifying characters
273: whose code point values are less than 256. By default, it uses a set of
274: tables for ASCII encoding that is part of the distribution. If you specify
275:
276: --enable-rebuild-chartables
277:
278: a program called dftables is compiled and run in the default C locale when
279: you obey "make". It builds a source file called pcre_chartables.c. If you do
280: not specify this option, pcre_chartables.c is created as a copy of
281: pcre_chartables.c.dist. See "Character tables" below for further information.
282:
283: . It is possible to compile PCRE for use on systems that use EBCDIC as their
284: character code (as opposed to ASCII) by specifying
285:
286: --enable-ebcdic
287:
288: This automatically implies --enable-rebuild-chartables (see above). However,
289: when PCRE is built this way, it always operates in EBCDIC. It cannot support
290: both EBCDIC and UTF-8.
291:
292: . It is possible to compile pcregrep to use libz and/or libbz2, in order to
293: read .gz and .bz2 files (respectively), by specifying one or both of
294:
295: --enable-pcregrep-libz
296: --enable-pcregrep-libbz2
297:
298: Of course, the relevant libraries must be installed on your system.
299:
300: . The default size of internal buffer used by pcregrep can be set by, for
301: example:
302:
303: --with-pcregrep-bufsize=50K
304:
305: The default value is 20K.
306:
307: . It is possible to compile pcretest so that it links with the libreadline
308: library, by specifying
309:
310: --enable-pcretest-libreadline
311:
312: If this is done, when pcretest's input is from a terminal, it reads it using
313: the readline() function. This provides line-editing and history facilities.
314: Note that libreadline is GPL-licenced, so if you distribute a binary of
315: pcretest linked in this way, there may be licensing issues.
316:
317: Setting this option causes the -lreadline option to be added to the pcretest
318: build. In many operating environments with a sytem-installed readline
319: library this is sufficient. However, in some environments (e.g. if an
320: unmodified distribution version of readline is in use), it may be necessary
321: to specify something like LIBS="-lncurses" as well. This is because, to quote
322: the readline INSTALL, "Readline uses the termcap functions, but does not link
323: with the termcap or curses library itself, allowing applications which link
324: with readline the to choose an appropriate library." If you get error
325: messages about missing functions tgetstr, tgetent, tputs, tgetflag, or tgoto,
326: this is the problem, and linking with the ncurses library should fix it.
327:
328: The "configure" script builds the following files for the basic C library:
329:
330: . Makefile the makefile that builds the library
331: . config.h build-time configuration options for the library
332: . pcre.h the public PCRE header file
333: . pcre-config script that shows the building settings such as CFLAGS
334: that were set for "configure"
335: . libpcre.pc ) data for the pkg-config command
336: . libpcreposix.pc )
337: . libtool script that builds shared and/or static libraries
338: . RunTest script for running tests on the basic C library
339: . RunGrepTest script for running tests on the pcregrep command
340:
341: Versions of config.h and pcre.h are distributed in the PCRE tarballs under the
342: names config.h.generic and pcre.h.generic. These are provided for those who
343: have to built PCRE without using "configure" or CMake. If you use "configure"
344: or CMake, the .generic versions are not used.
345:
346: If a C++ compiler is found, the following files are also built:
347:
348: . libpcrecpp.pc data for the pkg-config command
349: . pcrecpparg.h header file for calling PCRE via the C++ wrapper
350: . pcre_stringpiece.h header for the C++ "stringpiece" functions
351:
352: The "configure" script also creates config.status, which is an executable
353: script that can be run to recreate the configuration, and config.log, which
354: contains compiler output from tests that "configure" runs.
355:
356: Once "configure" has run, you can run "make". It builds two libraries, called
357: libpcre and libpcreposix, a test program called pcretest, and the pcregrep
358: command. If a C++ compiler was found on your system, and you did not disable it
359: with --disable-cpp, "make" also builds the C++ wrapper library, which is called
360: libpcrecpp, and some test programs called pcrecpp_unittest,
361: pcre_scanner_unittest, and pcre_stringpiece_unittest. If you enabled JIT
362: support with --enable-jit, a test program called pcre_jit_test is also built.
363:
364: The command "make check" runs all the appropriate tests. Details of the PCRE
365: tests are given below in a separate section of this document.
366:
367: You can use "make install" to install PCRE into live directories on your
368: system. The following are installed (file names are all relative to the
369: <prefix> that is set when "configure" is run):
370:
371: Commands (bin):
372: pcretest
373: pcregrep
374: pcre-config
375:
376: Libraries (lib):
377: libpcre
378: libpcreposix
379: libpcrecpp (if C++ support is enabled)
380:
381: Configuration information (lib/pkgconfig):
382: libpcre.pc
383: libpcreposix.pc
384: libpcrecpp.pc (if C++ support is enabled)
385:
386: Header files (include):
387: pcre.h
388: pcreposix.h
389: pcre_scanner.h )
390: pcre_stringpiece.h ) if C++ support is enabled
391: pcrecpp.h )
392: pcrecpparg.h )
393:
394: Man pages (share/man/man{1,3}):
395: pcregrep.1
396: pcretest.1
397: pcre-config.1
398: pcre.3
399: pcre*.3 (lots more pages, all starting "pcre")
400:
401: HTML documentation (share/doc/pcre/html):
402: index.html
403: *.html (lots more pages, hyperlinked from index.html)
404:
405: Text file documentation (share/doc/pcre):
406: AUTHORS
407: COPYING
408: ChangeLog
409: LICENCE
410: NEWS
411: README
412: pcre.txt (a concatenation of the man(3) pages)
413: pcretest.txt the pcretest man page
414: pcregrep.txt the pcregrep man page
415: pcre-config.txt the pcre-config man page
416:
417: If you want to remove PCRE from your system, you can run "make uninstall".
418: This removes all the files that "make install" installed. However, it does not
419: remove any directories, because these are often shared with other programs.
420:
421:
422: Retrieving configuration information on Unix-like systems
423: ---------------------------------------------------------
424:
425: Running "make install" installs the command pcre-config, which can be used to
426: recall information about the PCRE configuration and installation. For example:
427:
428: pcre-config --version
429:
430: prints the version number, and
431:
432: pcre-config --libs
433:
434: outputs information about where the library is installed. This command can be
435: included in makefiles for programs that use PCRE, saving the programmer from
436: having to remember too many details.
437:
438: The pkg-config command is another system for saving and retrieving information
439: about installed libraries. Instead of separate commands for each library, a
440: single command is used. For example:
441:
442: pkg-config --cflags pcre
443:
444: The data is held in *.pc files that are installed in a directory called
445: <prefix>/lib/pkgconfig.
446:
447:
448: Shared libraries on Unix-like systems
449: -------------------------------------
450:
451: The default distribution builds PCRE as shared libraries and static libraries,
452: as long as the operating system supports shared libraries. Shared library
453: support relies on the "libtool" script which is built as part of the
454: "configure" process.
455:
456: The libtool script is used to compile and link both shared and static
457: libraries. They are placed in a subdirectory called .libs when they are newly
458: built. The programs pcretest and pcregrep are built to use these uninstalled
459: libraries (by means of wrapper scripts in the case of shared libraries). When
460: you use "make install" to install shared libraries, pcregrep and pcretest are
461: automatically re-built to use the newly installed shared libraries before being
462: installed themselves. However, the versions left in the build directory still
463: use the uninstalled libraries.
464:
465: To build PCRE using static libraries only you must use --disable-shared when
466: configuring it. For example:
467:
468: ./configure --prefix=/usr/gnu --disable-shared
469:
470: Then run "make" in the usual way. Similarly, you can use --disable-static to
471: build only shared libraries.
472:
473:
474: Cross-compiling on Unix-like systems
475: ------------------------------------
476:
477: You can specify CC and CFLAGS in the normal way to the "configure" command, in
478: order to cross-compile PCRE for some other host. However, you should NOT
479: specify --enable-rebuild-chartables, because if you do, the dftables.c source
480: file is compiled and run on the local host, in order to generate the inbuilt
481: character tables (the pcre_chartables.c file). This will probably not work,
482: because dftables.c needs to be compiled with the local compiler, not the cross
483: compiler.
484:
485: When --enable-rebuild-chartables is not specified, pcre_chartables.c is created
486: by making a copy of pcre_chartables.c.dist, which is a default set of tables
487: that assumes ASCII code. Cross-compiling with the default tables should not be
488: a problem.
489:
490: If you need to modify the character tables when cross-compiling, you should
491: move pcre_chartables.c.dist out of the way, then compile dftables.c by hand and
492: run it on the local host to make a new version of pcre_chartables.c.dist.
493: Then when you cross-compile PCRE this new version of the tables will be used.
494:
495:
496: Using HP's ANSI C++ compiler (aCC)
497: ----------------------------------
498:
499: Unless C++ support is disabled by specifying the "--disable-cpp" option of the
500: "configure" script, you must include the "-AA" option in the CXXFLAGS
501: environment variable in order for the C++ components to compile correctly.
502:
503: Also, note that the aCC compiler on PA-RISC platforms may have a defect whereby
504: needed libraries fail to get included when specifying the "-AA" compiler
505: option. If you experience unresolved symbols when linking the C++ programs,
506: use the workaround of specifying the following environment variable prior to
507: running the "configure" script:
508:
509: CXXLDFLAGS="-lstd_v2 -lCsup_v2"
510:
511:
512: Using Sun's compilers for Solaris
513: ---------------------------------
514:
515: A user reports that the following configurations work on Solaris 9 sparcv9 and
516: Solaris 9 x86 (32-bit):
517:
518: Solaris 9 sparcv9: ./configure --disable-cpp CC=/bin/cc CFLAGS="-m64 -g"
519: Solaris 9 x86: ./configure --disable-cpp CC=/bin/cc CFLAGS="-g"
520:
521:
522: Using PCRE from MySQL
523: ---------------------
524:
525: On systems where both PCRE and MySQL are installed, it is possible to make use
526: of PCRE from within MySQL, as an alternative to the built-in pattern matching.
527: There is a web page that tells you how to do this:
528:
529: http://www.mysqludf.org/lib_mysqludf_preg/index.php
530:
531:
532: Making new tarballs
533: -------------------
534:
535: The command "make dist" creates three PCRE tarballs, in tar.gz, tar.bz2, and
536: zip formats. The command "make distcheck" does the same, but then does a trial
537: build of the new distribution to ensure that it works.
538:
539: If you have modified any of the man page sources in the doc directory, you
540: should first run the PrepareRelease script before making a distribution. This
541: script creates the .txt and HTML forms of the documentation from the man pages.
542:
543:
544: Testing PCRE
545: ------------
546:
547: To test the basic PCRE library on a Unix system, run the RunTest script that is
548: created by the configuring process. There is also a script called RunGrepTest
549: that tests the options of the pcregrep command. If the C++ wrapper library is
550: built, three test programs called pcrecpp_unittest, pcre_scanner_unittest, and
551: pcre_stringpiece_unittest are also built. When JIT support is enabled, another
552: test program called pcre_jit_test is built.
553:
554: Both the scripts and all the program tests are run if you obey "make check" or
555: "make test". For other systems, see the instructions in NON-UNIX-USE.
556:
557: The RunTest script runs the pcretest test program (which is documented in its
558: own man page) on each of the relevant testinput files in the testdata
559: directory, and compares the output with the contents of the corresponding
560: testoutput files. Some tests are relevant only when certain build-time options
561: were selected. For example, the tests for UTF-8 support are run only if
562: --enable-utf8 was used. RunTest outputs a comment when it skips a test.
563:
564: Many of the tests that are not skipped are run up to three times. The second
565: run forces pcre_study() to be called for all patterns except for a few in some
566: tests that are marked "never study" (see the pcretest program for how this is
567: done). If JIT support is available, the non-DFA tests are run a third time,
568: this time with a forced pcre_study() with the PCRE_STUDY_JIT_COMPILE option.
569:
570: RunTest uses a file called testtry to hold the main output from pcretest
571: (testsavedregex is also used as a working file). To run pcretest on just one of
572: the test files, give its number as an argument to RunTest, for example:
573:
574: RunTest 2
575:
576: The first test file can be fed directly into the perltest.pl script to check
577: that Perl gives the same results. The only difference you should see is in the
578: first few lines, where the Perl version is given instead of the PCRE version.
579:
580: The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(),
581: pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error
582: detection, and run-time flags that are specific to PCRE, as well as the POSIX
583: wrapper API. It also uses the debugging flags to check some of the internals of
584: pcre_compile().
585:
586: If you build PCRE with a locale setting that is not the standard C locale, the
587: character tables may be different (see next paragraph). In some cases, this may
588: cause failures in the second set of tests. For example, in a locale where the
589: isprint() function yields TRUE for characters in the range 128-255, the use of
590: [:isascii:] inside a character class defines a different set of characters, and
591: this shows up in this test as a difference in the compiled code, which is being
592: listed for checking. Where the comparison test output contains [\x00-\x7f] the
593: test will contain [\x00-\xff], and similarly in some other cases. This is not a
594: bug in PCRE.
595:
596: The third set of tests checks pcre_maketables(), the facility for building a
597: set of character tables for a specific locale and using them instead of the
598: default tables. The tests make use of the "fr_FR" (French) locale. Before
599: running the test, the script checks for the presence of this locale by running
600: the "locale" command. If that command fails, or if it doesn't include "fr_FR"
601: in the list of available locales, the third test cannot be run, and a comment
602: is output to say why. If running this test produces instances of the error
603:
604: ** Failed to set locale "fr_FR"
605:
606: in the comparison output, it means that locale is not available on your system,
607: despite being listed by "locale". This does not mean that PCRE is broken.
608:
609: [If you are trying to run this test on Windows, you may be able to get it to
610: work by changing "fr_FR" to "french" everywhere it occurs. Alternatively, use
611: RunTest.bat. The version of RunTest.bat included with PCRE 7.4 and above uses
612: Windows versions of test 2. More info on using RunTest.bat is included in the
613: document entitled NON-UNIX-USE.]
614:
615: The fourth test checks the UTF-8 support. This file can be also fed directly to
616: the perltest.pl script, provided you are running Perl 5.8 or higher.
617:
618: The fifth test checks error handling with UTF-8 encoding, and internal UTF-8
619: features of PCRE that are not relevant to Perl.
620:
621: The sixth test (which is Perl-5.10 compatible) checks the support for Unicode
622: character properties. This file can be also fed directly to the perltest.pl
623: script, provided you are running Perl 5.10 or higher.
624:
625: The seventh, eighth, and ninth tests check the pcre_dfa_exec() alternative
626: matching function, in non-UTF-8 mode, UTF-8 mode, and UTF-8 mode with Unicode
627: property support, respectively.
628:
629: The tenth test checks some internal offsets and code size features; it is run
630: only when the default "link size" of 2 is set (in other cases the sizes
631: change) and when Unicode property support is enabled.
632:
633: The eleventh and twelfth tests check out features that are new in Perl 5.10,
634: without and with UTF-8 support, respectively. This file can be also fed
635: directly to the perltest.pl script, provided you are running Perl 5.10 or
636: higher.
637:
638: The thirteenth test checks a number internals and non-Perl features concerned
639: with Unicode property support.
640:
641: The fourteenth test is run only when JIT support is available, and the
642: fifteenth test is run only when JIT support is not available. They test some
643: JIT-specific features such as information output from pcretest about JIT
644: compilation.
645:
646:
647: Character tables
648: ----------------
649:
650: For speed, PCRE uses four tables for manipulating and identifying characters
651: whose code point values are less than 256. The final argument of the
652: pcre_compile() function is a pointer to a block of memory containing the
653: concatenated tables. A call to pcre_maketables() can be used to generate a set
654: of tables in the current locale. If the final argument for pcre_compile() is
655: passed as NULL, a set of default tables that is built into the binary is used.
656:
657: The source file called pcre_chartables.c contains the default set of tables. By
658: default, this is created as a copy of pcre_chartables.c.dist, which contains
659: tables for ASCII coding. However, if --enable-rebuild-chartables is specified
660: for ./configure, a different version of pcre_chartables.c is built by the
661: program dftables (compiled from dftables.c), which uses the ANSI C character
662: handling functions such as isalnum(), isalpha(), isupper(), islower(), etc. to
663: build the table sources. This means that the default C locale which is set for
664: your system will control the contents of these default tables. You can change
665: the default tables by editing pcre_chartables.c and then re-building PCRE. If
666: you do this, you should take care to ensure that the file does not get
667: automatically re-generated. The best way to do this is to move
668: pcre_chartables.c.dist out of the way and replace it with your customized
669: tables.
670:
671: When the dftables program is run as a result of --enable-rebuild-chartables,
672: it uses the default C locale that is set on your system. It does not pay
673: attention to the LC_xxx environment variables. In other words, it uses the
674: system's default locale rather than whatever the compiling user happens to have
675: set. If you really do want to build a source set of character tables in a
676: locale that is specified by the LC_xxx variables, you can run the dftables
677: program by hand with the -L option. For example:
678:
679: ./dftables -L pcre_chartables.c.special
680:
681: The first two 256-byte tables provide lower casing and case flipping functions,
682: respectively. The next table consists of three 32-byte bit maps which identify
683: digits, "word" characters, and white space, respectively. These are used when
684: building 32-byte bit maps that represent character classes for code points less
685: than 256.
686:
687: The final 256-byte table has bits indicating various character types, as
688: follows:
689:
690: 1 white space character
691: 2 letter
692: 4 decimal digit
693: 8 hexadecimal digit
694: 16 alphanumeric or '_'
695: 128 regular expression metacharacter or binary zero
696:
697: You should not alter the set of characters that contain the 128 bit, as that
698: will cause PCRE to malfunction.
699:
700:
701: File manifest
702: -------------
703:
704: The distribution should contain the following files:
705:
706: (A) Source files of the PCRE library functions and their headers:
707:
708: dftables.c auxiliary program for building pcre_chartables.c
709: when --enable-rebuild-chartables is specified
710:
711: pcre_chartables.c.dist a default set of character tables that assume ASCII
712: coding; used, unless --enable-rebuild-chartables is
713: specified, by copying to pcre_chartables.c
714:
715: pcreposix.c )
716: pcre_compile.c )
717: pcre_config.c )
718: pcre_dfa_exec.c )
719: pcre_exec.c )
720: pcre_fullinfo.c )
721: pcre_get.c ) sources for the functions in the library,
722: pcre_globals.c ) and some internal functions that they use
723: pcre_info.c )
724: pcre_jit_compile.c )
725: pcre_maketables.c )
726: pcre_newline.c )
727: pcre_ord2utf8.c )
728: pcre_refcount.c )
729: pcre_study.c )
730: pcre_tables.c )
731: pcre_try_flipped.c )
732: pcre_ucd.c )
733: pcre_valid_utf8.c )
734: pcre_version.c )
735: pcre_xclass.c )
736: pcre_printint.src ) debugging function that is #included in pcretest,
737: ) and can also be #included in pcre_compile()
738: pcre.h.in template for pcre.h when built by "configure"
739: pcreposix.h header for the external POSIX wrapper API
740: pcre_internal.h header for internal use
741: sljit/* 16 files that make up the JIT compiler
742: ucp.h header for Unicode property handling
743:
744: config.h.in template for config.h, which is built by "configure"
745:
746: pcrecpp.h public header file for the C++ wrapper
747: pcrecpparg.h.in template for another C++ header file
748: pcre_scanner.h public header file for C++ scanner functions
749: pcrecpp.cc )
750: pcre_scanner.cc ) source for the C++ wrapper library
751:
752: pcre_stringpiece.h.in template for pcre_stringpiece.h, the header for the
753: C++ stringpiece functions
754: pcre_stringpiece.cc source for the C++ stringpiece functions
755:
756: (B) Source files for programs that use PCRE:
757:
758: pcredemo.c simple demonstration of coding calls to PCRE
759: pcregrep.c source of a grep utility that uses PCRE
760: pcretest.c comprehensive test program
761:
762: (C) Auxiliary files:
763:
764: 132html script to turn "man" pages into HTML
765: AUTHORS information about the author of PCRE
766: ChangeLog log of changes to the code
767: CleanTxt script to clean nroff output for txt man pages
768: Detrail script to remove trailing spaces
769: HACKING some notes about the internals of PCRE
770: INSTALL generic installation instructions
771: LICENCE conditions for the use of PCRE
772: COPYING the same, using GNU's standard name
773: Makefile.in ) template for Unix Makefile, which is built by
774: ) "configure"
775: Makefile.am ) the automake input that was used to create
776: ) Makefile.in
777: NEWS important changes in this release
778: NON-UNIX-USE notes on building PCRE on non-Unix systems
779: PrepareRelease script to make preparations for "make dist"
780: README this file
781: RunTest a Unix shell script for running tests
782: RunGrepTest a Unix shell script for pcregrep tests
783: aclocal.m4 m4 macros (generated by "aclocal")
784: config.guess ) files used by libtool,
785: config.sub ) used only when building a shared library
786: configure a configuring shell script (built by autoconf)
787: configure.ac ) the autoconf input that was used to build
788: ) "configure" and config.h
789: depcomp ) script to find program dependencies, generated by
790: ) automake
791: doc/*.3 man page sources for PCRE
792: doc/*.1 man page sources for pcregrep and pcretest
793: doc/index.html.src the base HTML page
794: doc/html/* HTML documentation
795: doc/pcre.txt plain text version of the man pages
796: doc/pcretest.txt plain text documentation of test program
797: doc/perltest.txt plain text documentation of Perl test program
798: install-sh a shell script for installing files
799: libpcre.pc.in template for libpcre.pc for pkg-config
800: libpcreposix.pc.in template for libpcreposix.pc for pkg-config
801: libpcrecpp.pc.in template for libpcrecpp.pc for pkg-config
802: ltmain.sh file used to build a libtool script
803: missing ) common stub for a few missing GNU programs while
804: ) installing, generated by automake
805: mkinstalldirs script for making install directories
806: perltest.pl Perl test program
807: pcre-config.in source of script which retains PCRE information
808: pcre_jit_test.c test program for the JIT compiler
809: pcrecpp_unittest.cc )
810: pcre_scanner_unittest.cc ) test programs for the C++ wrapper
811: pcre_stringpiece_unittest.cc )
812: testdata/testinput* test data for main library tests
813: testdata/testoutput* expected test results
814: testdata/grep* input and output for pcregrep tests
815:
816: (D) Auxiliary files for cmake support
817:
818: cmake/COPYING-CMAKE-SCRIPTS
819: cmake/FindPackageHandleStandardArgs.cmake
820: cmake/FindReadline.cmake
821: CMakeLists.txt
822: config-cmake.h.in
823:
824: (E) Auxiliary files for VPASCAL
825:
826: makevp.bat
827: makevp_c.txt
828: makevp_l.txt
829: pcregexp.pas
830:
831: (F) Auxiliary files for building PCRE "by hand"
832:
833: pcre.h.generic ) a version of the public PCRE header file
834: ) for use in non-"configure" environments
835: config.h.generic ) a version of config.h for use in non-"configure"
836: ) environments
837:
838: (F) Miscellaneous
839:
840: RunTest.bat a script for running tests under Windows
841:
842: Philip Hazel
843: Email local part: ph10
844: Email domain: cam.ac.uk
845: Last updated: 06 September 2011
FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>
![[BACK]](/icons/cvsweb/back.gif){kind=icon}
Return to README CVS log ![[TXT]](/icons/cvsweb/text.gif){kind=icon}
![[DIR]](/icons/cvsweb/dir.gif){kind=icon}
Up to [ELWIX - Embedded LightWeight unIX -] / embedaddon / pcre