embedaddon/pcre/NON-UNIX-USE - annotate

Return to NON-UNIX-USE CVS log

Up to [ELWIX - Embedded LightWeight unIX -] / embedaddon / pcre

Annotation of embedaddon/pcre/NON-UNIX-USE, revision 1.1.1.1

1.1 misho 1: Compiling PCRE on non-Unix systems
2: ----------------------------------
3:
4: This document contains the following sections:
5:
6: General
7: Generic instructions for the PCRE C library
8: The C++ wrapper functions
9: Building for virtual Pascal
10: Stack size in Windows environments
11: Linking programs in Windows environments
12: Comments about Win32 builds
13: Building PCRE on Windows with CMake
14: Use of relative paths with CMake on Windows
15: Testing with RunTest.bat
16: Building under Windows with BCC5.5
17: Building PCRE on OpenVMS
18: Building PCRE on Stratus OpenVOS
19:
20:
21: GENERAL
22:
23: I (Philip Hazel) have no experience of Windows or VMS sytems and how their
24: libraries work. The items in the PCRE distribution and Makefile that relate to
25: anything other than Unix-like systems are untested by me.
26:
27: There are some other comments and files (including some documentation in CHM
28: format) in the Contrib directory on the FTP site:
29:
30: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib
31:
32: If you want to compile PCRE for a non-Unix system (especially for a system that
33: does not support "configure" and "make" files), note that the basic PCRE
34: library consists entirely of code written in Standard C, and so should compile
35: successfully on any system that has a Standard C compiler and library. The C++
36: wrapper functions are a separate issue (see below).
37:
38: The PCRE distribution includes a "configure" file for use by the Configure/Make
39: build system, as found in many Unix-like environments. There is also support
40: for CMake, which some users prefer, especially in Windows environments. See
41: the instructions for CMake under Windows in the section entitled "Building
42: PCRE with CMake" below. CMake can also be used to build PCRE in Unix-like
43: systems.
44:
45:
46: GENERIC INSTRUCTIONS FOR THE PCRE C LIBRARY
47:
48: The following are generic instructions for building the PCRE C library "by
49: hand":
50:
51: (1) Copy or rename the file config.h.generic as config.h, and edit the macro
52: settings that it contains to whatever is appropriate for your environment.
53: In particular, if you want to force a specific value for newline, you can
54: define the NEWLINE macro. When you compile any of the PCRE modules, you
55: must specify -DHAVE_CONFIG_H to your compiler so that config.h is included
56: in the sources.
57:
58: An alternative approach is not to edit config.h, but to use -D on the
59: compiler command line to make any changes that you need to the
60: configuration options. In this case -DHAVE_CONFIG_H must not be set.
61:
62: NOTE: There have been occasions when the way in which certain parameters
63: in config.h are used has changed between releases. (In the configure/make
64: world, this is handled automatically.) When upgrading to a new release,
65: you are strongly advised to review config.h.generic before re-using what
66: you had previously.
67:
68: (2) Copy or rename the file pcre.h.generic as pcre.h.
69:
70: (3) EITHER:
71: Copy or rename file pcre_chartables.c.dist as pcre_chartables.c.
72:
73: OR:
74: Compile dftables.c as a stand-alone program (using -DHAVE_CONFIG_H if
75: you have set up config.h), and then run it with the single argument
76: "pcre_chartables.c". This generates a set of standard character tables
77: and writes them to that file. The tables are generated using the default
78: C locale for your system. If you want to use a locale that is specified
79: by LC_xxx environment variables, add the -L option to the dftables
80: command. You must use this method if you are building on a system that
81: uses EBCDIC code.
82:
83: The tables in pcre_chartables.c are defaults. The caller of PCRE can
84: specify alternative tables at run time.
85:
86: (4) Ensure that you have the following header files:
87:
88: pcre_internal.h
89: ucp.h
90:
91: (5) Also ensure that you have the following file, which is #included as source
92: when building a debugging version of PCRE, and is also used by pcretest.
93:
94: pcre_printint.src
95:
96: (6) Compile the following source files, setting -DHAVE_CONFIG_H as a compiler
97: option if you have set up config.h with your configuration, or else use
98: other -D settings to change the configuration as required.
99:
100: pcre_chartables.c
101: pcre_compile.c
102: pcre_config.c
103: pcre_dfa_exec.c
104: pcre_exec.c
105: pcre_fullinfo.c
106: pcre_get.c
107: pcre_globals.c
108: pcre_info.c
109: pcre_maketables.c
110: pcre_newline.c
111: pcre_ord2utf8.c
112: pcre_refcount.c
113: pcre_study.c
114: pcre_tables.c
115: pcre_try_flipped.c
116: pcre_ucd.c
117: pcre_valid_utf8.c
118: pcre_version.c
119: pcre_xclass.c
120:
121: Make sure that you include -I. in the compiler command (or equivalent for
122: an unusual compiler) so that all included PCRE header files are first
123: sought in the current directory. Otherwise you run the risk of picking up
124: a previously-installed file from somewhere else.
125:
126: (7) If you have defined SUPPORT_JIT in config.h, you must also compile
127:
128: pcre_jit_compile.c
129:
130: This file #includes sources from the sljit subdirectory, where there
131: should be 16 files, all of whose names begin with "sljit".
132:
133: (8) Now link all the compiled code into an object library in whichever form
134: your system keeps such libraries. This is the basic PCRE C library. If
135: your system has static and shared libraries, you may have to do this once
136: for each type.
137:
138: (9) Similarly, if you want to build the POSIX wrapper functions, ensure that
139: you have the pcreposix.h file and then compile pcreposix.c (remembering
140: -DHAVE_CONFIG_H if necessary). Link the result (on its own) as the
141: pcreposix library.
142:
143: (10) Compile the test program pcretest.c (again, don't forget -DHAVE_CONFIG_H).
144: This needs the functions in the PCRE library when linking. It also needs
145: the pcreposix wrapper functions unless you compile it with -DNOPOSIX. The
146: pcretest.c program also needs the pcre_printint.src source file, which it
147: #includes.
148:
149: (11) Run pcretest on the testinput files in the testdata directory, and check
150: that the output matches the corresponding testoutput files. Some tests are
151: relevant only when certain build-time options are selected. For example,
152: test 4 is for UTF-8 support, and will not run if you have build PCRE
153: without it. See the comments at the start of each testinput file. If you
154: have a suitable Unix-like shell, the RunTest script will run the
155: appropriate tests for you.
156:
157: Note that the supplied files are in Unix format, with just LF characters
158: as line terminators. You may need to edit them to change this if your
159: system uses a different convention. If you are using Windows, you probably
160: should use the wintestinput3 file instead of testinput3 (and the
161: corresponding output file). This is a locale test; wintestinput3 sets the
162: locale to "french" rather than "fr_FR", and there some minor output
163: differences.
164:
165: (12) If you have built PCRE with SUPPORT_JIT, the JIT features will be tested
166: by the testdata files. However, you might also like to build and run
167: the JIT test program, pcre_jit_test.c.
168:
169: (13) If you want to use the pcregrep command, compile and link pcregrep.c; it
170: uses only the basic PCRE library (it does not need the pcreposix library).
171:
172:
173: THE C++ WRAPPER FUNCTIONS
174:
175: The PCRE distribution also contains some C++ wrapper functions and tests,
176: contributed by Google Inc. On a system that can use "configure" and "make",
177: the functions are automatically built into a library called pcrecpp. It should
178: be straightforward to compile the .cc files manually on other systems. The
179: files called xxx_unittest.cc are test programs for each of the corresponding
180: xxx.cc files.
181:
182:
183: BUILDING FOR VIRTUAL PASCAL
184:
185: A script for building PCRE using Borland's C++ compiler for use with VPASCAL
186: was contributed by Alexander Tokarev. Stefan Weber updated the script and added
187: additional files. The following files in the distribution are for building PCRE
188: for use with VP/Borland: makevp_c.txt, makevp_l.txt, makevp.bat, pcregexp.pas.
189:
190:
191: STACK SIZE IN WINDOWS ENVIRONMENTS
192:
193: The default processor stack size of 1Mb in some Windows environments is too
194: small for matching patterns that need much recursion. In particular, test 2 may
195: fail because of this. Normally, running out of stack causes a crash, but there
196: have been cases where the test program has just died silently. See your linker
197: documentation for how to increase stack size if you experience problems. The
198: Linux default of 8Mb is a reasonable choice for the stack, though even that can
199: be too small for some pattern/subject combinations.
200:
201: PCRE has a compile configuration option to disable the use of stack for
202: recursion so that heap is used instead. However, pattern matching is
203: significantly slower when this is done. There is more about stack usage in the
204: "pcrestack" documentation.
205:
206:
207: LINKING PROGRAMS IN WINDOWS ENVIRONMENTS
208:
209: If you want to statically link a program against a PCRE library in the form of
210: a non-dll .a file, you must define PCRE_STATIC before including pcre.h or
211: pcrecpp.h, otherwise the pcre_malloc() and pcre_free() exported functions will
212: be declared __declspec(dllimport), with unwanted results.
213:
214:
215: CALLING CONVENTIONS IN WINDOWS ENVIRONMENTS
216:
217: It is possible to compile programs to use different calling conventions using
218: MSVC. Search the web for "calling conventions" for more information. To make it
219: easier to change the calling convention for the exported functions in the
220: PCRE library, the macro PCRE_CALL_CONVENTION is present in all the external
221: definitions. It can be set externally when compiling (e.g. in CFLAGS). If it is
222: not set, it defaults to empty; the default calling convention is then used
223: (which is what is wanted most of the time).
224:
225:
226: COMMENTS ABOUT WIN32 BUILDS (see also "BUILDING PCRE WITH CMAKE" below)
227:
228: There are two ways of building PCRE using the "configure, make, make install"
229: paradigm on Windows systems: using MinGW or using Cygwin. These are not at all
230: the same thing; they are completely different from each other. There is also
231: support for building using CMake, which some users find a more straightforward
232: way of building PCRE under Windows.
233:
234: The MinGW home page (http://www.mingw.org/) says this:
235:
236: MinGW: A collection of freely available and freely distributable Windows
237: specific header files and import libraries combined with GNU toolsets that
238: allow one to produce native Windows programs that do not rely on any
239: 3rd-party C runtime DLLs.
240:
241: The Cygwin home page (http://www.cygwin.com/) says this:
242:
243: Cygwin is a Linux-like environment for Windows. It consists of two parts:
244:
245: . A DLL (cygwin1.dll) which acts as a Linux API emulation layer providing
246: substantial Linux API functionality
247:
248: . A collection of tools which provide Linux look and feel.
249:
250: The Cygwin DLL currently works with all recent, commercially released x86 32
251: bit and 64 bit versions of Windows, with the exception of Windows CE.
252:
253: On both MinGW and Cygwin, PCRE should build correctly using:
254:
255: ./configure && make && make install
256:
257: This should create two libraries called libpcre and libpcreposix, and, if you
258: have enabled building the C++ wrapper, a third one called libpcrecpp. These are
259: independent libraries: when you link with libpcreposix or libpcrecpp you must
260: also link with libpcre, which contains the basic functions. (Some earlier
261: releases of PCRE included the basic libpcre functions in libpcreposix. This no
262: longer happens.)
263:
264: A user submitted a special-purpose patch that makes it easy to create
265: "pcre.dll" under mingw32 using the "msys" environment. It provides "pcre.dll"
266: as a special target. If you use this target, no other files are built, and in
267: particular, the pcretest and pcregrep programs are not built. An example of how
268: this might be used is:
269:
270: ./configure --enable-utf --disable-cpp CFLAGS="-03 -s"; make pcre.dll
271:
272: Using Cygwin's compiler generates libraries and executables that depend on
273: cygwin1.dll. If a library that is generated this way is distributed,
274: cygwin1.dll has to be distributed as well. Since cygwin1.dll is under the GPL
275: licence, this forces not only PCRE to be under the GPL, but also the entire
276: application. A distributor who wants to keep their own code proprietary must
277: purchase an appropriate Cygwin licence.
278:
279: MinGW has no such restrictions. The MinGW compiler generates a library or
280: executable that can run standalone on Windows without any third party dll or
281: licensing issues.
282:
283: But there is more complication:
284:
285: If a Cygwin user uses the -mno-cygwin Cygwin gcc flag, what that really does is
286: to tell Cygwin's gcc to use the MinGW gcc. Cygwin's gcc is only acting as a
287: front end to MinGW's gcc (if you install Cygwin's gcc, you get both Cygwin's
288: gcc and MinGW's gcc). So, a user can:
289:
290: . Build native binaries by using MinGW or by getting Cygwin and using
291: -mno-cygwin.
292:
293: . Build binaries that depend on cygwin1.dll by using Cygwin with the normal
294: compiler flags.
295:
296: The test files that are supplied with PCRE are in UNIX format, with LF
297: characters as line terminators. Unless your PCRE library uses a default newline
298: option that includes LF as a valid newline, it may be necessary to change the
299: line terminators in the test files to get some of the tests to work.
300:
301: BUILDING PCRE ON WINDOWS WITH CMAKE
302:
303: CMake is an alternative configuration facility that can be used instead of the
304: traditional Unix "configure". CMake creates project files (make files, solution
305: files, etc.) tailored to numerous development environments, including Visual
306: Studio, Borland, Msys, MinGW, NMake, and Unix. If possible, use short paths
307: with no spaces in the names for your CMake installation and your pcre
308: source and build directories.
309:
310: The following instructions were contributed by a PCRE user.
311:
312: 1. Install the latest CMake version available from http://www.cmake.org/, and
313: ensure that cmake\bin is on your path.
314:
315: 2. Unzip (retaining folder structure) the PCRE source tree into a source
316: directory such as C:\pcre. You should ensure your local date and time
317: is not earlier than the file dates in your source dir if the release is
318: very new.
319:
320: 3. Create a new, empty build directory, preferably a subdirectory of the
321: source dir. For example, C:\pcre\pcre-xx\build.
322:
323: 4. Run cmake-gui from the Shell envirornment of your build tool, for example,
324: Msys for Msys/MinGW or Visual Studio Command Prompt for VC/VC++.
325:
326: 5. Enter C:\pcre\pcre-xx and C:\pcre\pcre-xx\build for the source and build
327: directories, respectively.
328:
329: 6. Hit the "Configure" button.
330:
331: 7. Select the particular IDE / build tool that you are using (Visual
332: Studio, MSYS makefiles, MinGW makefiles, etc.)
333:
334: 8. The GUI will then list several configuration options. This is where
335: you can enable UTF-8 support or other PCRE optional features.
336:
337: 9. Hit "Configure" again. The adjacent "Generate" button should now be
338: active.
339:
340: 10. Hit "Generate".
341:
342: 11. The build directory should now contain a usable build system, be it a
343: solution file for Visual Studio, makefiles for MinGW, etc. Exit from
344: cmake-gui and use the generated build system with your compiler or IDE.
345: E.g., for MinGW you can run "make", or for Visual Studio, open the PCRE
346: solution, select the desired configuration (Debug, or Release, etc.) and
347: build the ALL_BUILD project.
348:
349: 12. If during configuration with cmake-gui you've elected to build the test
350: programs, you can execute them by building the test project. E.g., for
351: MinGW: "make test"; for Visual Studio build the RUN_TESTS project. The
352: most recent build configuration is targeted by the tests. A summary of
353: test results is presented. Complete test output is subsequently
354: available for review in Testing\Temporary under your build dir.
355:
356: USE OF RELATIVE PATHS WITH CMAKE ON WINDOWS
357:
358: A PCRE user comments as follows:
359:
360: I thought that others may want to know the current state of
361: CMAKE_USE_RELATIVE_PATHS support on Windows.
362:
363: Here it is:
364: -- AdditionalIncludeDirectories is only partially modified (only the
365: first path - see below)
366: -- Only some of the contained file paths are modified - shown below for
367: pcre.vcproj
368: -- It properly modifies
369:
370: I am sure CMake people can fix that if they want to. Until then one will
371: need to replace existing absolute paths in project files with relative
372: paths manually (e.g. from VS) - relative to project file location. I did
373: just that before being told to try CMAKE_USE_RELATIVE_PATHS. Not a big
374: deal.
375:
376: AdditionalIncludeDirectories="E:\builds\pcre\build;E:\builds\pcre\pcre-7.5;"
377: AdditionalIncludeDirectories=".;E:\builds\pcre\pcre-7.5;"
378:
379: RelativePath="pcre.h">
380: RelativePath="pcre_chartables.c">
381: RelativePath="pcre_chartables.c.rule">
382:
383:
384: TESTING WITH RUNTEST.BAT
385:
386: If configured with CMake, building the test project ("make test" or building
387: ALL_TESTS in Visual Studio) creates (and runs) pcre_test.bat (and depending
388: on your configuration options, possibly other test programs) in the build
389: directory. Pcre_test.bat runs RunTest.Bat with correct source and exe paths.
390:
391: For manual testing with RunTest.bat, provided the build dir is a subdirectory
392: of the source directory: Open command shell window. Chdir to the location
393: of your pcretest.exe and pcregrep.exe programs. Call RunTest.bat with
394: "..\RunTest.Bat" or "..\..\RunTest.bat" as appropriate.
395:
396: To run only a particular test with RunTest.Bat provide a test number argument.
397:
398: Otherwise:
399:
400: 1. Copy RunTest.bat into the directory where pcretest.exe and pcregrep.exe
401: have been created.
402:
403: 2. Edit RunTest.bat to indentify the full or relative location of
404: the pcre source (wherein which the testdata folder resides), e.g.:
405:
406: set srcdir=C:\pcre\pcre-8.20
407:
408: 3. In a Windows command environment, chdir to the location of your bat and
409: exe programs.
410:
411: 4. Run RunTest.bat. Test outputs will automatically be compared to expected
412: results, and discrepancies will be identified in the console output.
413:
414: To independently test the just-in-time compiler, run pcre_jit_test.exe.
415: To test pcrecpp, run pcrecpp_unittest.exe, pcre_stringpiece_unittest.exe and
416: pcre_scanner_unittest.exe.
417:
418: BUILDING UNDER WINDOWS WITH BCC5.5
419:
420: Michael Roy sent these comments about building PCRE under Windows with BCC5.5:
421:
422: Some of the core BCC libraries have a version of PCRE from 1998 built in,
423: which can lead to pcre_exec() giving an erroneous PCRE_ERROR_NULL from a
424: version mismatch. I'm including an easy workaround below, if you'd like to
425: include it in the non-unix instructions:
426:
427: When linking a project with BCC5.5, pcre.lib must be included before any of
428: the libraries cw32.lib, cw32i.lib, cw32mt.lib, and cw32mti.lib on the command
429: line.
430:
431:
432: BUILDING UNDER WINDOWS CE WITH VISUAL STUDIO 200x
433:
434: Vincent Richomme sent a zip archive of files to help with this process. They
435: can be found in the file "pcre-vsbuild.zip" in the Contrib directory of the FTP
436: site.
437:
438:
439: BUILDING PCRE ON OPENVMS
440:
441: Dan Mooney sent the following comments about building PCRE on OpenVMS. They
442: relate to an older version of PCRE that used fewer source files, so the exact
443: commands will need changing. See the current list of source files above.
444:
445: "It was quite easy to compile and link the library. I don't have a formal
446: make file but the attached file [reproduced below] contains the OpenVMS DCL
447: commands I used to build the library. I had to add #define
448: POSIX_MALLOC_THRESHOLD 10 to pcre.h since it was not defined anywhere.
449:
450: The library was built on:
451: O/S: HP OpenVMS v7.3-1
452: Compiler: Compaq C v6.5-001-48BCD
453: Linker: vA13-01
454:
455: The test results did not match 100% due to the issues you mention in your
456: documentation regarding isprint(), iscntrl(), isgraph() and ispunct(). I
457: modified some of the character tables temporarily and was able to get the
458: results to match. Tests using the fr locale did not match since I don't have
459: that locale loaded. The study size was always reported to be 3 less than the
460: value in the standard test output files."
461:
462: =========================
463: $! This DCL procedure builds PCRE on OpenVMS
464: $!
465: $! I followed the instructions in the non-unix-use file in the distribution.
466: $!
467: $ COMPILE == "CC/LIST/NOMEMBER_ALIGNMENT/PREFIX_LIBRARY_ENTRIES=ALL_ENTRIES
468: $ COMPILE DFTABLES.C
469: $ LINK/EXE=DFTABLES.EXE DFTABLES.OBJ
470: $ RUN DFTABLES.EXE/OUTPUT=CHARTABLES.C
471: $ COMPILE MAKETABLES.C
472: $ COMPILE GET.C
473: $ COMPILE STUDY.C
474: $! I had to set POSIX_MALLOC_THRESHOLD to 10 in PCRE.H since the symbol
475: $! did not seem to be defined anywhere.
476: $! I edited pcre.h and added #DEFINE SUPPORT_UTF8 to enable UTF8 support.
477: $ COMPILE PCRE.C
478: $ LIB/CREATE PCRE MAKETABLES.OBJ, GET.OBJ, STUDY.OBJ, PCRE.OBJ
479: $! I had to set POSIX_MALLOC_THRESHOLD to 10 in PCRE.H since the symbol
480: $! did not seem to be defined anywhere.
481: $ COMPILE PCREPOSIX.C
482: $ LIB/CREATE PCREPOSIX PCREPOSIX.OBJ
483: $ COMPILE PCRETEST.C
484: $ LINK/EXE=PCRETEST.EXE PCRETEST.OBJ, PCRE/LIB, PCREPOSIX/LIB
485: $! C programs that want access to command line arguments must be
486: $! defined as a symbol
487: $ PCRETEST :== "$ SYS$ROADSUSERS:[DMOONEY.REGEXP]PCRETEST.EXE"
488: $! Arguments must be enclosed in quotes.
489: $ PCRETEST "-C"
490: $! Test results:
491: $!
492: $! The test results did not match 100%. The functions isprint(), iscntrl(),
493: $! isgraph() and ispunct() on OpenVMS must not produce the same results
494: $! as the system that built the test output files provided with the
495: $! distribution.
496: $!
497: $! The study size did not match and was always 3 less on OpenVMS.
498: $!
499: $! Locale could not be set to fr
500: $!
501: =========================
502:
503:
504: BUILDING PCRE ON STRATUS OPENVOS
505:
506: These notes on the port of PCRE to VOS (lightly edited) were supplied by
507: Ashutosh Warikoo, whose email address has the local part awarikoo and the
508: domain nse.co.in. The port was for version 7.9 in August 2009.
509:
510: 1. Building PCRE
511:
512: I built pcre on OpenVOS Release 17.0.1at using GNU Tools 3.4a without any
513: problems. I used the following packages to build PCRE:
514:
515: ftp://ftp.stratus.com/pub/vos/posix/ga/posix.save.evf.gz
516:
517: Please read and follow the instructions that come with these packages. To start
518: the build of pcre, from the root of the package type:
519:
520: ./build.sh
521:
522: 2. Installing PCRE
523:
524: Once you have successfully built PCRE, login to the SysAdmin group, switch to
525: the root user, and type
526:
527: [ !create_dir (master_disk)>usr --if needed ]
528: [ !create_dir (master_disk)>usr>local --if needed ]
529: !gmake install
530:
531: This installs PCRE and its man pages into /usr/local. You can add
532: (master_disk)>usr>local>bin to your command search paths, or if you are in
533: BASH, add /usr/local/bin to the PATH environment variable.
534:
535: 4. Restrictions
536:
537: This port requires readline library optionally. However during the build I
538: faced some yet unexplored errors while linking with readline. As it was an
539: optional component I chose to disable it.
540:
541: 5. Known Problems
542:
543: I ran the test suite, but you will have to be your own judge of whether this
544: command, and this port, suits your purposes. If you find any problems that
545: appear to be related to the port itself, please let me know. Please see the
546: build.log file in the root of the package also.
547:
548:
549: =========================
550: Last Updated: 9 October 2011
551: ****

FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>