Annotation of embedaddon/curl/tests/runtests.1, revision 1.1
1.1 ! misho 1: .\" **************************************************************************
! 2: .\" * _ _ ____ _
! 3: .\" * Project ___| | | | _ \| |
! 4: .\" * / __| | | | |_) | |
! 5: .\" * | (__| |_| | _ <| |___
! 6: .\" * \___|\___/|_| \_\_____|
! 7: .\" *
! 8: .\" * Copyright (C) 1998 - 2020, Daniel Stenberg, <daniel@haxx.se>, et al.
! 9: .\" *
! 10: .\" * This software is licensed as described in the file COPYING, which
! 11: .\" * you should have received as part of this distribution. The terms
! 12: .\" * are also available at https://curl.haxx.se/docs/copyright.html.
! 13: .\" *
! 14: .\" * You may opt to use, copy, modify, merge, publish, distribute and/or sell
! 15: .\" * copies of the Software, and permit persons to whom the Software is
! 16: .\" * furnished to do so, under the terms of the COPYING file.
! 17: .\" *
! 18: .\" * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
! 19: .\" * KIND, either express or implied.
! 20: .\" *
! 21: .\" **************************************************************************
! 22: .\"
! 23: .TH runtests.pl 1 "March 03, 2020" "Curl 7.70.0" "runtests"
! 24:
! 25: .SH NAME
! 26: runtests.pl \- run one or more test cases
! 27: .SH SYNOPSIS
! 28: .B runtests.pl [options] [tests]
! 29: .SH DESCRIPTION
! 30: \fIruntests.pl\fP runs one, several or all the existing test cases in curl's
! 31: test suite. It is often called from the root Makefile of the curl package with
! 32: \&'make test'.
! 33: .SH "TESTS"
! 34: Specify which test(s) to run by specifying test numbers or keywords.
! 35:
! 36: If no test number or keyword is given, all existing tests that the script can
! 37: find will be considered for running. You can specify single test cases to run
! 38: by specifying test numbers space-separated, like "1 3 5 7 11", and you can
! 39: specify a range of tests like "45 to 67".
! 40:
! 41: Specify tests to not run with a leading exclamation point, like "!66", which
! 42: runs all available tests except number 66.
! 43:
! 44: Prefix a test number with a tilde (~) to still run it, but ignore the results.
! 45:
! 46: It is also possible to specify tests based on a keyword describing the test(s)
! 47: to run, like "FTPS". The keywords are strings used in the indiviual tests.
! 48:
! 49: You can also specify keywords with a leading exclamation point and the keyword
! 50: or phrase, like "!HTTP NTLM auth" to run all tests \fBexcept\fP those using
! 51: this keyword. Remember that the exclamation marks and spaces will need to be
! 52: quoted somehow when entered at many command shells.
! 53:
! 54: Prefix a keyword with a tilde (~) to still run it, but ignore the results.
! 55: .SH OPTIONS
! 56: .IP "-a"
! 57: Continue running the rest of the test cases even if one test fails. By
! 58: default, the test script stops as soon as an error is detected.
! 59: .IP "-bN"
! 60: Use N as the base TCP/UDP port number on which to start the test servers.
! 61: .IP "-c <curl>"
! 62: Provide a path to a custom curl binary to run the tests with. Default is the
! 63: curl executable in the build tree.
! 64: .IP "-d"
! 65: Enable protocol debug: have the servers display protocol output.
! 66: .IP "-e"
! 67: Run the test event-based (if possible). This will make runtests invoke curl
! 68: with --test-event option. This option only works if both curl and libcurl were
! 69: built debug-enabled.
! 70: .IP "-g"
! 71: Run the given test(s) with gdb. This is best used on a single test case and
! 72: curl built --disable-shared. This then fires up gdb with command line set to
! 73: run the specified test case. Simply (set a break-point and) type 'run' to
! 74: start.
! 75: .IP "-h"
! 76: Displays a help text about this program's command line options.
! 77: .IP "-k"
! 78: Keep output and log files in log/ after a test run, even if no error was
! 79: detected. Useful for debugging.
! 80: .IP "-l"
! 81: Lists all test case names.
! 82: .IP "-n"
! 83: Disable the check for and use of valgrind.
! 84: .IP "-p"
! 85: Prints out all files in "log/" to stdout when a test case fails. Very
! 86: practical when used in the automated and distributed tests since then the
! 87: people checking the failures and the reasons for them might not have physical
! 88: access to the machine and logs.
! 89: .IP "-R"
! 90: Run the tests in a scrambled, or randomized, order instead of sequentially.
! 91:
! 92: The random seed initially set for this is fixed per month and can be set with
! 93: \fI--seed\fP.
! 94: .IP "-r"
! 95: Display run time statistics. (Requires Perl Time::HiRes module)
! 96: .IP "-rf"
! 97: Display full run time statistics. (Requires Perl Time::HiRes module)
! 98: .IP "--repeat=[num]"
! 99: This will repeat the given set of test numbers this many times. If no test
! 100: numbers are given, it will repeat ALL tests this many times. It iteratively
! 101: adds the new sequence at the end of the initially given one.
! 102:
! 103: If \fB-R\fP is also used, the scrambling is done after the repeats have
! 104: extended the test sequence.
! 105: .IP "-s"
! 106: Shorter output. Speaks less than default.
! 107: .IP "--seed=[num]"
! 108: When using \fI--shallow\fP or \fI-R\rP that random certain aspects of the
! 109: behavior, this option can set the initial seed. If not set, the random seed
! 110: will be set based on the currently set local year and month and the first line
! 111: of the "curl -V" output.
! 112: .IP "--shallow=[num]"
! 113: Used together with \fB-t\fP. This limits the number of tests to fail in
! 114: torture mode to no more than 'num' per test case. If this reduces the amount,
! 115: the script will randomly discard entries to fail until the amount is 'num'.
! 116:
! 117: The random seed initially set for this is fixed per month and can be set with
! 118: \fI--seed\fP.
! 119: .IP "-t[num]"
! 120: Selects a \fBtorture\fP test for the given tests. This makes runtests.pl first
! 121: run the tests once and count the number of memory allocations made. It then
! 122: reruns the test that number of times, each time forcing one of the allocations
! 123: to fail until all allocs have been tested. By setting \fInum\fP you can force
! 124: the allocation with that number to be set to fail at once instead of looping
! 125: through everyone, which is very handy when debugging and then often in
! 126: combination with \fI-g\fP.
! 127: .IP "-v"
! 128: Enable verbose output. Speaks more than default.
! 129: .IP "-vc <curl>"
! 130: Provide a path to a custom curl binary to run when verifying that the servers
! 131: running are indeed our test servers. Default is the curl executable in the
! 132: build tree.
! 133: .SH "RUNNING TESTS"
! 134: Many tests have conditions that must be met before the test case can run
! 135: fine. They could depend on built-in features in libcurl or features present in
! 136: the operating system or even in third-party libraries that curl may or may not
! 137: use.
! 138: .P
! 139: The test script checks most of these by itself to determine when it is
! 140: safe to attempt to run each test. Those which cannot be run due to
! 141: failed requirements will simply be skipped and listed at the completion
! 142: of all test cases. In some unusual configurations, the test script
! 143: cannot make the correct determination for all tests. In these cases,
! 144: the problematic tests can be skipped using the "!keyword" skip feature
! 145: documented earlier.
! 146: .SH "WRITING TESTS"
! 147: The simplest way to write test cases is to start with a similar existing test,
! 148: save it with a new number and then adjust it to fit. There's an attempt to
! 149: document the test case file format in the tests/FILEFORMAT.
FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>
![[BACK]](/icons/cvsweb/back.gif){kind=icon}
Return to runtests.1 CVS log ![[TXT]](/icons/cvsweb/text.gif){kind=icon}
![[DIR]](/icons/cvsweb/dir.gif){kind=icon}
Up to [ELWIX - Embedded LightWeight unIX -] / embedaddon / curl / tests