Annotation of embedaddon/curl/tests/FILEFORMAT.md, revision 1.1.1.1
1.1 misho 1: # curl test suite file format
2:
3: The curl test suite's file format is very simple and extensible, closely
4: resembling XML. All data for a single test case resides in a single ASCII
5: file. Labels mark the beginning and the end of all sections, and each label
6: must be written in its own line. Comments are either XML-style (enclosed with
7: `<!--` and `-->`) or shell script style (beginning with `#`) and must appear
8: on their own lines and not alongside actual test data. Most test data files
9: are syntactically valid XML, although a few files are not (lack of support for
10: character entities and the preservation of CR/LF characters at the end of
11: lines are the biggest differences).
12:
13: Each test case exists as a file matching the format `tests/data/testNUM`,
14: where NUM is considered the unique test number.
15:
16: The file begins with a 'testcase' tag, which encompasses the remainder of the
17: file.
18:
19: # `<testcase>`
20:
21: Each test is always within the testcase tag. Each test case is split up in
22: four main sections: `info`, `reply`, `client` and `verify`.
23:
24: - **info** provides information about the test case
25:
26: - **reply** is used for the server to know what to send as a reply for the
27: requests curl sends
28:
29: - **client** defines how the client should behave
30:
31: - **verify** defines how to verify that the data stored after a command has
32: been run ended up correctly
33:
34: Each main section has a number of available subsections that can be specified,
35: that will be checked/used if specified.
36:
37: ## `<info>`
38:
39: ### `<keywords>`
40: A newline-separated list of keywords describing what this test case uses and
41: tests. Try to use an already used keyword. These keywords will be used for
42: statistical/informational purposes and for choosing or skipping classes
43: of tests. "Keywords" must begin with an alphabetic character, "-", "["
44: or "{" and may actually consist of multiple words separated by spaces
45: which are treated together as a single identifier.
46:
47: ## `<reply>`
48:
49: ### `<data [nocheck="yes"] [sendzero="yes"] [base64="yes"] [hex="yes"]>`
50:
51: data to be sent to the client on its request and later verified that it
52: arrived safely. Set `nocheck="yes"` to prevent the test script from verifying
53: the arrival of this data.
54:
55: If the data contains `swsclose` anywhere within the start and end tag, and
56: this is a HTTP test, then the connection will be closed by the server after
57: this response is sent. If not, the connection will be kept persistent.
58:
59: If the data contains `swsbounce` anywhere within the start and end tag, the
60: HTTP server will detect if this is a second request using the same test and
61: part number and will then increase the part number with one. This is useful
62: for auth tests and similar.
63:
64: `sendzero=yes` means that the (FTP) server will "send" the data even if the
65: size is zero bytes. Used to verify curl's behaviour on zero bytes transfers.
66:
67: `base64=yes` means that the data provided in the test-file is a chunk of data
68: encoded with base64. It is the only way a test case can contain binary
69: data. (This attribute can in fact be used on any section, but it doesn't make
70: much sense for other sections than "data").
71:
72: `hex=yes` means that the data is a sequence of hex pairs. It will get decoded
73: and used as "raw" data.
74:
75: For FTP file listings, the `<data>` section will be used *only* if you make
76: sure that there has been a CWD done first to a directory named `test-[num]`
77: where [num] is the test case number. Otherwise the ftp server can't know from
78: which test file to load the list content.
79:
80: ### `<dataNUM>`
81:
82: Send back this contents instead of the <data> one. The num is set by:
83:
84: - The test number in the request line is >10000 and this is the remainder
85: of [test case number]%10000.
86: - The request was HTTP and included digest details, which adds 1000 to NUM
87: - If a HTTP request is NTLM type-1, it adds 1001 to num
88: - If a HTTP request is NTLM type-3, it adds 1002 to num
89: - If a HTTP request is Basic and num is already >=1000, it adds 1 to num
90: - If a HTTP request is Negotiate, num gets incremented by one for each
91: request with Negotiate authorization header on the same test case.
92:
93: Dynamically changing num in this way allows the test harness to be used to
94: test authentication negotiation where several different requests must be sent
95: to complete a transfer. The response to each request is found in its own data
96: section. Validating the entire negotiation sequence can be done by specifying
97: a datacheck section.
98:
99: ### `<connect>`
100: The connect section is used instead of the 'data' for all CONNECT
101: requests. The remainder of the rules for the data section then apply but with
102: a connect prefix.
103:
104: ### `<datacheck [mode="text"] [nonewline="yes"]>`
105: if the data is sent but this is what should be checked afterwards. If
106: `nonewline=yes` is set, runtests will cut off the trailing newline from the
107: data before comparing with the one actually received by the client.
108:
109: Use the `mode="text"` attribute if the output is in text mode on platforms
110: that have a text/binary difference.
111:
112: ### `<datacheckNUM [nonewline="yes"] [mode="text"]>`
113: The contents of numbered datacheck sections are appended to the non-numbered
114: one.
115:
116: ### `<size>`
117: number to return on a ftp SIZE command (set to -1 to make this command fail)
118:
119: ### `<mdtm>`
120: what to send back if the client sends a (FTP) MDTM command, set to -1 to
121: have it return that the file doesn't exist
122:
123: ### `<postcmd>`
124: special purpose server-command to control its behavior *after* the
125: reply is sent
126: For HTTP/HTTPS, these are supported:
127:
128: `wait [secs]` - Pause for the given time
129:
130: ### `<servercmd>`
131: Special-commands for the server.
132:
133: The first line of this file will always be set to `Testnum [number]` by the
134: test script, to allow servers to read that to know what test the client is
135: about to issue.
136:
137: #### For FTP/SMTP/POP/IMAP
138:
139: - `REPLY [command] [return value] [response string]` - Changes how the server
140: responds to the [command]. [response string] is evaluated as a perl string,
141: so it can contain embedded \r\n, for example. There's a special [command]
142: named "welcome" (without quotes) which is the string sent immediately on
143: connect as a welcome.
144: - `REPLYLF` (like above but sends the response terminated with LF-only and not
145: CRLF)
146: - `COUNT [command] [num]` - Do the `REPLY` change for `[command]` only `[num]`
147: times and then go back to the built-in approach
148: - `DELAY [command] [secs]` - Delay responding to this command for the given
149: time
150: - `RETRWEIRDO` - Enable the "weirdo" RETR case when multiple response lines
151: appear at once when a file is transferred
152: - `RETRNOSIZE` - Make sure the RETR response doesn't contain the size of the
153: file
154: - `NOSAVE` - Don't actually save what is received
155: - `SLOWDOWN` - Send FTP responses with 0.01 sec delay between each byte
156: - `PASVBADIP` - makes PASV send back an illegal IP in its 227 response
157: - `CAPA [capabilities]` - Enables support for and specifies a list of space
158: separated capabilities to return to the client for the IMAP `CAPABILITY`,
159: POP3 `CAPA` and SMTP `EHLO` commands
160: - `AUTH [mechanisms]` - Enables support for SASL authentication and specifies
161: a list of space separated mechanisms for IMAP, POP3 and SMTP
162:
163: #### For HTTP/HTTPS
164:
165: - `auth_required` if this is set and a POST/PUT is made without auth, the
166: server will NOT wait for the full request body to get sent
167: - `idle` - do nothing after receiving the request, just "sit idle"
168: - `stream` - continuously send data to the client, never-ending
169: - `writedelay: [secs]` delay this amount between reply packets
170: - `skip: [num]` - instructs the server to ignore reading this many bytes from
171: a PUT or POST request
172: - `rtp: part [num] channel [num] size [num]` - stream a fake RTP packet for
173: the given part on a chosen channel with the given payload size
174: - `connection-monitor` - When used, this will log `[DISCONNECT]` to the
175: `server.input` log when the connection is disconnected.
176: - `upgrade` - when an HTTP upgrade header is found, the server will upgrade to
177: http2
178: - `swsclose` - instruct server to close connection after response
179: - `no-expect` - don't read the request body if Expect: is present
180:
181: #### For TFTP
182: `writedelay: [secs]` delay this amount between reply packets (each packet
183: being 512 bytes payload)
184:
185: ## `<client>`
186:
187: ### `<server>`
188: What server(s) this test case requires/uses. Available servers:
189:
190: - `file`
191: - `ftp-ipv6`
192: - `ftp`
193: - `ftps`
194: - `http-ipv6`
195: - `http-proxy`
196: - `http-unix`
197: - `http/2`
198: - `http`
199: - `https`
200: - `httptls+srp-ipv6`
201: - `httptls+srp`
202: - `imap`
203: - `mqtt`
204: - `none`
205: - `pop3`
206: - `rtsp-ipv6`
207: - `rtsp`
208: - `scp`
209: - `sftp`
210: - `smtp`
211: - `socks4`
212: - `socks5`
213:
214: Give only one per line. This subsection is mandatory.
215:
216: ### `<features>`
217: A list of features that MUST be present in the client/library for this test to
218: be able to run. If a required feature is not present then the test will be
219: SKIPPED.
220:
221: Alternatively a feature can be prefixed with an exclamation mark to indicate a
222: feature is NOT required. If the feature is present then the test will be
223: SKIPPED.
224:
225: Features testable here are:
226:
227: - `alt-svc`
228: - `crypto`
229: - `debug`
230: - `getrlimit`
231: - `GnuTLS`
232: - `GSS-API`
233: - `http/2`
234: - `idn`
235: - `ipv6`
236: - `Kerberos`
237: - `large_file`
238: - `ld_preload`
239: - `libz`
240: - `manual`
241: - `Metalink`
242: - `NSS`
243: - `NTLM`
244: - `OpenSSL`
245: - `PSL`
246: - `socks`
247: - `SPNEGO`
248: - `SSL`
249: - `SSLpinning`
250: - `SSPI`
251: - `threaded-resolver`
252: - `TLS-SRP`
253: - `TrackMemory`
254: - `unittest`
255: - `unix-sockets`
256: - `win32`
257: - `WinSSL`
258:
259: as well as each protocol that curl supports. A protocol only needs to be
260: specified if it is different from the server (useful when the server
261: is `none`).
262:
263: ### `<killserver>`
264: Using the same syntax as in `<server>` but when mentioned here these servers
265: are explicitly KILLED when this test case is completed. Only use this if there
266: is no other alternatives. Using this of course requires subsequent tests to
267: restart servers.
268:
269: ### `<precheck>`
270: A command line that if set gets run by the test script before the test. If an
271: output is displayed by the command or if the return code is non-zero, the test
272: will be skipped and the (single-line) output will be displayed as reason for
273: not running the test. Variables are substituted as in the `<command>`
274: section.
275:
276: ### `<postcheck>`
277: A command line that if set gets run by the test script after the test. If
278: the command exists with a non-zero status code, the test will be considered
279: to have failed. Variables are substituted as in the `<command>` section.
280:
281: ### `<tool>`
282: Name of tool to invoke instead of "curl". This tool must be built and exist
283: either in the libtest/ directory (if the tool name starts with 'lib') or in
284: the unit/ directory (if the tool name starts with 'unit').
285:
286: ### `<name>`
287: Brief test case description, shown when the test runs.
288:
289: ### `<setenv>`
290: variable1=contents1
291: variable2=contents2
292:
293: Set the given environment variables to the specified value before the actual
294: command is run. They are cleared again after the command has been run.
295: Variables are first substituted as in the `<command>` section.
296: ### `<command [option="no-output/no-include/force-output/binary-trace"] [timeout="secs"][delay="secs"][type="perl"]>`
297: Command line to run. There's a bunch of %variables that get replaced
298: accordingly.
299:
300: Note that the URL that gets passed to the server actually controls what data
301: that is returned. The last slash in the URL must be followed by a number. That
302: number (N) will be used by the test-server to load test case N and return the
303: data that is defined within the `<reply><data></data></reply>` section.
304:
305: If there's no test number found above, the HTTP test server will use the
306: number following the last dot in the given hostname (made so that a CONNECT
307: can still pass on test number) so that "foo.bar.123" gets treated as test case
308: 123. Alternatively, if an IPv6 address is provided to CONNECT, the last
309: hexadecimal group in the address will be used as the test number! For example
310: the address "[1234::ff]" would be treated as test case 255.
311:
312: Set `type="perl"` to write the test case as a perl script. It implies that
313: there's no memory debugging and valgrind gets shut off for this test.
314:
315: Set `option="no-output"` to prevent the test script to slap on the `--output`
316: argument that directs the output to a file. The `--output` is also not added
317: if the verify/stdout section is used.
318:
319: Set `option="force-output"` to make use of `--output` even when the test is
320: otherwise written to verify stdout.
321:
322: Set `option="no-include"` to prevent the test script to slap on the
323: `--include` argument.
324:
325: Set `option="binary-trace"` to use `--trace` instead of `--trace-ascii` for
326: tracing. Suitable for binary-oriented protocols such as MQTT.
327:
328: Set `timeout="secs"` to override default server logs advisor read lock
329: timeout. This timeout is used by the test harness, once that the command has
330: completed execution, to wait for the test server to write out server side log
331: files and remove the lock that advised not to read them. The "secs" parameter
332: is the not negative integer number of seconds for the timeout. This `timeout`
333: attribute is documented for completeness sake, but is deep test harness stuff
334: and only needed for very singular and specific test cases. Avoid using it.
335:
336: Set `delay="secs"` to introduce a time delay once that the command has
337: completed execution and before the `<postcheck>` section runs. The "secs"
338: parameter is the not negative integer number of seconds for the delay. This
339: 'delay' attribute is intended for very specific test cases, and normally not
340: needed.
341:
342: Available substitute variables include:
343:
344: - `%CLIENT6IP` - IPv6 address of the client running curl
345: - `%CLIENTIP` - IPv4 address of the client running curl
346: - `%CURL` - Path to the curl executable
347: - `%FILE_PWD` - Current directory, on windows prefixed with a slash
348: - `%FTP2PORT` - Port number of the FTP server 2
349: - `%FTP6PORT` - IPv6 port number of the FTP server
350: - `%FTPPORT` - Port number of the FTP server
351: - `%FTPSPORT` - Port number of the FTPS server
352: - `%FTPTIME2` - Timeout in seconds that should be just sufficient to receive a response from the test FTP server
353: - `%FTPTIME3` - Even longer than %FTPTIME2
354: - `%GOPHER6PORT` - IPv6 port number of the Gopher server
355: - `%GOPHERPORT` - Port number of the Gopher server
356: - `%HOST6IP` - IPv6 address of the host running this test
357: - `%HOSTIP` - IPv4 address of the host running this test
358: - `%HTTP6PORT` - IPv6 port number of the HTTP server
359: - `%HTTPPORT` - Port number of the HTTP server
360: - `%HTTPSPORT` - Port number of the HTTPS server
361: - `%HTTPTLS6PORT` - IPv6 port number of the HTTP TLS server
362: - `%HTTPTLSPORT` - Port number of the HTTP TLS server
363: - `%HTTPUNIXPATH` - Path to the Unix socket of the HTTP server
364: - `%IMAP6PORT` - IPv6 port number of the IMAP server
365: - `%IMAPPORT` - Port number of the IMAP server
366: - `%MQTTPORT` - Port number of the MQTT server
367: - `%NEGTELNETPORT` - Port number of the telnet server
368: - `%NOLISTENPORT` - Port number where no service is listening
369: - `%POP36PORT` - IPv6 port number of the POP3 server
370: - `%POP3PORT` - Port number of the POP3 server
371: - `%POSIX_PWD` - Current directory somewhat mingw friendly
372: - `%PROXYPORT` - Port number of the HTTP proxy
373: - `%PWD` - Current directory
374: - `%RTSP6PORT` - IPv6 port number of the RTSP server
375: - `%RTSPPORT` - Port number of the RTSP server
376: - `%SMBPORT` - Port number of the SMB server
377: - `%SMBSPORT` - Port number of the SMBS server
378: - `%SMTP6PORT` - IPv6 port number of the SMTP server
379: - `%SMTPPORT` - Port number of the SMTP server
380: - `%SOCKSPORT` - Port number of the SOCKS4/5 server
381: - `%SRCDIR` - Full path to the source dir
382: - `%SSHPORT` - Port number of the SCP/SFTP server
383: - `%SSHSRVMD5` - MD5 of SSH server's public key
384: - `%TFTP6PORT` - IPv6 port number of the TFTP server
385: - `%TFTPPORT` - Port number of the TFTP server
386: - `%USER` - Login ID of the user running the test
387:
388: ### `<file name="log/filename">`
389: This creates the named file with this content before the test case is run,
390: which is useful if the test case needs a file to act on. Variables are
391: substituted on the contents of the file as in the `<command>` section.
392:
393: ### `<stdin [nonewline="yes"]>`
394: Pass this given data on stdin to the tool.
395:
396: If 'nonewline' is set, we will cut off the trailing newline of this given data
397: before comparing with the one actually received by the client
398:
399: ## `<verify>`
400: ### `<errorcode>`
401: numerical error code curl is supposed to return. Specify a list of accepted
402: error codes by separating multiple numbers with comma. See test 237 for an
403: example.
404:
405: ### `<strip>`
406: One regex per line that is removed from the protocol dumps before the
407: comparison is made. This is very useful to remove dependencies on dynamically
408: changing protocol data such as port numbers or user-agent strings.
409:
410: ### `<strippart>`
411: One perl op per line that operates on the protocol dump. This is pretty
412: advanced. Example: `s/^EPRT .*/EPRT stripped/`.
413:
414: ### `<protocol [nonewline="yes"]>`
415:
416: the protocol dump curl should transmit, if 'nonewline' is set, we will cut off
417: the trailing newline of this given data before comparing with the one actually
418: sent by the client Variables are substituted as in the `<command>` section.
419: The `<strip>` and `<strippart>` rules are applied before comparisons are made.
420:
421: ### `<proxy [nonewline="yes"]>`
422:
423: The protocol dump curl should transmit to a HTTP proxy (when the http-proxy
424: server is used), if 'nonewline' is set, we will cut off the trailing newline
425: of this given data before comparing with the one actually sent by the client
426: Variables are substituted as in the `<command>` section. The `<strip>` and
427: `<strippart>` rules are applied before comparisons are made.
428:
429: ### `<stdout [mode="text"] [nonewline="yes"]>`
430: This verifies that this data was passed to stdout. Variables are
431: substituted as in the `<command>` section.
432:
433: Use the mode="text" attribute if the output is in text mode on platforms that
434: have a text/binary difference.
435:
436: If 'nonewline' is set, we will cut off the trailing newline of this given data
437: before comparing with the one actually received by the client
438:
439: ### `<file name="log/filename" [mode="text"]>`
440: The file's contents must be identical to this after the test is complete. Use
441: the mode="text" attribute if the output is in text mode on platforms that have
442: a text/binary difference. Variables are substituted as in the `<command>`
443: section.
444:
445: ### `<file1>`
446: 1 to 4 can be appended to 'file' to compare more files.
447:
448: ### `<file2>`
449:
450: ### `<file3>`
451:
452: ### `<file4>`
453:
454: ### `<stripfile>`
455: One perl op per line that operates on the output file or stdout before being
456: compared with what is stored in the test file. This is pretty
457: advanced. Example: "s/^EPRT .*/EPRT stripped/"
458:
459: ### `<stripfile1>`
460: 1 to 4 can be appended to 'stripfile' to strip the corresponding <fileN>
461: content
462:
463: ### `<stripfile2>`
464:
465: ### `<stripfile3>`
466:
467: ### `<stripfile4>`
468:
469: ### `<upload>`
470: the contents of the upload data curl should have sent
471:
472: ### `<valgrind>`
473: disable - disables the valgrind log check for this test
FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>
![[BACK]](/icons/cvsweb/back.gif){kind=icon}
Return to FILEFORMAT.md CVS log ![[TXT]](/icons/cvsweb/text.gif){kind=icon}
![[DIR]](/icons/cvsweb/dir.gif){kind=icon}
Up to [ELWIX - Embedded LightWeight unIX -] / embedaddon / curl / tests