Annotation of embedaddon/curl/docs/FAQ, revision 1.1
1.1 ! misho 1: _ _ ____ _
! 2: ___| | | | _ \| |
! 3: / __| | | | |_) | |
! 4: | (__| |_| | _ <| |___
! 5: \___|\___/|_| \_\_____|
! 6:
! 7: FAQ
! 8:
! 9: 1. Philosophy
! 10: 1.1 What is cURL?
! 11: 1.2 What is libcurl?
! 12: 1.3 What is curl not?
! 13: 1.4 When will you make curl do XXXX ?
! 14: 1.5 Who makes curl?
! 15: 1.6 What do you get for making curl?
! 16: 1.7 What about CURL from curl.com?
! 17: 1.8 I have a problem who do I mail?
! 18: 1.9 Where do I buy commercial support for curl?
! 19: 1.10 How many are using curl?
! 20: 1.11 Why don't you update ca-bundle.crt
! 21: 1.12 I have a problem who can I chat with?
! 22: 1.13 curl's ECCN number?
! 23: 1.14 How do I submit my patch?
! 24: 1.15 How do I port libcurl to my OS?
! 25:
! 26: 2. Install Related Problems
! 27: 2.1 configure doesn't find OpenSSL even when it is installed
! 28: 2.1.1 native linker doesn't find OpenSSL
! 29: 2.1.2 only the libssl lib is missing
! 30: 2.2 Does curl work/build with other SSL libraries?
! 31: 2.3 Where can I find a copy of LIBEAY32.DLL?
! 32: 2.4 Does curl support SOCKS (RFC 1928) ?
! 33:
! 34: 3. Usage Problems
! 35: 3.1 curl: (1) SSL is disabled, https: not supported
! 36: 3.2 How do I tell curl to resume a transfer?
! 37: 3.3 Why doesn't my posting using -F work?
! 38: 3.4 How do I tell curl to run custom FTP commands?
! 39: 3.5 How can I disable the Accept: */* header?
! 40: 3.6 Does curl support ASP, XML, XHTML or HTML version Y?
! 41: 3.7 Can I use curl to delete/rename a file through FTP?
! 42: 3.8 How do I tell curl to follow HTTP redirects?
! 43: 3.9 How do I use curl in my favorite programming language?
! 44: 3.10 What about SOAP, WebDAV, XML-RPC or similar protocols over HTTP?
! 45: 3.11 How do I POST with a different Content-Type?
! 46: 3.12 Why do FTP-specific features over HTTP proxy fail?
! 47: 3.13 Why do my single/double quotes fail?
! 48: 3.14 Does curl support Javascript or PAC (automated proxy config)?
! 49: 3.15 Can I do recursive fetches with curl?
! 50: 3.16 What certificates do I need when I use SSL?
! 51: 3.17 How do I list the root dir of an FTP server?
! 52: 3.18 Can I use curl to send a POST/PUT and not wait for a response?
! 53: 3.19 How do I get HTTP from a host using a specific IP address?
! 54: 3.20 How to SFTP from my user's home directory?
! 55: 3.21 Protocol xxx not supported or disabled in libcurl
! 56: 3.22 curl -X gives me HTTP problems
! 57:
! 58: 4. Running Problems
! 59: 4.1 Problems connecting to SSL servers.
! 60: 4.2 Why do I get problems when I use & or % in the URL?
! 61: 4.3 How can I use {, }, [ or ] to specify multiple URLs?
! 62: 4.4 Why do I get downloaded data even though the web page doesn't exist?
! 63: 4.5 Why do I get return code XXX from a HTTP server?
! 64: 4.5.1 "400 Bad Request"
! 65: 4.5.2 "401 Unauthorized"
! 66: 4.5.3 "403 Forbidden"
! 67: 4.5.4 "404 Not Found"
! 68: 4.5.5 "405 Method Not Allowed"
! 69: 4.5.6 "301 Moved Permanently"
! 70: 4.6 Can you tell me what error code 142 means?
! 71: 4.7 How do I keep user names and passwords secret in Curl command lines?
! 72: 4.8 I found a bug!
! 73: 4.9 Curl can't authenticate to the server that requires NTLM?
! 74: 4.10 My HTTP request using HEAD, PUT or DELETE doesn't work!
! 75: 4.11 Why do my HTTP range requests return the full document?
! 76: 4.12 Why do I get "certificate verify failed" ?
! 77: 4.13 Why is curl -R on Windows one hour off?
! 78: 4.14 Redirects work in browser but not with curl!
! 79: 4.15 FTPS doesn't work
! 80: 4.16 My HTTP POST or PUT requests are slow!
! 81: 4.17 Non-functional connect timeouts on Windows
! 82: 4.18 file:// URLs containing drive letters (Windows, NetWare)
! 83: 4.19 Why doesn't curl return an error when the network cable is unplugged?
! 84: 4.20 curl doesn't return error for HTTP non-200 responses!
! 85: 4.21 Why is there a HTTP/1.1 in my HTTP/2 request?
! 86:
! 87: 5. libcurl Issues
! 88: 5.1 Is libcurl thread-safe?
! 89: 5.2 How can I receive all data into a large memory chunk?
! 90: 5.3 How do I fetch multiple files with libcurl?
! 91: 5.4 Does libcurl do Winsock initing on win32 systems?
! 92: 5.5 Does CURLOPT_WRITEDATA and CURLOPT_READDATA work on win32 ?
! 93: 5.6 What about Keep-Alive or persistent connections?
! 94: 5.7 Link errors when building libcurl on Windows!
! 95: 5.8 libcurl.so.X: open failed: No such file or directory
! 96: 5.9 How does libcurl resolve host names?
! 97: 5.10 How do I prevent libcurl from writing the response to stdout?
! 98: 5.11 How do I make libcurl not receive the whole HTTP response?
! 99: 5.12 Can I make libcurl fake or hide my real IP address?
! 100: 5.13 How do I stop an ongoing transfer?
! 101: 5.14 Using C++ non-static functions for callbacks?
! 102: 5.15 How do I get an FTP directory listing?
! 103: 5.16 I want a different time-out!
! 104: 5.17 Can I write a server with libcurl?
! 105: 5.18 Does libcurl use threads?
! 106:
! 107: 6. License Issues
! 108: 6.1 I have a GPL program, can I use the libcurl library?
! 109: 6.2 I have a closed-source program, can I use the libcurl library?
! 110: 6.3 I have a BSD licensed program, can I use the libcurl library?
! 111: 6.4 I have a program that uses LGPL libraries, can I use libcurl?
! 112: 6.5 Can I modify curl/libcurl for my program and keep the changes secret?
! 113: 6.6 Can you please change the curl/libcurl license to XXXX?
! 114: 6.7 What are my obligations when using libcurl in my commercial apps?
! 115:
! 116: 7. PHP/CURL Issues
! 117: 7.1 What is PHP/CURL?
! 118: 7.2 Who wrote PHP/CURL?
! 119: 7.3 Can I perform multiple requests using the same handle?
! 120: 7.4 Does PHP/CURL have dependencies?
! 121:
! 122: ==============================================================================
! 123:
! 124: 1. Philosophy
! 125:
! 126: 1.1 What is cURL?
! 127:
! 128: cURL is the name of the project. The name is a play on 'Client for URLs',
! 129: originally with URL spelled in uppercase to make it obvious it deals with
! 130: URLs. The fact it can also be pronounced 'see URL' also helped, it works as
! 131: an abbreviation for "Client URL Request Library" or why not the recursive
! 132: version: "Curl URL Request Library".
! 133:
! 134: The cURL project produces two products:
! 135:
! 136: libcurl
! 137:
! 138: A free and easy-to-use client-side URL transfer library, supporting DICT,
! 139: FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3,
! 140: POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP.
! 141:
! 142: libcurl supports HTTPS certificates, HTTP POST, HTTP PUT, FTP uploading,
! 143: Kerberos, SPNEGO, HTTP form based upload, proxies, cookies, user+password
! 144: authentication, file transfer resume, http proxy tunneling and more!
! 145:
! 146: libcurl is highly portable, it builds and works identically on numerous
! 147: platforms, including Solaris, NetBSD, FreeBSD, OpenBSD, Darwin, HP-UX,
! 148: IRIX, AIX, Tru64, Linux, UnixWare, HURD, Windows, Amiga, OS/2, BeOS, Mac
! 149: OS X, Ultrix, QNX, OpenVMS, RISC OS, Novell NetWare, DOS, Symbian, OSF,
! 150: Android, Minix, IBM TPF and more...
! 151:
! 152: libcurl is free, thread-safe, IPv6 compatible, feature rich, well
! 153: supported and fast.
! 154:
! 155: curl
! 156:
! 157: A command line tool for getting or sending files using URL syntax.
! 158:
! 159: Since curl uses libcurl, curl supports the same wide range of common
! 160: Internet protocols that libcurl does.
! 161:
! 162: We pronounce curl with an initial k sound. It rhymes with words like girl
! 163: and earl. This is a short WAV file to help you:
! 164:
! 165: https://media.merriam-webster.com/soundc11/c/curl0001.wav
! 166:
! 167: There are numerous sub-projects and related projects that also use the word
! 168: curl in the project names in various combinations, but you should take
! 169: notice that this FAQ is directed at the command-line tool named curl (and
! 170: libcurl the library), and may therefore not be valid for other curl-related
! 171: projects. (There is however a small section for the PHP/CURL in this FAQ.)
! 172:
! 173: 1.2 What is libcurl?
! 174:
! 175: libcurl is a reliable and portable library which provides you with an easy
! 176: interface to a range of common Internet protocols.
! 177:
! 178: You can use libcurl for free in your application, be it open source,
! 179: commercial or closed-source.
! 180:
! 181: libcurl is most probably the most portable, most powerful and most often
! 182: used C-based multi-platform file transfer library on this planet - be it
! 183: open source or commercial.
! 184:
! 185: 1.3 What is curl not?
! 186:
! 187: Curl is not a wget clone. That is a common misconception. Never, during
! 188: curl's development, have we intended curl to replace wget or compete on its
! 189: market. Curl is targeted at single-shot file transfers.
! 190:
! 191: Curl is not a web site mirroring program. If you want to use curl to mirror
! 192: something: fine, go ahead and write a script that wraps around curl to make
! 193: it reality (like curlmirror.pl does).
! 194:
! 195: Curl is not an FTP site mirroring program. Sure, get and send FTP with curl
! 196: but if you want systematic and sequential behavior you should write a
! 197: script (or write a new program that interfaces libcurl) and do it.
! 198:
! 199: Curl is not a PHP tool, even though it works perfectly well when used from
! 200: or with PHP (when using the PHP/CURL module).
! 201:
! 202: Curl is not a program for a single operating system. Curl exists, compiles,
! 203: builds and runs under a wide range of operating systems, including all
! 204: modern Unixes (and a bunch of older ones too), Windows, Amiga, BeOS, OS/2,
! 205: OS X, QNX etc.
! 206:
! 207: 1.4 When will you make curl do XXXX ?
! 208:
! 209: We love suggestions of what to change in order to make curl and libcurl
! 210: better. We do however believe in a few rules when it comes to the future of
! 211: curl:
! 212:
! 213: Curl -- the command line tool -- is to remain a non-graphical command line
! 214: tool. If you want GUIs or fancy scripting capabilities, you should look for
! 215: another tool that uses libcurl.
! 216:
! 217: We do not add things to curl that other small and available tools already do
! 218: very well at the side. Curl's output can be piped into another program or
! 219: redirected to another file for the next program to interpret.
! 220:
! 221: We focus on protocol related issues and improvements. If you want to do more
! 222: magic with the supported protocols than curl currently does, chances are good
! 223: we will agree. If you want to add more protocols, we may very well agree.
! 224:
! 225: If you want someone else to do all the work while you wait for us to
! 226: implement it for you, that is not a very friendly attitude. We spend a
! 227: considerable time already on maintaining and developing curl. In order to
! 228: get more out of us, you should consider trading in some of your time and
! 229: effort in return. Simply go to the GitHub repo which resides at
! 230: https://github.com/curl/curl, fork the project, and create pull requests
! 231: with your proposed changes.
! 232:
! 233: If you write the code, chances are better that it will get into curl faster.
! 234:
! 235: 1.5 Who makes curl?
! 236:
! 237: curl and libcurl are not made by any single individual. Daniel Stenberg is
! 238: project leader and main developer, but other persons' submissions are
! 239: important and crucial. Anyone can contribute and post their changes and
! 240: improvements and have them inserted in the main sources (of course on the
! 241: condition that developers agree that the fixes are good).
! 242:
! 243: The full list of all contributors is found in the docs/THANKS file.
! 244:
! 245: curl is developed by a community, with Daniel at the wheel.
! 246:
! 247: 1.6 What do you get for making curl?
! 248:
! 249: Project cURL is entirely free and open. No person gets paid for developing
! 250: curl full time. We do this voluntarily, mostly in our spare time.
! 251: Occasionally companies pay individual developers to work on curl, but that's
! 252: up to each company and developer. This is not controlled by nor supervised in
! 253: any way by the project.
! 254:
! 255: We still get help from companies. Haxx provides web site, bandwidth, mailing
! 256: lists etc, GitHub hosts the primary git repository and other services like
! 257: the bug tracker at https://github.com/curl/curl. Also again, some companies
! 258: have sponsored certain parts of the development in the past and I hope some
! 259: will continue to do so in the future.
! 260:
! 261: If you want to support our project, consider a donation or a banner-program
! 262: or even better: by helping us with coding, documenting or testing etc.
! 263:
! 264: 1.7 What about CURL from curl.com?
! 265:
! 266: During the summer of 2001, curl.com was busy advertising their client-side
! 267: programming language for the web, named CURL.
! 268:
! 269: We are in no way associated with curl.com or their CURL programming
! 270: language.
! 271:
! 272: Our project name curl has been in effective use since 1998. We were not the
! 273: first computer related project to use the name "curl" and do not claim any
! 274: rights to the name.
! 275:
! 276: We recognize that we will be living in parallel with curl.com and wish them
! 277: every success.
! 278:
! 279: 1.8 I have a problem whom do I mail?
! 280:
! 281: Please do not mail any single individual unless you really need to. Keep
! 282: curl-related questions on a suitable mailing list. All available mailing
! 283: lists are listed in the MANUAL document and online at
! 284: https://curl.haxx.se/mail/
! 285:
! 286: Keeping curl-related questions and discussions on mailing lists allows
! 287: others to join in and help, to share their ideas, to contribute their
! 288: suggestions and to spread their wisdom. Keeping discussions on public mailing
! 289: lists also allows for others to learn from this (both current and future
! 290: users thanks to the web based archives of the mailing lists), thus saving us
! 291: from having to repeat ourselves even more. Thanks for respecting this.
! 292:
! 293: If you have found or simply suspect a security problem in curl or libcurl,
! 294: mail curl-security at haxx.se (closed list of receivers, mails are not
! 295: disclosed) and tell. Then we can produce a fix in a timely manner before the
! 296: flaw is announced to the world, thus lessen the impact the problem will have
! 297: on existing users.
! 298:
! 299: 1.9 Where do I buy commercial support for curl?
! 300:
! 301: curl is fully open source. It means you can hire any skilled engineer to fix
! 302: your curl-related problems.
! 303:
! 304: We list available alternatives on the curl web site:
! 305: https://curl.haxx.se/support.html
! 306:
! 307: 1.10 How many are using curl?
! 308:
! 309: It is impossible to tell.
! 310:
! 311: We don't know how many users that knowingly have installed and use curl.
! 312:
! 313: We don't know how many users that use curl without knowing that they are in
! 314: fact using it.
! 315:
! 316: We don't know how many users that downloaded or installed curl and then
! 317: never use it.
! 318:
! 319: In May 2012 Daniel did a counting game and came up with a number that may
! 320: be completely wrong or somewhat accurate. Over 500 million!
! 321:
! 322: See https://daniel.haxx.se/blog/2012/05/16/300m-users/
! 323:
! 324: 1.11 Why don't you update ca-bundle.crt
! 325:
! 326: The ca cert bundle that used to be shipped with curl was very outdated and
! 327: must be replaced with an up-to-date version by anyone who wants to verify
! 328: peers. It is no longer provided by curl. The last curl release that ever
! 329: shipped a ca cert bundle was curl 7.18.0.
! 330:
! 331: In the cURL project we've decided not to attempt to keep this file updated
! 332: (or even present anymore) since deciding what to add to a ca cert bundle is
! 333: an undertaking we've not been ready to accept, and the one we can get from
! 334: Mozilla is perfectly fine so there's no need to duplicate that work.
! 335:
! 336: Today, with many services performed over HTTPS, every operating system
! 337: should come with a default ca cert bundle that can be deemed somewhat
! 338: trustworthy and that collection (if reasonably updated) should be deemed to
! 339: be a lot better than a private curl version.
! 340:
! 341: If you want the most recent collection of ca certs that Mozilla Firefox
! 342: uses, we recommend that you extract the collection yourself from Mozilla
! 343: Firefox (by running 'make ca-bundle), or by using our online service setup
! 344: for this purpose: https://curl.haxx.se/docs/caextract.html
! 345:
! 346: 1.12 I have a problem who can I chat with?
! 347:
! 348: There's a bunch of friendly people hanging out in the #curl channel on the
! 349: IRC network irc.freenode.net. If you're polite and nice, chances are good
! 350: that you can get -- or provide -- help instantly.
! 351:
! 352: 1.13 curl's ECCN number?
! 353:
! 354: The US government restricts exports of software that contains or uses
! 355: cryptography. When doing so, the Export Control Classification Number (ECCN)
! 356: is used to identify the level of export control etc.
! 357:
! 358: Apache Software Foundation gives a good explanation of ECCNs at
! 359: https://www.apache.org/dev/crypto.html
! 360:
! 361: We believe curl's number might be ECCN 5D002, another possibility is
! 362: 5D992. It seems necessary to write them (the authority that administers ECCN
! 363: numbers), asking to confirm.
! 364:
! 365: Comprehensible explanations of the meaning of such numbers and how to obtain
! 366: them (resp.) are here
! 367:
! 368: https://www.bis.doc.gov/licensing/exportingbasics.htm
! 369: https://www.bis.doc.gov/licensing/do_i_needaneccn.html
! 370:
! 371: An incomprehensible description of the two numbers above is here
! 372: https://www.bis.doc.gov/index.php/documents/new-encryption/1653-ccl5-pt2-3
! 373:
! 374: 1.14 How do I submit my patch?
! 375:
! 376: When you have made a patch or a change of whatever sort, and want to submit
! 377: that to the project, there are a few different ways we prefer:
! 378:
! 379: o send a patch to the curl-library mailing list. We're many subscribers
! 380: there and there are lots of people who can review patches, comment on them
! 381: and "receive" them properly.
! 382:
! 383: o if your patch changes or fixes a bug, you can also opt to submit a bug
! 384: report in the bug tracker and attach your patch there. There are less
! 385: people involved there.
! 386:
! 387: Lots of more details are found in the CONTRIBUTE and INTERNALS docs.
! 388:
! 389: 1.15 How do I port libcurl to my OS?
! 390:
! 391: Here's a rough step-by-step:
! 392:
! 393: 1. copy a suitable lib/config-*.h file as a start to lib/config-[youros].h
! 394:
! 395: 2. edit lib/config-[youros].h to match your OS and setup
! 396:
! 397: 3. edit lib/curl_setup.h to include config-[youros].h when your OS is
! 398: detected by the preprocessor, in the style others already exist
! 399:
! 400: 4. compile lib/*.c and make them into a library
! 401:
! 402:
! 403: 2. Install Related Problems
! 404:
! 405: 2.1 configure doesn't find OpenSSL even when it is installed
! 406:
! 407: This may be because of several reasons.
! 408:
! 409: 2.1.1 native linker doesn't find openssl
! 410:
! 411: Affected platforms:
! 412: Solaris (native cc compiler)
! 413: HPUX (native cc compiler)
! 414: SGI IRIX (native cc compiler)
! 415: SCO UNIX (native cc compiler)
! 416:
! 417: When configuring curl, I specify --with-ssl. OpenSSL is installed in
! 418: /usr/local/ssl Configure reports SSL in /usr/local/ssl, but fails to find
! 419: CRYPTO_lock in -lcrypto
! 420:
! 421: Cause: The cc for this test places the -L/usr/local/ssl/lib AFTER
! 422: -lcrypto, so ld can't find the library. This is due to a bug in the GNU
! 423: autoconf tool.
! 424:
! 425: Workaround: Specifying "LDFLAGS=-L/usr/local/ssl/lib" in front of
! 426: ./configure places the -L/usr/local/ssl/lib early enough in the command
! 427: line to make things work
! 428:
! 429: 2.1.2 only the libssl lib is missing
! 430:
! 431: If all include files and the libcrypto lib is present, with only the
! 432: libssl being missing according to configure, this is most likely because
! 433: a few functions are left out from the libssl.
! 434:
! 435: If the function names missing include RSA or RSAREF you can be certain
! 436: that this is because libssl requires the RSA and RSAREF libs to build.
! 437:
! 438: See the INSTALL file section that explains how to add those libs to
! 439: configure. Make sure that you remove the config.cache file before you
! 440: rerun configure with the new flags.
! 441:
! 442: 2.2 Does curl work/build with other SSL libraries?
! 443:
! 444: Curl has been written to use a generic SSL function layer internally, and
! 445: that SSL functionality can then be provided by one out of many different SSL
! 446: backends.
! 447:
! 448: curl can be built to use one of the following SSL alternatives: OpenSSL,
! 449: libressl, BoringSSL, GnuTLS, wolfSSL, NSS, mbedTLS, MesaLink, Secure
! 450: Transport (native iOS/OS X), Schannel (native Windows), GSKit (native IBM
! 451: i), or BearSSL. They all have their pros and cons, and we try to maintain a
! 452: comparison of them here: https://curl.haxx.se/docs/ssl-compared.html
! 453:
! 454: 2.3 Where can I find a copy of LIBEAY32.DLL?
! 455:
! 456: That is an OpenSSL binary built for Windows.
! 457:
! 458: Curl can be built with OpenSSL to do the SSL stuff. The LIBEAY32.DLL is then
! 459: what curl needs on a windows machine to do https:// etc. Check out the curl
! 460: web site to find accurate and up-to-date pointers to recent OpenSSL DLLs and
! 461: other binary packages.
! 462:
! 463: 2.4 Does curl support SOCKS (RFC 1928) ?
! 464:
! 465: Yes, SOCKS 4 and 5 are supported.
! 466:
! 467:
! 468: 3. Usage problems
! 469:
! 470: 3.1 curl: (1) SSL is disabled, https: not supported
! 471:
! 472: If you get this output when trying to get anything from a https:// server,
! 473: it means that the instance of curl/libcurl that you're using was built
! 474: without support for this protocol.
! 475:
! 476: This could've happened if the configure script that was run at build time
! 477: couldn't find all libs and include files curl requires for SSL to work. If
! 478: the configure script fails to find them, curl is simply built without SSL
! 479: support.
! 480:
! 481: To get the https:// support into a curl that was previously built but that
! 482: reports that https:// is not supported, you should dig through the document
! 483: and logs and check out why the configure script doesn't find the SSL libs
! 484: and/or include files.
! 485:
! 486: Also, check out the other paragraph in this FAQ labeled "configure doesn't
! 487: find OpenSSL even when it is installed".
! 488:
! 489: 3.2 How do I tell curl to resume a transfer?
! 490:
! 491: Curl supports resumed transfers both ways on both FTP and HTTP.
! 492: Try the -C option.
! 493:
! 494: 3.3 Why doesn't my posting using -F work?
! 495:
! 496: You can't arbitrarily use -F or -d, the choice between -F or -d depends on the
! 497: HTTP operation you need curl to do and what the web server that will receive
! 498: your post expects.
! 499:
! 500: If the form you're trying to submit uses the type 'multipart/form-data', then
! 501: and only then you must use the -F type. In all the most common cases, you
! 502: should use -d which then causes a posting with the type
! 503: 'application/x-www-form-urlencoded'.
! 504:
! 505: This is described in some detail in the MANUAL and TheArtOfHttpScripting
! 506: documents, and if you don't understand it the first time, read it again
! 507: before you post questions about this to the mailing list. Also, try reading
! 508: through the mailing list archives for old postings and questions regarding
! 509: this.
! 510:
! 511: 3.4 How do I tell curl to run custom FTP commands?
! 512:
! 513: You can tell curl to perform optional commands both before and/or after a
! 514: file transfer. Study the -Q/--quote option.
! 515:
! 516: Since curl is used for file transfers, you don't normally use curl to
! 517: perform FTP commands without transferring anything. Therefore you must
! 518: always specify a URL to transfer to/from even when doing custom FTP
! 519: commands, or use -I which implies the "no body" option sent to libcurl.
! 520:
! 521: 3.5 How can I disable the Accept: */* header?
! 522:
! 523: You can change all internally generated headers by adding a replacement with
! 524: the -H/--header option. By adding a header with empty contents you safely
! 525: disable that one. Use -H "Accept:" to disable that specific header.
! 526:
! 527: 3.6 Does curl support ASP, XML, XHTML or HTML version Y?
! 528:
! 529: To curl, all contents are alike. It doesn't matter how the page was
! 530: generated. It may be ASP, PHP, Perl, shell-script, SSI or plain HTML
! 531: files. There's no difference to curl and it doesn't even know what kind of
! 532: language that generated the page.
! 533:
! 534: See also item 3.14 regarding javascript.
! 535:
! 536: 3.7 Can I use curl to delete/rename a file through FTP?
! 537:
! 538: Yes. You specify custom FTP commands with -Q/--quote.
! 539:
! 540: One example would be to delete a file after you have downloaded it:
! 541:
! 542: curl -O ftp://download.com/coolfile -Q '-DELE coolfile'
! 543:
! 544: or rename a file after upload:
! 545:
! 546: curl -T infile ftp://upload.com/dir/ -Q "-RNFR infile" -Q "-RNTO newname"
! 547:
! 548: 3.8 How do I tell curl to follow HTTP redirects?
! 549:
! 550: Curl does not follow so-called redirects by default. The Location: header
! 551: that informs the client about this is only interpreted if you're using the
! 552: -L/--location option. As in:
! 553:
! 554: curl -L http://redirector.com
! 555:
! 556: Not all redirects are HTTP ones, see 4.14
! 557:
! 558: 3.9 How do I use curl in my favorite programming language?
! 559:
! 560: Many programming languages have interfaces/bindings that allow you to use
! 561: curl without having to use the command line tool. If you are fluent in such
! 562: a language, you may prefer to use one of these interfaces instead.
! 563:
! 564: Find out more about which languages that support curl directly, and how to
! 565: install and use them, in the libcurl section of the curl web site:
! 566: https://curl.haxx.se/libcurl/
! 567:
! 568: All the various bindings to libcurl are made by other projects and people,
! 569: outside of the cURL project. The cURL project itself only produces libcurl
! 570: with its plain C API. If you don't find anywhere else to ask you can ask
! 571: about bindings on the curl-library list too, but be prepared that people on
! 572: that list may not know anything about bindings.
! 573:
! 574: In February 2019, there were interfaces available for the following
! 575: languages: Ada95, Basic, C, C++, Ch, Cocoa, D, Delphi, Dylan, Eiffel,
! 576: Euphoria, Falcon, Ferite, Gambas, glib/GTK+, Go, Guile, Harbour, Haskell,
! 577: Java, Julia, Lisp, Lua, Mono, .NET, node.js, Object-Pascal, OCaml, Pascal,
! 578: Perl, PHP, PostgreSQL, Python, R, Rexx, Ring, RPG, Ruby, Rust, Scheme,
! 579: Scilab, S-Lang, Smalltalk, SP-Forth, SPL, Tcl, Visual Basic, Visual FoxPro,
! 580: Q, wxwidgets, XBLite and Xoho. By the time you read this, additional ones
! 581: may have appeared!
! 582:
! 583: 3.10 What about SOAP, WebDAV, XML-RPC or similar protocols over HTTP?
! 584:
! 585: Curl adheres to the HTTP spec, which basically means you can play with *any*
! 586: protocol that is built on top of HTTP. Protocols such as SOAP, WEBDAV and
! 587: XML-RPC are all such ones. You can use -X to set custom requests and -H to
! 588: set custom headers (or replace internally generated ones).
! 589:
! 590: Using libcurl is of course just as good and you'd just use the proper
! 591: library options to do the same.
! 592:
! 593: 3.11 How do I POST with a different Content-Type?
! 594:
! 595: You can always replace the internally generated headers with -H/--header.
! 596: To make a simple HTTP POST with text/xml as content-type, do something like:
! 597:
! 598: curl -d "datatopost" -H "Content-Type: text/xml" [URL]
! 599:
! 600: 3.12 Why do FTP-specific features over HTTP proxy fail?
! 601:
! 602: Because when you use a HTTP proxy, the protocol spoken on the network will
! 603: be HTTP, even if you specify a FTP URL. This effectively means that you
! 604: normally can't use FTP-specific features such as FTP upload and FTP quote
! 605: etc.
! 606:
! 607: There is one exception to this rule, and that is if you can "tunnel through"
! 608: the given HTTP proxy. Proxy tunneling is enabled with a special option (-p)
! 609: and is generally not available as proxy admins usually disable tunneling to
! 610: ports other than 443 (which is used for HTTPS access through proxies).
! 611:
! 612: 3.13 Why do my single/double quotes fail?
! 613:
! 614: To specify a command line option that includes spaces, you might need to
! 615: put the entire option within quotes. Like in:
! 616:
! 617: curl -d " with spaces " url.com
! 618:
! 619: or perhaps
! 620:
! 621: curl -d ' with spaces ' url.com
! 622:
! 623: Exactly what kind of quotes and how to do this is entirely up to the shell
! 624: or command line interpreter that you are using. For most unix shells, you
! 625: can more or less pick either single (') or double (") quotes. For
! 626: Windows/DOS prompts I believe you're forced to use double (") quotes.
! 627:
! 628: Please study the documentation for your particular environment. Examples in
! 629: the curl docs will use a mix of both of these as shown above. You must
! 630: adjust them to work in your environment.
! 631:
! 632: Remember that curl works and runs on more operating systems than most single
! 633: individuals have ever tried.
! 634:
! 635: 3.14 Does curl support Javascript or PAC (automated proxy config)?
! 636:
! 637: Many web pages do magic stuff using embedded Javascript. Curl and libcurl
! 638: have no built-in support for that, so it will be treated just like any other
! 639: contents.
! 640:
! 641: .pac files are a netscape invention and are sometimes used by organizations
! 642: to allow them to differentiate which proxies to use. The .pac contents is
! 643: just a Javascript program that gets invoked by the browser and that returns
! 644: the name of the proxy to connect to. Since curl doesn't support Javascript,
! 645: it can't support .pac proxy configuration either.
! 646:
! 647: Some workarounds usually suggested to overcome this Javascript dependency:
! 648:
! 649: Depending on the Javascript complexity, write up a script that translates it
! 650: to another language and execute that.
! 651:
! 652: Read the Javascript code and rewrite the same logic in another language.
! 653:
! 654: Implement a Javascript interpreter, people have successfully used the
! 655: Mozilla Javascript engine in the past.
! 656:
! 657: Ask your admins to stop this, for a static proxy setup or similar.
! 658:
! 659: 3.15 Can I do recursive fetches with curl?
! 660:
! 661: No. curl itself has no code that performs recursive operations, such as
! 662: those performed by wget and similar tools.
! 663:
! 664: There exists wrapper scripts with that functionality (for example the
! 665: curlmirror perl script), and you can write programs based on libcurl to do
! 666: it, but the command line tool curl itself cannot.
! 667:
! 668: 3.16 What certificates do I need when I use SSL?
! 669:
! 670: There are three different kinds of "certificates" to keep track of when we
! 671: talk about using SSL-based protocols (HTTPS or FTPS) using curl or libcurl.
! 672:
! 673: CLIENT CERTIFICATE
! 674:
! 675: The server you communicate with may require that you can provide this in
! 676: order to prove that you actually are who you claim to be. If the server
! 677: doesn't require this, you don't need a client certificate.
! 678:
! 679: A client certificate is always used together with a private key, and the
! 680: private key has a pass phrase that protects it.
! 681:
! 682: SERVER CERTIFICATE
! 683:
! 684: The server you communicate with has a server certificate. You can and should
! 685: verify this certificate to make sure that you are truly talking to the real
! 686: server and not a server impersonating it.
! 687:
! 688: CERTIFICATE AUTHORITY CERTIFICATE ("CA cert")
! 689:
! 690: You often have several CA certs in a CA cert bundle that can be used to
! 691: verify a server certificate that was signed by one of the authorities in the
! 692: bundle. curl does not come with a CA cert bundle but most curl installs
! 693: provide one. You can also override the default.
! 694:
! 695: The server certificate verification process is made by using a Certificate
! 696: Authority certificate ("CA cert") that was used to sign the server
! 697: certificate. Server certificate verification is enabled by default in curl
! 698: and libcurl and is often the reason for problems as explained in FAQ entry
! 699: 4.12 and the SSLCERTS document
! 700: (https://curl.haxx.se/docs/sslcerts.html). Server certificates that are
! 701: "self-signed" or otherwise signed by a CA that you do not have a CA cert
! 702: for, cannot be verified. If the verification during a connect fails, you are
! 703: refused access. You then need to explicitly disable the verification to
! 704: connect to the server.
! 705:
! 706: 3.17 How do I list the root dir of an FTP server?
! 707:
! 708: There are two ways. The way defined in the RFC is to use an encoded slash
! 709: in the first path part. List the "/tmp" dir like this:
! 710:
! 711: curl ftp://ftp.sunet.se/%2ftmp/
! 712:
! 713: or the not-quite-kosher-but-more-readable way, by simply starting the path
! 714: section of the URL with a slash:
! 715:
! 716: curl ftp://ftp.sunet.se//tmp/
! 717:
! 718: 3.18 Can I use curl to send a POST/PUT and not wait for a response?
! 719:
! 720: No.
! 721:
! 722: But you could easily write your own program using libcurl to do such stunts.
! 723:
! 724: 3.19 How do I get HTTP from a host using a specific IP address?
! 725:
! 726: For example, you may be trying out a web site installation that isn't yet in
! 727: the DNS. Or you have a site using multiple IP addresses for a given host
! 728: name and you want to address a specific one out of the set.
! 729:
! 730: Set a custom Host: header that identifies the server name you want to reach
! 731: but use the target IP address in the URL:
! 732:
! 733: curl --header "Host: www.example.com" http://127.0.0.1/
! 734:
! 735: You can also opt to add faked host name entries to curl with the --resolve
! 736: option. That has the added benefit that things like redirects will also work
! 737: properly. The above operation would instead be done as:
! 738:
! 739: curl --resolve www.example.com:80:127.0.0.1 http://www.example.com/
! 740:
! 741: 3.20 How to SFTP from my user's home directory?
! 742:
! 743: Contrary to how FTP works, SFTP and SCP URLs specify the exact directory to
! 744: work with. It means that if you don't specify that you want the user's home
! 745: directory, you get the actual root directory.
! 746:
! 747: To specify a file in your user's home directory, you need to use the correct
! 748: URL syntax which for SFTP might look similar to:
! 749:
! 750: curl -O -u user:password sftp://example.com/~/file.txt
! 751:
! 752: and for SCP it is just a different protocol prefix:
! 753:
! 754: curl -O -u user:password scp://example.com/~/file.txt
! 755:
! 756: 3.21 Protocol xxx not supported or disabled in libcurl
! 757:
! 758: When passing on a URL to curl to use, it may respond that the particular
! 759: protocol is not supported or disabled. The particular way this error message
! 760: is phrased is because curl doesn't make a distinction internally of whether
! 761: a particular protocol is not supported (i.e. never got any code added that
! 762: knows how to speak that protocol) or if it was explicitly disabled. curl can
! 763: be built to only support a given set of protocols, and the rest would then
! 764: be disabled or not supported.
! 765:
! 766: Note that this error will also occur if you pass a wrongly spelled protocol
! 767: part as in "htpt://example.com" or as in the less evident case if you prefix
! 768: the protocol part with a space as in " http://example.com/".
! 769:
! 770: 3.22 curl -X gives me HTTP problems
! 771:
! 772: In normal circumstances, -X should hardly ever be used.
! 773:
! 774: By default you use curl without explicitly saying which request method to
! 775: use when the URL identifies a HTTP transfer. If you just pass in a URL like
! 776: "curl http://example.com" it will use GET. If you use -d or -F curl will use
! 777: POST, -I will cause a HEAD and -T will make it a PUT.
! 778:
! 779: If for whatever reason you're not happy with these default choices that curl
! 780: does for you, you can override those request methods by specifying -X
! 781: [WHATEVER]. This way you can for example send a DELETE by doing "curl -X
! 782: DELETE [URL]".
! 783:
! 784: It is thus pointless to do "curl -XGET [URL]" as GET would be used
! 785: anyway. In the same vein it is pointless to do "curl -X POST -d data
! 786: [URL]"... But you can make a fun and somewhat rare request that sends a
! 787: request-body in a GET request with something like "curl -X GET -d data
! 788: [URL]"
! 789:
! 790: Note that -X doesn't actually change curl's behavior as it only modifies the
! 791: actual string sent in the request, but that may of course trigger a
! 792: different set of events.
! 793:
! 794: Accordingly, by using -XPOST on a command line that for example would follow
! 795: a 303 redirect, you will effectively prevent curl from behaving
! 796: correctly. Be aware.
! 797:
! 798:
! 799: 4. Running Problems
! 800:
! 801: 4.1 Problems connecting to SSL servers.
! 802:
! 803: It took a very long time before we could sort out why curl had problems to
! 804: connect to certain SSL servers when using SSLeay or OpenSSL v0.9+. The
! 805: error sometimes showed up similar to:
! 806:
! 807: 16570:error:1407D071:SSL routines:SSL2_READ:bad mac decode:s2_pkt.c:233:
! 808:
! 809: It turned out to be because many older SSL servers don't deal with SSLv3
! 810: requests properly. To correct this problem, tell curl to select SSLv2 from
! 811: the command line (-2/--sslv2).
! 812:
! 813: There have also been examples where the remote server didn't like the SSLv2
! 814: request and instead you had to force curl to use SSLv3 with -3/--sslv3.
! 815:
! 816: 4.2 Why do I get problems when I use & or % in the URL?
! 817:
! 818: In general unix shells, the & symbol is treated specially and when used, it
! 819: runs the specified command in the background. To safely send the & as a part
! 820: of a URL, you should quote the entire URL by using single (') or double (")
! 821: quotes around it. Similar problems can also occur on some shells with other
! 822: characters, including ?*!$~(){}<>\|;`. When in doubt, quote the URL.
! 823:
! 824: An example that would invoke a remote CGI that uses &-symbols could be:
! 825:
! 826: curl 'http://www.altavista.com/cgi-bin/query?text=yes&q=curl'
! 827:
! 828: In Windows, the standard DOS shell treats the percent sign specially and you
! 829: need to use TWO percent signs for each single one you want to use in the
! 830: URL.
! 831:
! 832: If you want a literal percent sign to be part of the data you pass in a POST
! 833: using -d/--data you must encode it as '%25' (which then also needs the
! 834: percent sign doubled on Windows machines).
! 835:
! 836: 4.3 How can I use {, }, [ or ] to specify multiple URLs?
! 837:
! 838: Because those letters have a special meaning to the shell, to be used in
! 839: a URL specified to curl you must quote them.
! 840:
! 841: An example that downloads two URLs (sequentially) would be:
! 842:
! 843: curl '{curl,www}.haxx.se'
! 844:
! 845: To be able to use those characters as actual parts of the URL (without using
! 846: them for the curl URL "globbing" system), use the -g/--globoff option:
! 847:
! 848: curl -g 'www.site.com/weirdname[].html'
! 849:
! 850: 4.4 Why do I get downloaded data even though the web page doesn't exist?
! 851:
! 852: Curl asks remote servers for the page you specify. If the page doesn't exist
! 853: at the server, the HTTP protocol defines how the server should respond and
! 854: that means that headers and a "page" will be returned. That's simply how
! 855: HTTP works.
! 856:
! 857: By using the --fail option you can tell curl explicitly to not get any data
! 858: if the HTTP return code doesn't say success.
! 859:
! 860: 4.5 Why do I get return code XXX from a HTTP server?
! 861:
! 862: RFC2616 clearly explains the return codes. This is a short transcript. Go
! 863: read the RFC for exact details:
! 864:
! 865: 4.5.1 "400 Bad Request"
! 866:
! 867: The request could not be understood by the server due to malformed
! 868: syntax. The client SHOULD NOT repeat the request without modifications.
! 869:
! 870: 4.5.2 "401 Unauthorized"
! 871:
! 872: The request requires user authentication.
! 873:
! 874: 4.5.3 "403 Forbidden"
! 875:
! 876: The server understood the request, but is refusing to fulfill it.
! 877: Authorization will not help and the request SHOULD NOT be repeated.
! 878:
! 879: 4.5.4 "404 Not Found"
! 880:
! 881: The server has not found anything matching the Request-URI. No indication
! 882: is given of whether the condition is temporary or permanent.
! 883:
! 884: 4.5.5 "405 Method Not Allowed"
! 885:
! 886: The method specified in the Request-Line is not allowed for the resource
! 887: identified by the Request-URI. The response MUST include an Allow header
! 888: containing a list of valid methods for the requested resource.
! 889:
! 890: 4.5.6 "301 Moved Permanently"
! 891:
! 892: If you get this return code and an HTML output similar to this:
! 893:
! 894: <H1>Moved Permanently</H1> The document has moved <A
! 895: HREF="http://same_url_now_with_a_trailing_slash/">here</A>.
! 896:
! 897: it might be because you requested a directory URL but without the trailing
! 898: slash. Try the same operation again _with_ the trailing URL, or use the
! 899: -L/--location option to follow the redirection.
! 900:
! 901: 4.6 Can you tell me what error code 142 means?
! 902:
! 903: All curl error codes are described at the end of the man page, in the
! 904: section called "EXIT CODES".
! 905:
! 906: Error codes that are larger than the highest documented error code means
! 907: that curl has exited due to a crash. This is a serious error, and we
! 908: appreciate a detailed bug report from you that describes how we could go
! 909: ahead and repeat this!
! 910:
! 911: 4.7 How do I keep user names and passwords secret in Curl command lines?
! 912:
! 913: This problem has two sides:
! 914:
! 915: The first part is to avoid having clear-text passwords in the command line
! 916: so that they don't appear in 'ps' outputs and similar. That is easily
! 917: avoided by using the "-K" option to tell curl to read parameters from a file
! 918: or stdin to which you can pass the secret info. curl itself will also
! 919: attempt to "hide" the given password by blanking out the option - this
! 920: doesn't work on all platforms.
! 921:
! 922: To keep the passwords in your account secret from the rest of the world is
! 923: not a task that curl addresses. You could of course encrypt them somehow to
! 924: at least hide them from being read by human eyes, but that is not what
! 925: anyone would call security.
! 926:
! 927: Also note that regular HTTP (using Basic authentication) and FTP passwords
! 928: are sent as cleartext across the network. All it takes for anyone to fetch
! 929: them is to listen on the network. Eavesdropping is very easy. Use more secure
! 930: authentication methods (like Digest, Negotiate or even NTLM) or consider the
! 931: SSL-based alternatives HTTPS and FTPS.
! 932:
! 933: 4.8 I found a bug!
! 934:
! 935: It is not a bug if the behavior is documented. Read the docs first.
! 936: Especially check out the KNOWN_BUGS file, it may be a documented bug!
! 937:
! 938: If it is a problem with a binary you've downloaded or a package for your
! 939: particular platform, try contacting the person who built the package/archive
! 940: you have.
! 941:
! 942: If there is a bug, read the BUGS document first. Then report it as described
! 943: in there.
! 944:
! 945: 4.9 Curl can't authenticate to the server that requires NTLM?
! 946:
! 947: NTLM support requires OpenSSL, GnuTLS, mbedTLS, NSS, Secure Transport, or
! 948: Microsoft Windows libraries at build-time to provide this functionality.
! 949:
! 950: NTLM is a Microsoft proprietary protocol. Proprietary formats are evil. You
! 951: should not use such ones.
! 952:
! 953: 4.10 My HTTP request using HEAD, PUT or DELETE doesn't work!
! 954:
! 955: Many web servers allow or demand that the administrator configures the
! 956: server properly for these requests to work on the web server.
! 957:
! 958: Some servers seem to support HEAD only on certain kinds of URLs.
! 959:
! 960: To fully grasp this, try the documentation for the particular server
! 961: software you're trying to interact with. This is not anything curl can do
! 962: anything about.
! 963:
! 964: 4.11 Why do my HTTP range requests return the full document?
! 965:
! 966: Because the range may not be supported by the server, or the server may
! 967: choose to ignore it and return the full document anyway.
! 968:
! 969: 4.12 Why do I get "certificate verify failed" ?
! 970:
! 971: You invoke curl 7.10 or later to communicate on a https:// URL and get an
! 972: error back looking something similar to this:
! 973:
! 974: curl: (35) SSL: error:14090086:SSL routines:
! 975: SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
! 976:
! 977: Then it means that curl couldn't verify that the server's certificate was
! 978: good. Curl verifies the certificate using the CA cert bundle that comes with
! 979: the curl installation.
! 980:
! 981: To disable the verification (which makes it act like curl did before 7.10),
! 982: use -k. This does however enable man-in-the-middle attacks.
! 983:
! 984: If you get this failure but are having a CA cert bundle installed and used,
! 985: the server's certificate is not signed by one of the CA's in the bundle. It
! 986: might for example be self-signed. You then correct this problem by obtaining
! 987: a valid CA cert for the server. Or again, decrease the security by disabling
! 988: this check.
! 989:
! 990: Details are also in the SSLCERTS file in the release archives, found online
! 991: here: https://curl.haxx.se/docs/sslcerts.html
! 992:
! 993: 4.13 Why is curl -R on Windows one hour off?
! 994:
! 995: Since curl 7.53.0 this issue should be fixed as long as curl was built with
! 996: any modern compiler that allows for a 64-bit curl_off_t type. For older
! 997: compilers or prior curl versions it may set a time that appears one hour off.
! 998: This happens due to a flaw in how Windows stores and uses file modification
! 999: times and it is not easily worked around. For more details read this:
! 1000: https://www.codeproject.com/Articles/1144/Beating-the-Daylight-Savings-Time-bug-and-getting
! 1001:
! 1002: 4.14 Redirects work in browser but not with curl!
! 1003:
! 1004: curl supports HTTP redirects well (see item 3.8). Browsers generally support
! 1005: at least two other ways to perform redirects that curl does not:
! 1006:
! 1007: Meta tags. You can write a HTML tag that will cause the browser to redirect
! 1008: to another given URL after a certain time.
! 1009:
! 1010: Javascript. You can write a Javascript program embedded in a HTML page that
! 1011: redirects the browser to another given URL.
! 1012:
! 1013: There is no way to make curl follow these redirects. You must either
! 1014: manually figure out what the page is set to do, or write a script that parses
! 1015: the results and fetches the new URL.
! 1016:
! 1017: 4.15 FTPS doesn't work
! 1018:
! 1019: curl supports FTPS (sometimes known as FTP-SSL) both implicit and explicit
! 1020: mode.
! 1021:
! 1022: When a URL is used that starts with FTPS://, curl assumes implicit SSL on
! 1023: the control connection and will therefore immediately connect and try to
! 1024: speak SSL. FTPS:// connections default to port 990.
! 1025:
! 1026: To use explicit FTPS, you use a FTP:// URL and the --ftp-ssl option (or one
! 1027: of its related flavors). This is the most common method, and the one
! 1028: mandated by RFC4217. This kind of connection will then of course use the
! 1029: standard FTP port 21 by default.
! 1030:
! 1031: 4.16 My HTTP POST or PUT requests are slow!
! 1032:
! 1033: libcurl makes all POST and PUT requests (except for POST requests with a
! 1034: very tiny request body) use the "Expect: 100-continue" header. This header
! 1035: allows the server to deny the operation early so that libcurl can bail out
! 1036: before having to send any data. This is useful in authentication
! 1037: cases and others.
! 1038:
! 1039: However, many servers don't implement the Expect: stuff properly and if the
! 1040: server doesn't respond (positively) within 1 second libcurl will continue
! 1041: and send off the data anyway.
! 1042:
! 1043: You can disable libcurl's use of the Expect: header the same way you disable
! 1044: any header, using -H / CURLOPT_HTTPHEADER, or by forcing it to use HTTP 1.0.
! 1045:
! 1046: 4.17 Non-functional connect timeouts
! 1047:
! 1048: In most Windows setups having a timeout longer than 21 seconds make no
! 1049: difference, as it will only send 3 TCP SYN packets and no more. The second
! 1050: packet sent three seconds after the first and the third six seconds after
! 1051: the second. No more than three packets are sent, no matter how long the
! 1052: timeout is set.
! 1053:
! 1054: See option TcpMaxConnectRetransmissions on this page:
! 1055: https://support.microsoft.com/en-us/kb/175523/en-us
! 1056:
! 1057: Also, even on non-Windows systems there may run a firewall or anti-virus
! 1058: software or similar that accepts the connection but does not actually do
! 1059: anything else. This will make (lib)curl to consider the connection connected
! 1060: and thus the connect timeout won't trigger.
! 1061:
! 1062: 4.18 file:// URLs containing drive letters (Windows, NetWare)
! 1063:
! 1064: When using curl to try to download a local file, one might use a URL
! 1065: in this format:
! 1066:
! 1067: file://D:/blah.txt
! 1068:
! 1069: You'll find that even if D:\blah.txt does exist, curl returns a 'file
! 1070: not found' error.
! 1071:
! 1072: According to RFC 1738 (https://www.ietf.org/rfc/rfc1738.txt),
! 1073: file:// URLs must contain a host component, but it is ignored by
! 1074: most implementations. In the above example, 'D:' is treated as the
! 1075: host component, and is taken away. Thus, curl tries to open '/blah.txt'.
! 1076: If your system is installed to drive C:, that will resolve to 'C:\blah.txt',
! 1077: and if that doesn't exist you will get the not found error.
! 1078:
! 1079: To fix this problem, use file:// URLs with *three* leading slashes:
! 1080:
! 1081: file:///D:/blah.txt
! 1082:
! 1083: Alternatively, if it makes more sense, specify 'localhost' as the host
! 1084: component:
! 1085:
! 1086: file://localhost/D:/blah.txt
! 1087:
! 1088: In either case, curl should now be looking for the correct file.
! 1089:
! 1090: 4.19 Why doesn't curl return an error when the network cable is unplugged?
! 1091:
! 1092: Unplugging a cable is not an error situation. The TCP/IP protocol stack
! 1093: was designed to be fault tolerant, so even though there may be a physical
! 1094: break somewhere the connection shouldn't be affected, just possibly
! 1095: delayed. Eventually, the physical break will be fixed or the data will be
! 1096: re-routed around the physical problem through another path.
! 1097:
! 1098: In such cases, the TCP/IP stack is responsible for detecting when the
! 1099: network connection is irrevocably lost. Since with some protocols it is
! 1100: perfectly legal for the client to wait indefinitely for data, the stack may
! 1101: never report a problem, and even when it does, it can take up to 20 minutes
! 1102: for it to detect an issue. The curl option --keepalive-time enables
! 1103: keep-alive support in the TCP/IP stack which makes it periodically probe the
! 1104: connection to make sure it is still available to send data. That should
! 1105: reliably detect any TCP/IP network failure.
! 1106:
! 1107: But even that won't detect the network going down before the TCP/IP
! 1108: connection is established (e.g. during a DNS lookup) or using protocols that
! 1109: don't use TCP. To handle those situations, curl offers a number of timeouts
! 1110: on its own. --speed-limit/--speed-time will abort if the data transfer rate
! 1111: falls too low, and --connect-timeout and --max-time can be used to put an
! 1112: overall timeout on the connection phase or the entire transfer.
! 1113:
! 1114: A libcurl-using application running in a known physical environment (e.g.
! 1115: an embedded device with only a single network connection) may want to act
! 1116: immediately if its lone network connection goes down. That can be achieved
! 1117: by having the application monitor the network connection on its own using an
! 1118: OS-specific mechanism, then signaling libcurl to abort (see also item 5.13).
! 1119:
! 1120: 4.20 curl doesn't return error for HTTP non-200 responses!
! 1121:
! 1122: Correct. Unless you use -f (--fail).
! 1123:
! 1124: When doing HTTP transfers, curl will perform exactly what you're asking it
! 1125: to do and if successful it will not return an error. You can use curl to
! 1126: test your web server's "file not found" page (that gets 404 back), you can
! 1127: use it to check your authentication protected web pages (that gets a 401
! 1128: back) and so on.
! 1129:
! 1130: The specific HTTP response code does not constitute a problem or error for
! 1131: curl. It simply sends and delivers HTTP as you asked and if that worked,
! 1132: everything is fine and dandy. The response code is generally providing more
! 1133: higher level error information that curl doesn't care about. The error was
! 1134: not in the HTTP transfer.
! 1135:
! 1136: If you want your command line to treat error codes in the 400 and up range
! 1137: as errors and thus return a non-zero value and possibly show an error
! 1138: message, curl has a dedicated option for that: -f (CURLOPT_FAILONERROR in
! 1139: libcurl speak).
! 1140:
! 1141: You can also use the -w option and the variable %{response_code} to extract
! 1142: the exact response code that was returned in the response.
! 1143:
! 1144: 4.21 Why is there a HTTP/1.1 in my HTTP/2 request?
! 1145:
! 1146: If you use verbose to see the HTTP request when you send off a HTTP/2
! 1147: request, it will still say 1.1.
! 1148:
! 1149: The reason for this is that we first generate the request to send using the
! 1150: old 1.1 style and show that request in the verbose output, and then we
! 1151: convert it over to the binary header-compressed HTTP/2 style. The actual
! 1152: "1.1" part from that request is then not actually used in the transfer.
! 1153: The binary HTTP/2 headers are not human readable.
! 1154:
! 1155: 5. libcurl Issues
! 1156:
! 1157: 5.1 Is libcurl thread-safe?
! 1158:
! 1159: Yes.
! 1160:
! 1161: We have written the libcurl code specifically adjusted for multi-threaded
! 1162: programs. libcurl will use thread-safe functions instead of non-safe ones if
! 1163: your system has such. Note that you must never share the same handle in
! 1164: multiple threads.
! 1165:
! 1166: There may be some exceptions to thread safety depending on how libcurl was
! 1167: built. Please review the guidelines for thread safety to learn more:
! 1168: https://curl.haxx.se/libcurl/c/threadsafe.html
! 1169:
! 1170: 5.2 How can I receive all data into a large memory chunk?
! 1171:
! 1172: [ See also the examples/getinmemory.c source ]
! 1173:
! 1174: You are in full control of the callback function that gets called every time
! 1175: there is data received from the remote server. You can make that callback do
! 1176: whatever you want. You do not have to write the received data to a file.
! 1177:
! 1178: One solution to this problem could be to have a pointer to a struct that you
! 1179: pass to the callback function. You set the pointer using the
! 1180: CURLOPT_WRITEDATA option. Then that pointer will be passed to the callback
! 1181: instead of a FILE * to a file:
! 1182:
! 1183: /* imaginary struct */
! 1184: struct MemoryStruct {
! 1185: char *memory;
! 1186: size_t size;
! 1187: };
! 1188:
! 1189: /* imaginary callback function */
! 1190: size_t
! 1191: WriteMemoryCallback(void *ptr, size_t size, size_t nmemb, void *data)
! 1192: {
! 1193: size_t realsize = size * nmemb;
! 1194: struct MemoryStruct *mem = (struct MemoryStruct *)data;
! 1195:
! 1196: mem->memory = (char *)realloc(mem->memory, mem->size + realsize + 1);
! 1197: if (mem->memory) {
! 1198: memcpy(&(mem->memory[mem->size]), ptr, realsize);
! 1199: mem->size += realsize;
! 1200: mem->memory[mem->size] = 0;
! 1201: }
! 1202: return realsize;
! 1203: }
! 1204:
! 1205: 5.3 How do I fetch multiple files with libcurl?
! 1206:
! 1207: libcurl has excellent support for transferring multiple files. You should
! 1208: just repeatedly set new URLs with curl_easy_setopt() and then transfer it
! 1209: with curl_easy_perform(). The handle you get from curl_easy_init() is not
! 1210: only reusable, but you're even encouraged to reuse it if you can, as that
! 1211: will enable libcurl to use persistent connections.
! 1212:
! 1213: 5.4 Does libcurl do Winsock initialization on win32 systems?
! 1214:
! 1215: Yes, if told to in the curl_global_init() call.
! 1216:
! 1217: 5.5 Does CURLOPT_WRITEDATA and CURLOPT_READDATA work on win32 ?
! 1218:
! 1219: Yes, but you cannot open a FILE * and pass the pointer to a DLL and have
! 1220: that DLL use the FILE * (as the DLL and the client application cannot access
! 1221: each others' variable memory areas). If you set CURLOPT_WRITEDATA you must
! 1222: also use CURLOPT_WRITEFUNCTION as well to set a function that writes the
! 1223: file, even if that simply writes the data to the specified FILE *.
! 1224: Similarly, if you use CURLOPT_READDATA you must also specify
! 1225: CURLOPT_READFUNCTION.
! 1226:
! 1227: 5.6 What about Keep-Alive or persistent connections?
! 1228:
! 1229: curl and libcurl have excellent support for persistent connections when
! 1230: transferring several files from the same server. Curl will attempt to reuse
! 1231: connections for all URLs specified on the same command line/config file, and
! 1232: libcurl will reuse connections for all transfers that are made using the
! 1233: same libcurl handle.
! 1234:
! 1235: When you use the easy interface the connection cache is kept within the easy
! 1236: handle. If you instead use the multi interface, the connection cache will be
! 1237: kept within the multi handle and will be shared among all the easy handles
! 1238: that are used within the same multi handle.
! 1239:
! 1240: 5.7 Link errors when building libcurl on Windows!
! 1241:
! 1242: You need to make sure that your project, and all the libraries (both static
! 1243: and dynamic) that it links against, are compiled/linked against the same run
! 1244: time library.
! 1245:
! 1246: This is determined by the /MD, /ML, /MT (and their corresponding /M?d)
! 1247: options to the command line compiler. /MD (linking against MSVCRT dll) seems
! 1248: to be the most commonly used option.
! 1249:
! 1250: When building an application that uses the static libcurl library, you must
! 1251: add -DCURL_STATICLIB to your CFLAGS. Otherwise the linker will look for
! 1252: dynamic import symbols. If you're using Visual Studio, you need to instead
! 1253: add CURL_STATICLIB in the "Preprocessor Definitions" section.
! 1254:
! 1255: If you get linker error like "unknown symbol __imp__curl_easy_init ..." you
! 1256: have linked against the wrong (static) library. If you want to use the
! 1257: libcurl.dll and import lib, you don't need any extra CFLAGS, but use one of
! 1258: the import libraries below. These are the libraries produced by the various
! 1259: lib/Makefile.* files:
! 1260:
! 1261: Target: static lib. import lib for libcurl*.dll.
! 1262: -----------------------------------------------------------
! 1263: MingW: libcurl.a libcurldll.a
! 1264: MSVC (release): libcurl.lib libcurl_imp.lib
! 1265: MSVC (debug): libcurld.lib libcurld_imp.lib
! 1266: Borland: libcurl.lib libcurl_imp.lib
! 1267:
! 1268: 5.8 libcurl.so.X: open failed: No such file or directory
! 1269:
! 1270: This is an error message you might get when you try to run a program linked
! 1271: with a shared version of libcurl and your run-time linker (ld.so) couldn't
! 1272: find the shared library named libcurl.so.X. (Where X is the number of the
! 1273: current libcurl ABI, typically 3 or 4).
! 1274:
! 1275: You need to make sure that ld.so finds libcurl.so.X. You can do that
! 1276: multiple ways, and it differs somewhat between different operating systems,
! 1277: but they are usually:
! 1278:
! 1279: * Add an option to the linker command line that specify the hard-coded path
! 1280: the run-time linker should check for the lib (usually -R)
! 1281:
! 1282: * Set an environment variable (LD_LIBRARY_PATH for example) where ld.so
! 1283: should check for libs
! 1284:
! 1285: * Adjust the system's config to check for libs in the directory where you've
! 1286: put the dir (like Linux's /etc/ld.so.conf)
! 1287:
! 1288: 'man ld.so' and 'man ld' will tell you more details
! 1289:
! 1290: 5.9 How does libcurl resolve host names?
! 1291:
! 1292: libcurl supports a large a number of different name resolve functions. One
! 1293: of them is picked at build-time and will be used unconditionally. Thus, if
! 1294: you want to change name resolver function you must rebuild libcurl and tell
! 1295: it to use a different function.
! 1296:
! 1297: - The non-IPv6 resolver that can use one of four different host name resolve
! 1298: calls (depending on what your system supports):
! 1299:
! 1300: A - gethostbyname()
! 1301: B - gethostbyname_r() with 3 arguments
! 1302: C - gethostbyname_r() with 5 arguments
! 1303: D - gethostbyname_r() with 6 arguments
! 1304:
! 1305: - The IPv6-resolver that uses getaddrinfo()
! 1306:
! 1307: - The c-ares based name resolver that uses the c-ares library for resolves.
! 1308: Using this offers asynchronous name resolves.
! 1309:
! 1310: - The threaded resolver (default option on Windows). It uses:
! 1311:
! 1312: A - gethostbyname() on plain IPv4 hosts
! 1313: B - getaddrinfo() on IPv6 enabled hosts
! 1314:
! 1315: Also note that libcurl never resolves or reverse-lookups addresses given as
! 1316: pure numbers, such as 127.0.0.1 or ::1.
! 1317:
! 1318: 5.10 How do I prevent libcurl from writing the response to stdout?
! 1319:
! 1320: libcurl provides a default built-in write function that writes received data
! 1321: to stdout. Set the CURLOPT_WRITEFUNCTION to receive the data, or possibly
! 1322: set CURLOPT_WRITEDATA to a different FILE * handle.
! 1323:
! 1324: 5.11 How do I make libcurl not receive the whole HTTP response?
! 1325:
! 1326: You make the write callback (or progress callback) return an error and
! 1327: libcurl will then abort the transfer.
! 1328:
! 1329: 5.12 Can I make libcurl fake or hide my real IP address?
! 1330:
! 1331: No. libcurl operates on a higher level. Besides, faking IP address would
! 1332: imply sending IP packets with a made-up source address, and then you normally
! 1333: get a problem with receiving the packet sent back as they would then not be
! 1334: routed to you!
! 1335:
! 1336: If you use a proxy to access remote sites, the sites will not see your local
! 1337: IP address but instead the address of the proxy.
! 1338:
! 1339: Also note that on many networks NATs or other IP-munging techniques are used
! 1340: that makes you see and use a different IP address locally than what the
! 1341: remote server will see you coming from. You may also consider using
! 1342: https://www.torproject.org/ .
! 1343:
! 1344: 5.13 How do I stop an ongoing transfer?
! 1345:
! 1346: With the easy interface you make sure to return the correct error code from
! 1347: one of the callbacks, but none of them are instant. There is no function you
! 1348: can call from another thread or similar that will stop it immediately.
! 1349: Instead, you need to make sure that one of the callbacks you use returns an
! 1350: appropriate value that will stop the transfer. Suitable callbacks that you
! 1351: can do this with include the progress callback, the read callback and the
! 1352: write callback.
! 1353:
! 1354: If you're using the multi interface, you can also stop a transfer by
! 1355: removing the particular easy handle from the multi stack at any moment you
! 1356: think the transfer is done or when you wish to abort the transfer.
! 1357:
! 1358: 5.14 Using C++ non-static functions for callbacks?
! 1359:
! 1360: libcurl is a C library, it doesn't know anything about C++ member functions.
! 1361:
! 1362: You can overcome this "limitation" with relative ease using a static
! 1363: member function that is passed a pointer to the class:
! 1364:
! 1365: // f is the pointer to your object.
! 1366: static size_t YourClass::func(void *buffer, size_t sz, size_t n, void *f)
! 1367: {
! 1368: // Call non-static member function.
! 1369: static_cast<YourClass*>(f)->nonStaticFunction();
! 1370: }
! 1371:
! 1372: // This is how you pass pointer to the static function:
! 1373: curl_easy_setopt(hcurl, CURLOPT_WRITEFUNCTION, YourClass::func);
! 1374: curl_easy_setopt(hcurl, CURLOPT_WRITEDATA, this);
! 1375:
! 1376: 5.15 How do I get an FTP directory listing?
! 1377:
! 1378: If you end the FTP URL you request with a slash, libcurl will provide you
! 1379: with a directory listing of that given directory. You can also set
! 1380: CURLOPT_CUSTOMREQUEST to alter what exact listing command libcurl would use
! 1381: to list the files.
! 1382:
! 1383: The follow-up question tends to be how is a program supposed to parse the
! 1384: directory listing. How does it know what's a file and what's a dir and what's
! 1385: a symlink etc. If the FTP server supports the MLSD command then it will
! 1386: return data in a machine-readable format that can be parsed for type. The
! 1387: types are specified by RFC3659 section 7.5.1. If MLSD is not supported then
! 1388: you have to work with what you're given. The LIST output format is entirely
! 1389: at the server's own liking and the NLST output doesn't reveal any types and
! 1390: in many cases doesn't even include all the directory entries. Also, both LIST
! 1391: and NLST tend to hide unix-style hidden files (those that start with a dot)
! 1392: by default so you need to do "LIST -a" or similar to see them.
! 1393:
! 1394: Example - List only directories.
! 1395: ftp.funet.fi supports MLSD and ftp.kernel.org does not:
! 1396:
! 1397: curl -s ftp.funet.fi/pub/ -X MLSD | \
! 1398: perl -lne 'print if s/(?:^|;)type=dir;[^ ]+ (.+)$/$1/'
! 1399:
! 1400: curl -s ftp.kernel.org/pub/linux/kernel/ | \
! 1401: perl -lne 'print if s/^d[-rwx]{9}(?: +[^ ]+){7} (.+)$/$1/'
! 1402:
! 1403: If you need to parse LIST output in libcurl one such existing
! 1404: list parser is available at https://cr.yp.to/ftpparse.html Versions of
! 1405: libcurl since 7.21.0 also provide the ability to specify a wildcard to
! 1406: download multiple files from one FTP directory.
! 1407:
! 1408: 5.16 I want a different time-out!
! 1409:
! 1410: Time and time again users realize that CURLOPT_TIMEOUT and
! 1411: CURLOPT_CONNECTIMEOUT are not sufficiently advanced or flexible to cover all
! 1412: the various use cases and scenarios applications end up with.
! 1413:
! 1414: libcurl offers many more ways to time-out operations. A common alternative
! 1415: is to use the CURLOPT_LOW_SPEED_LIMIT and CURLOPT_LOW_SPEED_TIME options to
! 1416: specify the lowest possible speed to accept before to consider the transfer
! 1417: timed out.
! 1418:
! 1419: The most flexible way is by writing your own time-out logic and using
! 1420: CURLOPT_XFERINFOFUNCTION (perhaps in combination with other callbacks) and
! 1421: use that to figure out exactly when the right condition is met when the
! 1422: transfer should get stopped.
! 1423:
! 1424: 5.17 Can I write a server with libcurl?
! 1425:
! 1426: No. libcurl offers no functions or building blocks to build any kind of
! 1427: internet protocol server. libcurl is only a client-side library. For server
! 1428: libraries, you need to continue your search elsewhere but there exist many
! 1429: good open source ones out there for most protocols you could possibly want a
! 1430: server for. And there are really good stand-alone ones that have been tested
! 1431: and proven for many years. There's no need for you to reinvent them!
! 1432:
! 1433: 5.18 Does libcurl use threads?
! 1434:
! 1435: Put simply: no, libcurl will execute in the same thread you call it in. All
! 1436: callbacks will be called in the same thread as the one you call libcurl in.
! 1437:
! 1438: If you want to avoid your thread to be blocked by the libcurl call, you make
! 1439: sure you use the non-blocking API which will do transfers asynchronously -
! 1440: but still in the same single thread.
! 1441:
! 1442: libcurl will potentially internally use threads for name resolving, if it
! 1443: was built to work like that, but in those cases it'll create the child
! 1444: threads by itself and they will only be used and then killed internally by
! 1445: libcurl and never exposed to the outside.
! 1446:
! 1447: 6. License Issues
! 1448:
! 1449: Curl and libcurl are released under a MIT/X derivate license. The license is
! 1450: very liberal and should not impose a problem for your project. This section
! 1451: is just a brief summary for the cases we get the most questions. (Parts of
! 1452: this section was much enhanced by Bjorn Reese.)
! 1453:
! 1454: We are not lawyers and this is not legal advice. You should probably consult
! 1455: one if you want true and accurate legal insights without our prejudice. Note
! 1456: especially that this section concerns the libcurl license only; compiling in
! 1457: features of libcurl that depend on other libraries (e.g. OpenSSL) may affect
! 1458: the licensing obligations of your application.
! 1459:
! 1460: 6.1 I have a GPL program, can I use the libcurl library?
! 1461:
! 1462: Yes!
! 1463:
! 1464: Since libcurl may be distributed under the MIT/X derivate license, it can be
! 1465: used together with GPL in any software.
! 1466:
! 1467: 6.2 I have a closed-source program, can I use the libcurl library?
! 1468:
! 1469: Yes!
! 1470:
! 1471: libcurl does not put any restrictions on the program that uses the library.
! 1472:
! 1473: 6.3 I have a BSD licensed program, can I use the libcurl library?
! 1474:
! 1475: Yes!
! 1476:
! 1477: libcurl does not put any restrictions on the program that uses the library.
! 1478:
! 1479: 6.4 I have a program that uses LGPL libraries, can I use libcurl?
! 1480:
! 1481: Yes!
! 1482:
! 1483: The LGPL license doesn't clash with other licenses.
! 1484:
! 1485: 6.5 Can I modify curl/libcurl for my program and keep the changes secret?
! 1486:
! 1487: Yes!
! 1488:
! 1489: The MIT/X derivate license practically allows you to do almost anything with
! 1490: the sources, on the condition that the copyright texts in the sources are
! 1491: left intact.
! 1492:
! 1493: 6.6 Can you please change the curl/libcurl license to XXXX?
! 1494:
! 1495: No.
! 1496:
! 1497: We have carefully picked this license after years of development and
! 1498: discussions and a large amount of people have contributed with source code
! 1499: knowing that this is the license we use. This license puts the restrictions
! 1500: we want on curl/libcurl and it does not spread to other programs or
! 1501: libraries that use it. It should be possible for everyone to use libcurl or
! 1502: curl in their projects, no matter what license they already have in use.
! 1503:
! 1504: 6.7 What are my obligations when using libcurl in my commercial apps?
! 1505:
! 1506: Next to none. All you need to adhere to is the MIT-style license (stated in
! 1507: the COPYING file) which basically says you have to include the copyright
! 1508: notice in "all copies" and that you may not use the copyright holder's name
! 1509: when promoting your software.
! 1510:
! 1511: You do not have to release any of your source code.
! 1512:
! 1513: You do not have to reveal or make public any changes to the libcurl source
! 1514: code.
! 1515:
! 1516: You do not have to broadcast to the world that you are using libcurl within
! 1517: your app.
! 1518:
! 1519: All we ask is that you disclose "the copyright notice and this permission
! 1520: notice" somewhere. Most probably like in the documentation or in the section
! 1521: where other third party dependencies already are mentioned and acknowledged.
! 1522:
! 1523: As can be seen here: https://curl.haxx.se/docs/companies.html and elsewhere,
! 1524: more and more companies are discovering the power of libcurl and take
! 1525: advantage of it even in commercial environments.
! 1526:
! 1527:
! 1528: 7. PHP/CURL Issues
! 1529:
! 1530: 7.1 What is PHP/CURL?
! 1531:
! 1532: The module for PHP that makes it possible for PHP programs to access curl-
! 1533: functions from within PHP.
! 1534:
! 1535: In the cURL project we call this module PHP/CURL to differentiate it from
! 1536: curl the command line tool and libcurl the library. The PHP team however
! 1537: does not refer to it like this (for unknown reasons). They call it plain
! 1538: CURL (often using all caps) or sometimes ext/curl, but both cause much
! 1539: confusion to users which in turn gives us a higher question load.
! 1540:
! 1541: 7.2 Who wrote PHP/CURL?
! 1542:
! 1543: PHP/CURL was initially written by Sterling Hughes.
! 1544:
! 1545: 7.3 Can I perform multiple requests using the same handle?
! 1546:
! 1547: Yes - at least in PHP version 4.3.8 and later (this has been known to not
! 1548: work in earlier versions, but the exact version when it started to work is
! 1549: unknown to me).
! 1550:
! 1551: After a transfer, you just set new options in the handle and make another
! 1552: transfer. This will make libcurl re-use the same connection if it can.
! 1553:
! 1554: 7.4 Does PHP/CURL have dependencies?
! 1555:
! 1556: PHP/CURL is a module that comes with the regular PHP package. It depends on
! 1557: and uses libcurl, so you need to have libcurl installed properly before
! 1558: PHP/CURL can be used.
FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>
![[BACK]](/icons/cvsweb/back.gif){kind=icon}
Return to FAQ CVS log ![[TXT]](/icons/cvsweb/text.gif){kind=icon}
![[DIR]](/icons/cvsweb/dir.gif){kind=icon}
Up to [ELWIX - Embedded LightWeight unIX -] / embedaddon / curl / docs