Return to libcurl-multi.3 CVS log

Up to [ELWIX - Embedded LightWeight unIX -] / embedaddon / curl / docs / libcurl

curl

    1: .\" **************************************************************************
    2: .\" *                                  _   _ ____  _
    3: .\" *  Project                     ___| | | |  _ \| |
    4: .\" *                             / __| | | | |_) | |
    5: .\" *                            | (__| |_| |  _ <| |___
    6: .\" *                             \___|\___/|_| \_\_____|
    7: .\" *
    8: .\" * Copyright (C) 1998 - 2020, Daniel Stenberg, <daniel@haxx.se>, et al.
    9: .\" *
   10: .\" * This software is licensed as described in the file COPYING, which
   11: .\" * you should have received as part of this distribution. The terms
   12: .\" * are also available at https://curl.haxx.se/docs/copyright.html.
   13: .\" *
   14: .\" * You may opt to use, copy, modify, merge, publish, distribute and/or sell
   15: .\" * copies of the Software, and permit persons to whom the Software is
   16: .\" * furnished to do so, under the terms of the COPYING file.
   17: .\" *
   18: .\" * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
   19: .\" * KIND, either express or implied.
   20: .\" *
   21: .\" **************************************************************************
   22: .\"
   23: .TH libcurl-multi 3 "April 26, 2020" "libcurl 7.70.0" "libcurl multi interface"
   24: 
   25: .SH NAME
   26: libcurl-multi \- how to use the multi interface
   27: .SH DESCRIPTION
   28: This is an overview on how to use the libcurl multi interface in your C
   29: programs. There are specific man pages for each function mentioned in
   30: here. There's also the \fIlibcurl-tutorial(3)\fP man page for a complete
   31: tutorial to programming with libcurl and the \fIlibcurl-easy(3)\fP man page
   32: for an overview of the libcurl easy interface.
   33: 
   34: All functions in the multi interface are prefixed with curl_multi.
   35: .SH "OBJECTIVES"
   36: The multi interface offers several abilities that the easy interface doesn't.
   37: They are mainly:
   38: 
   39: 1. Enable a "pull" interface. The application that uses libcurl decides where
   40: and when to ask libcurl to get/send data.
   41: 
   42: 2. Enable multiple simultaneous transfers in the same thread without making it
   43: complicated for the application.
   44: 
   45: 3. Enable the application to wait for action on its own file descriptors and
   46: curl's file descriptors simultaneously.
   47: 
   48: 4. Enable event-based handling and scaling transfers up to and beyond
   49: thousands of parallel connections.
   50: .SH "ONE MULTI HANDLE MANY EASY HANDLES"
   51: To use the multi interface, you must first create a 'multi handle' with
   52: \fIcurl_multi_init(3)\fP. This handle is then used as input to all further
   53: curl_multi_* functions.
   54: 
   55: With a multi handle and the multi interface you can do several simultaneous
   56: transfers in parallel. Each single transfer is built up around an easy
   57: handle. You create all the easy handles you need, and setup the appropriate
   58: options for each easy handle using \fIcurl_easy_setopt(3)\fP.
   59: 
   60: There are two flavours of the multi interface, the select() oriented one and
   61: the event based one we call multi_socket. You will benefit from reading
   62: through the description of both versions to fully understand how they work and
   63: differentiate. We start out with the select() oriented version.
   64: 
   65: When an easy handle is setup and ready for transfer, then instead of using
   66: \fIcurl_easy_perform(3)\fP like when using the easy interface for transfers,
   67: you should add the easy handle to the multi handle with
   68: \fIcurl_multi_add_handle(3)\fP. You can add more easy handles to a multi
   69: handle at any point, even if other transfers are already running.
   70: 
   71: Should you change your mind, the easy handle is again removed from the multi
   72: stack using \fIcurl_multi_remove_handle(3)\fP. Once removed from the multi
   73: handle, you can again use other easy interface functions like
   74: \fIcurl_easy_perform(3)\fP on the handle or whatever you think is
   75: necessary. You can remove handles at any point in time during transfers.
   76: 
   77: Adding the easy handle to the multi handle does not start the transfer.
   78: Remember that one of the main ideas with this interface is to let your
   79: application drive. You drive the transfers by invoking
   80: \fIcurl_multi_perform(3)\fP. libcurl will then transfer data if there is
   81: anything available to transfer. It'll use the callbacks and everything else
   82: you have setup in the individual easy handles. It'll transfer data on all
   83: current transfers in the multi stack that are ready to transfer anything. It
   84: may be all, it may be none. When there's nothing more to do for now, it
   85: returns back to the calling application.
   86: 
   87: Your application extracts info from libcurl about when it would like to get
   88: invoked to transfer data or do other work. The most convenient way is to use
   89: \fIcurl_multi_poll(3)\fP that will help you wait until the application should
   90: call libcurl again. The older API to accomplish the same thing is
   91: \fIcurl_multi_fdset(3)\fP that extracts fd_sets from libcurl to use in
   92: select() or poll() calls in order to get to know when the transfers in the
   93: multi stack might need attention. Both these APIs allow for your program to
   94: wait for input on your own private file descriptors at the same time.
   95: \fIcurl_multi_timeout(3)\fP also helps you with providing a suitable timeout
   96: period for your select() calls.
   97: 
   98: \fIcurl_multi_perform(3)\fP stores the number of still running transfers in
   99: one of its input arguments, and by reading that you can figure out when all
  100: the transfers in the multi handles are done. 'done' does not mean
  101: successful. One or more of the transfers may have failed. 
  102: 
  103: To get information about completed transfers, to figure out success or not and
  104: similar, \fIcurl_multi_info_read(3)\fP should be called. It can return a
  105: message about a current or previous transfer. Repeated invokes of the function
  106: get more messages until the message queue is empty. The information you
  107: receive there includes an easy handle pointer which you may use to identify
  108: which easy handle the information regards.
  109: 
  110: When a single transfer is completed, the easy handle is still left added to
  111: the multi stack. You need to first remove the easy handle with
  112: \fIcurl_multi_remove_handle(3)\fP and then close it with
  113: \fIcurl_easy_cleanup(3)\fP, or possibly set new options to it and add it again
  114: with \fIcurl_multi_add_handle(3)\fP to start another transfer.
  115: 
  116: When all transfers in the multi stack are done, close the multi handle with
  117: \fIcurl_multi_cleanup(3)\fP. Be careful and please note that you \fBMUST\fP
  118: invoke separate \fIcurl_easy_cleanup(3)\fP calls for every single easy handle
  119: to clean them up properly.
  120: 
  121: If you want to re-use an easy handle that was added to the multi handle for
  122: transfer, you must first remove it from the multi stack and then re-add it
  123: again (possibly after having altered some options at your own choice).
  124: .SH "MULTI_SOCKET"
  125: \fIcurl_multi_socket_action(3)\fP function offers a way for applications to
  126: not only avoid being forced to use select(), but it also offers a much more
  127: high-performance API that will make a significant difference for applications
  128: using large numbers of simultaneous connections.
  129: 
  130: \fIcurl_multi_socket_action(3)\fP is then used instead of
  131: \fIcurl_multi_perform(3)\fP.
  132: 
  133: When using this API, you add easy handles to the multi handle just as with the
  134: normal multi interface. Then you also set two callbacks with the
  135: \fICURLMOPT_SOCKETFUNCTION(3)\fP and \fICURLMOPT_TIMERFUNCTION(3)\fP options
  136: to \fIcurl_multi_setopt(3)\fP. They are two callback functions that libcurl
  137: will call with information about what sockets to wait for, and for what
  138: activity, and what the current timeout time is - if that expires libcurl
  139: should be notified.
  140: 
  141: The multi_socket API is designed to inform your application about which
  142: sockets libcurl is currently using and for what activities (read and/or write)
  143: on those sockets your application is expected to wait for.
  144: 
  145: Your application must make sure to receive all sockets informed about in the
  146: \fICURLMOPT_SOCKETFUNCTION(3)\fP callback and make sure it reacts on the given
  147: activity on them. When a socket has the given activity, you call
  148: \fIcurl_multi_socket_action(3)\fP specifying which socket and action there
  149: are.
  150: 
  151: The \fICURLMOPT_TIMERFUNCTION(3)\fP callback is called to set a timeout. When
  152: that timeout expires, your application should call the
  153: \fIcurl_multi_socket_action(3)\fP function saying it was due to a timeout.
  154: 
  155: This API is typically used with an event-driven underlying functionality (like
  156: libevent, libev, kqueue, epoll or similar) with which the application
  157: "subscribes" on socket changes. This allows applications and libcurl to much
  158: better scale upward and beyond thousands of simultaneous transfers without
  159: losing performance.
  160: 
  161: When you've added your initial set of handles, you call
  162: \fIcurl_multi_socket_action(3)\fP with CURL_SOCKET_TIMEOUT set in the sockfd
  163: argument, and you'll get callbacks call that sets you up and you then continue
  164: to call \fIcurl_multi_socket_action(3)\fP accordingly when you get activity on
  165: the sockets you've been asked to wait on, or if the timeout timer expires.
  166: 
  167: You can poll \fIcurl_multi_info_read(3)\fP to see if any transfer has
  168: completed, as it then has a message saying so.
  169: .SH "BLOCKING"
  170: A few areas in the code are still using blocking code, even when used from the
  171: multi interface. While we certainly want and intend for these to get fixed in
  172: the future, you should be aware of the following current restrictions:
  173: 
  174: .nf
  175:  - Name resolves unless the c-ares or threaded-resolver backends are used
  176:  - SOCKS proxy handshakes
  177:  - file:// transfers
  178:  - TELNET transfers
  179: .fi
  180: .SH "SEE ALSO"
  181: .BR libcurl-errors "(3), " libcurl-easy "(3), " libcurl "(3) "