{"schema":"libjg2-1", "vpath":"/git/", "avatar":"/git/avatar/", "alang":"", "gen_ut":1745908605, "reponame":"libwebsockets", "desc":"libwebsockets lightweight C networking library", "owner": { "name": "Andy Green", "email": "andy@warmcat.com", "md5": "c50933ca2aa61e0fe2c43d46bb6b59cb" },"url":"https://libwebsockets.org/repo/libwebsockets", "f":3, "items": [ {"schema":"libjg2-1", "cid":"c982fc294aab975b12d9cf557084c0ee", "commit": {"type":"commit", "time": 1522982283, "time_ofs": 480, "oid_tree": { "oid": "eded3c034c184299195c5aa28d97401e752ce05d", "alias": []}, "oid":{ "oid": "04e1661411554108e391a62774e551ea952ee64a", "alias": []}, "msg": "client: http1.1 pipelining", "sig_commit": { "git_time": { "time": 1522982283, "offset": 480 }, "name": "Andy Green", "email": "andy@warmcat.com", "md5": "c50933ca2aa61e0fe2c43d46bb6b59cb" }, "sig_author": { "git_time": { "time": 1522037104, "offset": 480 }, "name": "Andy Green", "email": "andy@warmcat.com", "md5": "c50933ca2aa61e0fe2c43d46bb6b59cb" }}, "body": "client: http1.1 pipelining" , "diff": "diff --git a/READMEs/README.coding.md b/READMEs/README.coding.md\nindex 9ac87e6..9866073 100644\n--- a/READMEs/README.coding.md\n+++ b/READMEs/README.coding.md\n@@ -1080,7 +1080,29 @@ prepare the client SSL context for the vhost after creating the vhost, since\n this is not normally done if the vhost was set up to listen / serve. Call\n the api lws_init_vhost_client_ssl() to also allow client SSL on the vhost.\n \n+@section clipipe Pipelining Client Requests to same host\n \n+If you are opening more client requests to the same host and port, you\n+can give the flag LCCSCF_PIPELINE on `info.ssl_connection` to indicate\n+you wish to pipeline them.\n+\n+Without the flag, the client connections will occur concurrently using a\n+socket and tls wrapper if requested for each connection individually.\n+That is fast, but resource-intensive.\n+\n+With the flag, lws will queue subsequent client connections on the first\n+connection to the same host and port. When it has confirmed from the\n+first connection that pipelining / keep-alive is supported by the server,\n+it lets the queued client pipeline connections send their headers ahead\n+of time to create a pipeline of requests on the server side.\n+\n+In this way only one tcp connection and tls wrapper is required to transfer\n+all the transactions sequentially. It takes a little longer but it\n+can make a significant difference to resources on both sides.\n+\n+If lws learns from the first response header that keepalive is not possible,\n+then it marks itself with that information and detaches any queued clients\n+to make their own individual connections as a fallback.\n \n @section vhosts Using lws vhosts\n \ndiff --git a/lib/client/client-handshake.c b/lib/client/client-handshake.c\nindex 70e3378..c3bc147 100644\n--- a/lib/client/client-handshake.c\n+++ b/lib/client/client-handshake.c\n@@ -28,15 +28,16 @@ lws_getaddrinfo46(struct lws *wsi, const char *ads, struct addrinfo **result)\n struct lws *\n lws_client_connect_2(struct lws *wsi)\n {\n-\tsockaddr46 sa46;\n-\tstruct addrinfo *result;\n \tstruct lws_context *context \u003d wsi-\u003econtext;\n \tstruct lws_context_per_thread *pt \u003d \u0026context-\u003ept[(int)wsi-\u003etsi];\n+\tconst char *cce \u003d \u0022\u0022, *iface, *adsin, *meth;\n+\tstruct lws *wsi_piggy \u003d NULL;\n+\tstruct addrinfo *result;\n \tstruct lws_pollfd pfd;\n-\tconst char *cce \u003d \u0022\u0022, *iface;\n-\tint n, port;\n \tssize_t plen \u003d 0;\n \tconst char *ads;\n+\tsockaddr46 sa46;\n+\tint n, port;\n #ifdef LWS_WITH_IPV6\n \tchar ipv6only \u003d lws_check_opt(wsi-\u003evhost-\u003eoptions,\n \t\t\tLWS_SERVER_OPTION_IPV6_V6ONLY_MODIFY |\n@@ -55,6 +56,61 @@ lws_client_connect_2(struct lws *wsi)\n \t\tgoto oom4;\n \t}\n \n+\t/* we can only piggyback GET */\n+\n+\tmeth \u003d lws_hdr_simple_ptr(wsi, _WSI_TOKEN_CLIENT_METHOD);\n+\tif (meth \u0026\u0026 strcmp(meth, \u0022GET\u0022))\n+\t\tgoto create_new_conn;\n+\n+\t/* we only pipeline connections that said it was okay */\n+\n+\tif (!wsi-\u003eclient_pipeline)\n+\t\tgoto create_new_conn;\n+\n+\t/*\n+\t * let's take a look first and see if there are any already-active\n+\t * client connections we can piggy-back on.\n+\t */\n+\n+\tadsin \u003d lws_hdr_simple_ptr(wsi, _WSI_TOKEN_CLIENT_PEER_ADDRESS);\n+\tlws_vhost_lock(wsi-\u003evhost);\n+\tlws_start_foreach_dll_safe(struct lws_dll_lws *, d, d1,\n+\t\t\t\t wsi-\u003evhost-\u003edll_active_client_conns.next) {\n+\t\tstruct lws *w \u003d lws_container_of(d, struct lws,\n+\t\t\t\t\t\t dll_active_client_conns);\n+\n+\t\tif (w-\u003eah \u0026\u0026 !strcmp(adsin, lws_hdr_simple_ptr(w,\n+\t\t\t _WSI_TOKEN_CLIENT_PEER_ADDRESS)) \u0026\u0026\n+\t\t wsi-\u003ec_port \u003d\u003d w-\u003ec_port) {\n+\t\t\t/* someone else is already connected to the right guy */\n+\n+\t\t\t/* do we know for a fact pipelining won't fly? */\n+\t\t\tif (w-\u003ekeepalive_rejected) {\n+\t\t\t\tlwsl_info(\u0022defeating pipelining due to no KA on server\u005cn\u0022);\n+\t\t\t\tgoto create_new_conn;\n+\t\t\t}\n+\t\t\t/*\n+\t\t\t * ...let's add ourselves to his transaction queue...\n+\t\t\t */\n+\t\t\tlws_dll_lws_add_front(\u0026wsi-\u003edll_client_transaction_queue,\n+\t\t\t\t\u0026w-\u003edll_client_transaction_queue_head);\n+\t\t\t/*\n+\t\t\t * pipeline our headers out on him, and wait for our\n+\t\t\t * turn at client transaction_complete to take over\n+\t\t\t * parsing the rx.\n+\t\t\t */\n+\n+\t\t\twsi_piggy \u003d w;\n+\n+\t\t\tlws_vhost_unlock(wsi-\u003evhost);\n+\t\t\tgoto send_hs;\n+\t\t}\n+\n+\t} lws_end_foreach_dll_safe(d, d1);\n+\tlws_vhost_unlock(wsi-\u003evhost);\n+\n+create_new_conn:\n+\n \t/*\n \t * start off allowing ipv6 on connection if vhost allows it\n \t */\n@@ -378,31 +434,71 @@ lws_client_connect_2(struct lws *wsi)\n \t}\n #endif\n \n-\t/*\n-\t * provoke service to issue the handshake directly\n-\t * we need to do it this way because in the proxy case, this is the\n-\t * next state and executed only if and when we get a good proxy\n-\t * response inside the state machine... but notice in SSL case this\n-\t * may not have sent anything yet with 0 return, and won't until some\n-\t * many retries from main loop. To stop that becoming endless,\n-\t * cover with a timeout.\n-\t */\n+send_hs:\n+\tif (!lws_dll_is_null(\u0026wsi-\u003edll_client_transaction_queue)) {\n+\t\t/*\n+\t\t * We are pipelining on an already-established connection...\n+\t\t * we can skip tls establishment.\n+\t\t */\n+\t\twsi-\u003emode \u003d LWSCM_WSCL_ISSUE_HANDSHAKE2;\n+\n+\t\t/*\n+\t\t * we can't send our headers directly, because they have to\n+\t\t * be sent when the parent is writeable. The parent will check\n+\t\t * for anybody on his client transaction queue that is in\n+\t\t * LWSCM_WSCL_ISSUE_HANDSHAKE2, and let them write.\n+\t\t *\n+\t\t * If we are trying to do this too early, before the master\n+\t\t * connection has written his own headers,\n+\t\t */\n+\t\tlws_callback_on_writable(wsi_piggy);\n+\t\tlwsl_debug(\u0022wsi %p: waiting to send headers\u005cn\u0022, wsi);\n+\t} else {\n+\t\t/* we are making our own connection */\n+\t\twsi-\u003emode \u003d LWSCM_WSCL_ISSUE_HANDSHAKE;\n+\n+\t\t/*\n+\t\t * provoke service to issue the handshake directly.\n+\t\t *\n+\t\t * we need to do it this way because in the proxy case, this is\n+\t\t * the next state and executed only if and when we get a good\n+\t\t * proxy response inside the state machine... but notice in\n+\t\t * SSL case this may not have sent anything yet with 0 return,\n+\t\t * and won't until many retries from main loop. To stop that\n+\t\t * becoming endless, cover with a timeout.\n+\t\t */\n \n-\tlws_set_timeout(wsi, PENDING_TIMEOUT_SENT_CLIENT_HANDSHAKE,\n-\t\t\tAWAITING_TIMEOUT);\n+\t\tlws_set_timeout(wsi, PENDING_TIMEOUT_SENT_CLIENT_HANDSHAKE,\n+\t\t\t\tAWAITING_TIMEOUT);\n \n-\twsi-\u003emode \u003d LWSCM_WSCL_ISSUE_HANDSHAKE;\n-\tpfd.fd \u003d wsi-\u003edesc.sockfd;\n-\tpfd.events \u003d LWS_POLLIN;\n-\tpfd.revents \u003d LWS_POLLIN;\n+\t\tpfd.fd \u003d wsi-\u003edesc.sockfd;\n+\t\tpfd.events \u003d LWS_POLLIN;\n+\t\tpfd.revents \u003d LWS_POLLIN;\n \n-\tn \u003d lws_service_fd(context, \u0026pfd);\n-\tif (n \u003c 0) {\n-\t\tcce \u003d \u0022first service failed\u0022;\n-\t\tgoto failed;\n+\t\tn \u003d lws_service_fd(context, \u0026pfd);\n+\t\tif (n \u003c 0) {\n+\t\t\tcce \u003d \u0022first service failed\u0022;\n+\t\t\tgoto failed;\n+\t\t}\n+\t\tif (n) /* returns 1 on failure after closing wsi */\n+\t\t\treturn NULL;\n+\t}\n+\n+\t/*\n+\t * If we made our own connection, and we're doing a method that can take\n+\t * a pipeline, we are an \u0022active client connection\u0022.\n+\t *\n+\t * Add ourselves to the vhost list of those so that others can\n+\t * piggyback on our transaction queue\n+\t */\n+\n+\tif (meth \u0026\u0026 !strcmp(meth, \u0022GET\u0022) \u0026\u0026\n+\t lws_dll_is_null(\u0026wsi-\u003edll_client_transaction_queue)) {\n+\t\tlws_vhost_lock(wsi-\u003evhost);\n+\t\tlws_dll_lws_add_front(\u0026wsi-\u003edll_active_client_conns,\n+\t\t\t\t \u0026wsi-\u003evhost-\u003edll_active_client_conns);\n+\t\tlws_vhost_unlock(wsi-\u003evhost);\n \t}\n-\tif (n) /* returns 1 on failure after closing wsi */\n-\t\treturn NULL;\n \n \treturn wsi;\n \n@@ -452,7 +548,8 @@ LWS_VISIBLE struct lws *\n lws_client_reset(struct lws **pwsi, int ssl, const char *address, int port,\n \t\t const char *path, const char *host)\n {\n-\tchar origin[300] \u003d \u0022\u0022, protocol[300] \u003d \u0022\u0022, method[32] \u003d \u0022\u0022, iface[16] \u003d \u0022\u0022, *p;\n+\tchar origin[300] \u003d \u0022\u0022, protocol[300] \u003d \u0022\u0022, method[32] \u003d \u0022\u0022,\n+\t iface[16] \u003d \u0022\u0022, *p;\n \tstruct lws *wsi \u003d *pwsi;\n \n \tif (wsi-\u003eredirects \u003d\u003d 3) {\n@@ -778,6 +875,7 @@ lws_client_connect_via_info(struct lws_client_connect_info *i)\n \t}\n \n \twsi-\u003eprotocol \u003d \u0026wsi-\u003evhost-\u003eprotocols[0];\n+\twsi-\u003eclient_pipeline \u003d !!(i-\u003essl_connection \u0026 LCCSCF_PIPELINE);\n \n \t/*\n \t * 1) for http[s] connection, allow protocol selection by name\ndiff --git a/lib/client/client.c b/lib/client/client.c\nindex f368da4..3827d3e 100644\n--- a/lib/client/client.c\n+++ b/lib/client/client.c\n@@ -77,12 +77,67 @@ lws_client_http_body_pending(struct lws *wsi, int something_left_to_send)\n \twsi-\u003eclient_http_body_pending \u003d !!something_left_to_send;\n }\n \n+/*\n+ * return self, or queued client wsi we are acting on behalf of\n+ */\n+\n+struct lws *\n+lws_client_wsi_effective(struct lws *wsi)\n+{\n+\tstruct lws *wsi_eff \u003d wsi;\n+\n+\tif (!wsi-\u003etransaction_from_pipeline_queue ||\n+\t !wsi-\u003edll_client_transaction_queue_head.next)\n+\t\treturn wsi;\n+\n+\t/*\n+\t * The head is the last queued transaction... so\n+\t * the guy we are fulfilling here is the tail\n+\t */\n+\n+\tlws_vhost_lock(wsi-\u003evhost);\n+\tlws_start_foreach_dll_safe(struct lws_dll_lws *, d, d1,\n+\t\t\t\t wsi-\u003edll_client_transaction_queue_head.next) {\n+\t\tif (d-\u003enext \u003d\u003d NULL)\n+\t\t\twsi_eff \u003d lws_container_of(d, struct lws,\n+\t\t\t\t\tdll_client_transaction_queue);\n+\t} lws_end_foreach_dll_safe(d, d1);\n+\tlws_vhost_unlock(wsi-\u003evhost);\n+\n+\treturn wsi_eff;\n+}\n+\n+/*\n+ * return self or the guy we are queued under\n+ */\n+\n+struct lws *\n+lws_client_wsi_master(struct lws *wsi)\n+{\n+\tstruct lws *wsi_eff \u003d wsi;\n+\tstruct lws_dll_lws *d;\n+\n+\tlws_vhost_lock(wsi-\u003evhost);\n+\td \u003d wsi-\u003edll_client_transaction_queue.prev;\n+\twhile (d) {\n+\t\twsi_eff \u003d lws_container_of(d, struct lws,\n+\t\t\t\t\tdll_client_transaction_queue_head);\n+\n+\t\td \u003d d-\u003eprev;\n+\t}\n+\tlws_vhost_unlock(wsi-\u003evhost);\n+\n+\treturn wsi_eff;\n+}\n+\n int\n-lws_client_socket_service(struct lws_context *context, struct lws *wsi,\n-\t\t\t struct lws_pollfd *pollfd)\n+lws_client_socket_service(struct lws *wsi, struct lws_pollfd *pollfd,\n+\t\t\t struct lws *wsi_conn)\n {\n+\tstruct lws_context *context \u003d wsi-\u003econtext;\n \tstruct lws_context_per_thread *pt \u003d \u0026context-\u003ept[(int)wsi-\u003etsi];\n \tchar *p \u003d (char *)\u0026pt-\u003eserv_buf[0];\n+\tstruct lws *w;\n #if defined(LWS_OPENSSL_SUPPORT)\n \tchar ebuf[128];\n #endif\n@@ -95,6 +150,35 @@ lws_client_socket_service(struct lws_context *context, struct lws *wsi,\n \tchar conn_mode \u003d 0, pending_timeout \u003d 0;\n #endif\n \n+\tif ((pollfd-\u003erevents \u0026 LWS_POLLOUT) \u0026\u0026\n+\t wsi-\u003ekeepalive_active \u0026\u0026\n+\t wsi-\u003edll_client_transaction_queue_head.next) {\n+\n+\t\tlwsl_debug(\u0022%s: pollout HANDSHAKE2\u005cn\u0022, __func__);\n+\n+\t\t/* we have a transaction queue that wants to pipeline */\n+\t\tlws_vhost_lock(wsi-\u003evhost);\n+\t\tlws_start_foreach_dll_safe(struct lws_dll_lws *, d, d1,\n+\t\t\t\t\t wsi-\u003edll_client_transaction_queue_head.next) {\n+\t\t\tstruct lws *w \u003d lws_container_of(d, struct lws,\n+\t\t\t\t\t\t dll_client_transaction_queue);\n+\n+\t\t\tif (w-\u003emode \u003d\u003d LWSCM_WSCL_ISSUE_HANDSHAKE2) {\n+\t\t\t\t/*\n+\t\t\t\t * pollfd has the master sockfd in it... we\n+\t\t\t\t * need to use that in HANDSHAKE2 to understand\n+\t\t\t\t * which wsi to actually write on\n+\t\t\t\t */\n+\t\t\t\tlws_client_socket_service(w, pollfd, wsi);\n+\t\t\t\tlws_callback_on_writable(wsi);\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t} lws_end_foreach_dll_safe(d, d1);\n+\t\tlws_vhost_unlock(wsi-\u003evhost);\n+\n+\t\treturn 0;\n+\t}\n+\n \tswitch (wsi-\u003emode) {\n \n \tcase LWSCM_WSCL_WAITING_CONNECT:\n@@ -320,7 +404,11 @@ start_ws_handshake:\n \t\t/* send our request to the server */\n \t\tlws_latency_pre(context, wsi);\n \n-\t\tn \u003d lws_ssl_capable_write(wsi, (unsigned char *)sb, (int)(p - sb));\n+\t\tw \u003d lws_client_wsi_master(wsi);\n+\t\tlwsl_debug(\u0022%s: HANDSHAKE2: %p: sending headers on %p\u005cn\u0022,\n+\t\t\t\t__func__, wsi, w);\n+\n+\t\tn \u003d lws_ssl_capable_write(w, (unsigned char *)sb, (int)(p - sb));\n \t\tlws_latency(context, wsi, \u0022send lws_issue_raw\u0022, n,\n \t\t\t n \u003d\u003d p - sb);\n \t\tswitch (n) {\n@@ -342,6 +430,8 @@ start_ws_handshake:\n \t\t\tbreak;\n \t\t}\n \n+\t\tlws_callback_on_writable(w);\n+\n \t\tgoto client_http_body_sent;\n \n \tcase LWSCM_WSCL_ISSUE_HTTP_BODY:\n@@ -353,6 +443,7 @@ start_ws_handshake:\n \t\t\tbreak;\n \t\t}\n client_http_body_sent:\n+\t\t/* prepare ourselves to do the parsing */\n \t\twsi-\u003eah-\u003eparser_state \u003d WSI_TOKEN_NAME_PART;\n \t\twsi-\u003eah-\u003elextable_pos \u003d 0;\n \t\twsi-\u003emode \u003d LWSCM_WSCL_WAITING_SERVER_REPLY;\n@@ -477,41 +568,89 @@ strtolower(char *s)\n int LWS_WARN_UNUSED_RESULT\n lws_http_transaction_completed_client(struct lws *wsi)\n {\n-\tlwsl_debug(\u0022%s: wsi %p\u005cn\u0022, __func__, wsi);\n-\t/* if we can't go back to accept new headers, drop the connection */\n-\tif (wsi-\u003ehttp.connection_type !\u003d HTTP_CONNECTION_KEEP_ALIVE) {\n-\t\tlwsl_info(\u0022%s: %p: close connection\u005cn\u0022, __func__, wsi);\n-\t\treturn 1;\n+\tstruct lws *wsi_eff \u003d lws_client_wsi_effective(wsi);\n+\n+\tlwsl_info(\u0022%s: wsi: %p, wsi_eff: %p\u005cn\u0022, __func__, wsi, wsi_eff);\n+\n+\tif (user_callback_handle_rxflow(wsi_eff-\u003eprotocol-\u003ecallback,\n+\t\t\twsi_eff, LWS_CALLBACK_COMPLETED_CLIENT_HTTP,\n+\t\t\twsi_eff-\u003euser_space, NULL, 0)) {\n+\t\tlwsl_debug(\u0022%s: Completed call returned nonzero (mode %d)\u005cn\u0022,\n+\t\t\t\t\t\t__func__, wsi_eff-\u003emode);\n+\t\treturn -1;\n \t}\n \n-\t/* we don't support chained client connections yet */\n-\treturn 1;\n-#if 0\n+\t/*\n+\t * Are we constitutionally capable of having a queue, ie, we are on\n+\t * the \u0022active client connections\u0022 list?\n+\t *\n+\t * If not, that's it for us.\n+\t */\n+\n+\tif (lws_dll_is_null(\u0026wsi-\u003edll_active_client_conns))\n+\t\treturn -1;\n+\n+\t/* if this was a queued guy, close him and remove from queue */\n+\n+\tif (wsi-\u003etransaction_from_pipeline_queue) {\n+\t\tlwsl_debug(\u0022closing queued wsi %p\u005cn\u0022, wsi_eff);\n+\t\t/* so the close doesn't trigger a CCE */\n+\t\twsi_eff-\u003ealready_did_cce \u003d 1;\n+\t\t__lws_close_free_wsi(wsi_eff,\n+\t\t\tLWS_CLOSE_STATUS_CLIENT_TRANSACTION_DONE,\n+\t\t\t\u0022queued client done\u0022);\n+\t}\n+\n+\t/* after the first one, they can only be coming from the queue */\n+\twsi-\u003etransaction_from_pipeline_queue \u003d 1;\n+\n+\t/* is there a new tail after removing that one? */\n+\twsi_eff \u003d lws_client_wsi_effective(wsi);\n+\n+\t/*\n+\t * Do we have something pipelined waiting?\n+\t * it's OK if he hasn't managed to send his headers yet... he's next\n+\t * in line to do that...\n+\t */\n+\tif (wsi_eff \u003d\u003d wsi) {\n+\t\t/*\n+\t\t * Nothing pipelined... we should hang around a bit\n+\t\t * in case something turns up...\n+\t\t */\n+\t\tlwsl_info(\u0022%s: nothing pipelined waiting\u005cn\u0022, __func__);\n+\t\tif (wsi-\u003eah) {\n+\t\t\tlws_header_table_force_to_detachable_state(wsi);\n+\t\t\tlws_header_table_detach(wsi, 0);\n+\t\t}\n+\t\tlws_set_timeout(wsi, PENDING_TIMEOUT_CLIENT_CONN_IDLE, 5);\n+\n+\t\treturn 0;\n+\t}\n+\n+\t/*\n+\t * H1: we can serialize the queued guys into into the same ah\n+\t * (H2: everybody needs their own ah until STREAM_END)\n+\t */\n+\n \t/* otherwise set ourselves up ready to go again */\n \twsi-\u003estate \u003d LWSS_CLIENT_HTTP_ESTABLISHED;\n-\twsi-\u003emode \u003d LWSCM_HTTP_CLIENT_ACCEPTED;\n \twsi-\u003ehttp.rx_content_length \u003d 0;\n \twsi-\u003ehdr_parsing_completed \u003d 0;\n \n-\t/* He asked for it to stay alive indefinitely */\n-\tlws_set_timeout(wsi, NO_PENDING_TIMEOUT, 0);\n+\twsi-\u003eah-\u003eparser_state \u003d WSI_TOKEN_NAME_PART;\n+\twsi-\u003eah-\u003elextable_pos \u003d 0;\n+\twsi-\u003emode \u003d LWSCM_WSCL_WAITING_SERVER_REPLY;\n \n-\t/*\n-\t * As client, nothing new is going to come until we ask for it\n-\t * we can drop the ah, if any\n-\t */\n-\tif (wsi-\u003eah) {\n-\t\tlws_header_table_force_to_detachable_state(wsi);\n-\t\tlws_header_table_detach(wsi, 0);\n-\t}\n+\tlws_set_timeout(wsi, PENDING_TIMEOUT_AWAITING_SERVER_RESPONSE,\n+\t\t\twsi-\u003econtext-\u003etimeout_secs);\n \n \t/* If we're (re)starting on headers, need other implied init */\n-\twsi-\u003eues \u003d URIES_IDLE;\n+\twsi-\u003eah-\u003eues \u003d URIES_IDLE;\n \n-\tlwsl_info(\u0022%s: %p: keep-alive await new transaction\u005cn\u0022, __func__, wsi);\n+\tlwsl_info(\u0022%s: %p: new queued transaction as %p\u005cn\u0022, __func__, wsi, wsi_eff);\n+\tlws_callback_on_writable(wsi);\n \n \treturn 0;\n-#endif\n }\n \n LWS_VISIBLE LWS_EXTERN unsigned int\n@@ -546,6 +685,7 @@ lws_client_interpret_server_handshake(struct lws *wsi)\n \tstruct lws_context *context \u003d wsi-\u003econtext;\n \tconst char *pc, *prot, *ads \u003d NULL, *path, *cce \u003d NULL;\n \tstruct allocated_headers *ah \u003d NULL;\n+\tstruct lws *w \u003d lws_client_wsi_effective(wsi);\n \tchar *p, *q;\n \tchar new_path[300];\n #if !defined(LWS_WITHOUT_EXTENSIONS)\n@@ -683,6 +823,48 @@ lws_client_interpret_server_handshake(struct lws *wsi)\n \n \tif (!wsi-\u003edo_ws) {\n \n+\t\t/* if keepalive is allowed, enable the queued pipeline guys */\n+\n+\t\tif (w \u003d\u003d wsi) { /* ie, coming to this for the first time */\n+\t\t\tif (wsi-\u003ehttp.connection_type \u003d\u003d HTTP_CONNECTION_KEEP_ALIVE)\n+\t\t\t\twsi-\u003ekeepalive_active \u003d 1;\n+\t\t\telse {\n+\t\t\t\t/*\n+\t\t\t\t * Ugh... now the main http connection has seen\n+\t\t\t\t * both sides, we learn the server doesn't\n+\t\t\t\t * support keepalive.\n+\t\t\t\t *\n+\t\t\t\t * That means any guys queued on us are going\n+\t\t\t\t * to have to be restarted from connect2 with\n+\t\t\t\t * their own connections.\n+\t\t\t\t */\n+\n+\t\t\t\t/*\n+\t\t\t\t * stick around telling any new guys they can't\n+\t\t\t\t * pipeline to this server\n+\t\t\t\t */\n+\t\t\t\twsi-\u003ekeepalive_rejected \u003d 1;\n+\n+\t\t\t\tlws_vhost_lock(wsi-\u003evhost);\n+\t\t\t\tlws_start_foreach_dll_safe(struct lws_dll_lws *, d, d1,\n+\t\t\t\t\t\t\t wsi-\u003edll_client_transaction_queue_head.next) {\n+\t\t\t\t\tstruct lws *ww \u003d lws_container_of(d, struct lws,\n+\t\t\t\t\t\t\t\t dll_client_transaction_queue);\n+\n+\t\t\t\t\t/* remove him from our queue */\n+\t\t\t\t\tlws_dll_lws_remove(\u0026ww-\u003edll_client_transaction_queue);\n+\t\t\t\t\t/* give up on pipelining */\n+\t\t\t\t\tww-\u003eclient_pipeline \u003d 0;\n+\n+\t\t\t\t\t/* go back to \u0022trying to connect\u0022 state */\n+\t\t\t\t\tlws_union_transition(ww, LWSCM_HTTP_CLIENT);\n+\t\t\t\t\tww-\u003euser_space \u003d NULL;\n+\t\t\t\t\tww-\u003estate \u003d LWSS_CLIENT_UNCONNECTED;\n+\t\t\t\t} lws_end_foreach_dll_safe(d, d1);\n+\t\t\t\tlws_vhost_unlock(wsi-\u003evhost);\n+\t\t\t}\n+\t\t}\n+\n #ifdef LWS_WITH_HTTP_PROXY\n \t\twsi-\u003eperform_rewrite \u003d 0;\n \t\tif (lws_hdr_total_length(wsi, WSI_TOKEN_HTTP_CONTENT_TYPE)) {\n@@ -716,7 +898,7 @@ lws_client_interpret_server_handshake(struct lws *wsi)\n \t\t\twsi-\u003ehttp.rx_content_length \u003d\n \t\t\t\t\tatoll(lws_hdr_simple_ptr(wsi,\n \t\t\t\t\t\tWSI_TOKEN_HTTP_CONTENT_LENGTH));\n-\t\t\tlwsl_notice(\u0022%s: incoming content length %llu\u005cn\u0022,\n+\t\t\tlwsl_info(\u0022%s: incoming content length %llu\u005cn\u0022,\n \t\t\t\t __func__, (unsigned long long)\n \t\t\t\t\t wsi-\u003ehttp.rx_content_length);\n \t\t\twsi-\u003ehttp.rx_content_remain \u003d\n@@ -751,10 +933,18 @@ lws_client_interpret_server_handshake(struct lws *wsi)\n \t\t\tgoto bail3;\n \t\t}\n \n-\t\t/* free up his parsing allocations */\n-\t\tlws_header_table_detach(wsi, 0);\n+\t\t/*\n+\t\t * for pipelining, master needs to keep his ah... guys who\n+\t\t * queued on him can drop it now though.\n+\t\t */\n+\n+\t\tif (w !\u003d wsi) {\n+\t\t\t/* free up parsing allocations for queued guy */\n+\t\t\tlws_header_table_force_to_detachable_state(w);\n+\t\t\tlws_header_table_detach(w, 0);\n+\t\t}\n \n-\t\tlwsl_notice(\u0022%s: client connection up\u005cn\u0022, __func__);\n+\t\tlwsl_info(\u0022%s: client connection up\u005cn\u0022, __func__);\n \n \t\treturn 0;\n \t}\ndiff --git a/lib/libwebsockets.c b/lib/libwebsockets.c\nindex fc822a6..e420eec 100644\n--- a/lib/libwebsockets.c\n+++ b/lib/libwebsockets.c\n@@ -539,6 +539,35 @@ __lws_close_free_wsi(struct lws *wsi, enum lws_close_status reason, const char *\n \tpt \u003d \u0026context-\u003ept[(int)wsi-\u003etsi];\n \tlws_stats_atomic_bump(wsi-\u003econtext, pt, LWSSTATS_C_API_CLOSE, 1);\n \n+#if !defined(LWS_NO_CLIENT)\n+\t/* we are no longer an active client connection that can piggyback */\n+\tlws_dll_lws_remove(\u0026wsi-\u003edll_active_client_conns);\n+\n+\t/*\n+\t * if we have wsi in our transaction queue, if we are closing we\n+\t * must go through and close all those first\n+\t */\n+\tlws_vhost_lock(wsi-\u003evhost);\n+\tlws_start_foreach_dll_safe(struct lws_dll_lws *, d, d1,\n+\t\t\t\twsi-\u003edll_client_transaction_queue_head.next) {\n+\t\tstruct lws *w \u003d lws_container_of(d, struct lws,\n+\t\t\t\t\t\t dll_client_transaction_queue);\n+\n+\t\t__lws_close_free_wsi(w, reason, \u0022trans q leader closing\u0022);\n+\t} lws_end_foreach_dll_safe(d, d1);\n+\n+\t/*\n+\t * !!! If we are closing, but we have pending pipelined transaction\n+\t * results we already sent headers for, that's going to destroy sync\n+\t * for HTTP/1 and leave H2 stream with no live swsi.\n+\t *\n+\t * However this is normal if we are being closed because the transaction\n+\t * queue leader is closing.\n+\t */\n+\tlws_dll_lws_remove(\u0026wsi-\u003edll_client_transaction_queue);\n+\tlws_vhost_unlock(wsi-\u003evhost);\n+#endif\n+\n \t/* if we have children, close them first */\n \tif (wsi-\u003echild_list) {\n \t\twsi2 \u003d wsi-\u003echild_list;\n@@ -823,12 +852,11 @@ just_kill_connection:\n \t}\n \n \tif ((wsi-\u003emode \u003d\u003d LWSCM_WSCL_WAITING_SERVER_REPLY ||\n-\t\t\t wsi-\u003emode \u003d\u003d LWSCM_WSCL_WAITING_CONNECT) \u0026\u0026\n-\t\t\t !wsi-\u003ealready_did_cce) {\n-\t\t\t\twsi-\u003eprotocol-\u003ecallback(wsi,\n-\t\t\t\t\tLWS_CALLBACK_CLIENT_CONNECTION_ERROR,\n+\t wsi-\u003emode \u003d\u003d LWSCM_WSCL_WAITING_CONNECT) \u0026\u0026\n+\t !wsi-\u003ealready_did_cce)\n+\t\twsi-\u003eprotocol-\u003ecallback(wsi,\n+\t\t\t\tLWS_CALLBACK_CLIENT_CONNECTION_ERROR,\n \t\t\t\t\t\twsi-\u003euser_space, NULL, 0);\n-\t}\n \n \tif (wsi-\u003emode \u0026 LWSCM_FLAG_IMPLIES_CALLBACK_CLOSED_CLIENT_HTTP) {\n \t\tconst struct lws_protocols *pro \u003d wsi-\u003eprotocol;\ndiff --git a/lib/libwebsockets.h b/lib/libwebsockets.h\nindex fa05e53..f77d23c 100644\n--- a/lib/libwebsockets.h\n+++ b/lib/libwebsockets.h\n@@ -800,6 +800,8 @@ enum lws_close_status {\n connection was closed due to a failure to perform a TLS handshake\n (e.g., the server certificate can't be verified). */\n \n+\tLWS_CLOSE_STATUS_CLIENT_TRANSACTION_DONE\t\t\u003d 2000,\n+\n \t/****** add new things just above ---^ ******/\n \n \tLWS_CLOSE_STATUS_NOSTATUS_CONTEXT_DESTROY\t\t\u003d 9999,\n@@ -3266,7 +3268,11 @@ enum lws_client_connect_ssl_connection_flags {\n \tLCCSCF_USE_SSL \t\t\t\t\u003d (1 \u003c\u003c 0),\n \tLCCSCF_ALLOW_SELFSIGNED\t\t\t\u003d (1 \u003c\u003c 1),\n \tLCCSCF_SKIP_SERVER_CERT_HOSTNAME_CHECK\t\u003d (1 \u003c\u003c 2),\n-\tLCCSCF_ALLOW_EXPIRED\t\t\t\u003d (1 \u003c\u003c 3)\n+\tLCCSCF_ALLOW_EXPIRED\t\t\t\u003d (1 \u003c\u003c 3),\n+\n+\tLCCSCF_PIPELINE\t\t\t\t\u003d (1 \u003c\u003c 16),\n+\t\t/**\u003c Serialize / pipeline multiple client connections\n+\t\t * on a single connection where possible. */\n };\n \n /** struct lws_client_connect_info - parameters to connect with when using\n@@ -3280,7 +3286,7 @@ struct lws_client_connect_info {\n \tint port;\n \t/**\u003c remote port to connect to */\n \tint ssl_connection;\n-\t/**\u003c nonzero for ssl */\n+\t/**\u003c 0, or a combination of LCCSCF_ flags */\n \tconst char *path;\n \t/**\u003c uri path */\n \tconst char *host;\n@@ -4556,6 +4562,7 @@ enum pending_timeout {\n \tPENDING_TIMEOUT_CLOSE_SEND\t\t\t\t\u003d 24,\n \tPENDING_TIMEOUT_HOLDING_AH\t\t\t\t\u003d 25,\n \tPENDING_TIMEOUT_UDP_IDLE\t\t\t\t\u003d 26,\n+\tPENDING_TIMEOUT_CLIENT_CONN_IDLE\t\t\t\u003d 27,\n \n \t/****** add new things just above ---^ ******/\n \n@@ -5474,6 +5481,8 @@ struct lws_dll_lws { /* typed as struct lws * */\n \tstruct lws_dll_lws *next;\n };\n \n+#define lws_dll_is_null(___dll) (!(___dll)-\u003eprev \u0026\u0026 !(___dll)-\u003enext)\n+\n static inline void\n lws_dll_lws_add_front(struct lws_dll_lws *_a, struct lws_dll_lws *_head)\n {\ndiff --git a/lib/private-libwebsockets.h b/lib/private-libwebsockets.h\nindex 4fcc60c..a0bbda4 100644\n--- a/lib/private-libwebsockets.h\n+++ b/lib/private-libwebsockets.h\n@@ -971,6 +971,9 @@ struct lws_vhost {\n \tconst struct lws_protocol_vhost_options *pvo;\n \tconst struct lws_protocol_vhost_options *headers;\n \tstruct lws **same_vh_protocol_list;\n+#if !defined(LWS_NO_CLIENT)\n+\tstruct lws_dll_lws dll_active_client_conns;\n+#endif\n \tconst char *error_document_404;\n #ifdef LWS_OPENSSL_SUPPORT\n \tlws_tls_ctx *ssl_ctx;\n@@ -1875,7 +1878,7 @@ struct lws {\n #endif\n \tconst struct lws_protocols *protocol;\n \tstruct lws **same_vh_protocol_prev, *same_vh_protocol_next;\n-\t/* we get on the list if either the timeout or the timer is valid */\n+\n \tstruct lws_dll_lws dll_timeout;\n \tstruct lws_dll_lws dll_hrtimer;\n #if defined(LWS_WITH_PEER_LIMITS)\n@@ -1887,6 +1890,9 @@ struct lws {\n \tunsigned char *preamble_rx;\n #ifndef LWS_NO_CLIENT\n \tstruct client_info_stash *stash;\n+\tstruct lws_dll_lws dll_active_client_conns;\n+\tstruct lws_dll_lws dll_client_transaction_queue_head;\n+\tstruct lws_dll_lws dll_client_transaction_queue;\n #endif\n \tvoid *user_space;\n \tvoid *opaque_parent_data;\n@@ -1962,7 +1968,7 @@ struct lws {\n \tunsigned int rxflow_will_be_applied:1;\n \tunsigned int event_pipe:1;\n \tunsigned int on_same_vh_list:1;\n-\tunsigned int handling_404;\n+\tunsigned int handling_404:1;\n \n \tunsigned int could_have_pending:1; /* detect back-to-back writes */\n \tunsigned int outer_will_close:1;\n@@ -1975,6 +1981,10 @@ struct lws {\n \tunsigned int chunked:1; /* if the clientside connection is chunked */\n \tunsigned int client_rx_avail:1;\n \tunsigned int client_http_body_pending:1;\n+\tunsigned int transaction_from_pipeline_queue:1;\n+\tunsigned int keepalive_active:1;\n+\tunsigned int keepalive_rejected:1;\n+\tunsigned int client_pipeline:1;\n #endif\n #ifdef LWS_WITH_HTTP_PROXY\n \tunsigned int perform_rewrite:1;\n@@ -2596,9 +2606,11 @@ lws_rewrite_parse(struct lws_rewrite *r, const unsigned char *in, int in_len);\n #endif\n \n #ifndef LWS_NO_CLIENT\n-LWS_EXTERN int lws_client_socket_service(struct lws_context *context,\n-\t\t\t\t\t struct lws *wsi,\n-\t\t\t\t\t struct lws_pollfd *pollfd);\n+LWS_EXTERN int lws_client_socket_service(struct lws *wsi,\n+\t\t\t\t\t struct lws_pollfd *pollfd,\n+\t\t\t\t\t struct lws *wsi_conn);\n+LWS_EXTERN struct lws *\n+lws_client_wsi_effective(struct lws *wsi);\n LWS_EXTERN int LWS_WARN_UNUSED_RESULT\n lws_http_transaction_completed_client(struct lws *wsi);\n #ifdef LWS_OPENSSL_SUPPORT\ndiff --git a/lib/service.c b/lib/service.c\nindex 122564d..c6358fe 100644\n--- a/lib/service.c\n+++ b/lib/service.c\n@@ -1018,14 +1018,18 @@ spin_chunks:\n \t\tlws_rewrite_parse(wsi-\u003erw, (unsigned char *)*buf, n);\n \telse\n #endif\n-\t\tif (user_callback_handle_rxflow(wsi-\u003eprotocol-\u003ecallback,\n-\t\t\t\twsi, LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ,\n-\t\t\t\twsi-\u003euser_space, *buf, n)) {\n+\t{\n+\t\tstruct lws *wsi_eff \u003d lws_client_wsi_effective(wsi);\n+\n+\t\tif (user_callback_handle_rxflow(wsi_eff-\u003eprotocol-\u003ecallback,\n+\t\t\t\twsi_eff, LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ,\n+\t\t\t\twsi_eff-\u003euser_space, *buf, n)) {\n \t\t\tlwsl_debug(\u0022%s: RECEIVE_CLIENT_HTTP_READ returned -1\u005cn\u0022,\n \t\t\t\t __func__);\n \n \t\t\treturn -1;\n \t\t}\n+\t}\n \n \tif (wsi-\u003echunked \u0026\u0026 wsi-\u003echunk_remaining) {\n \t\t(*buf) +\u003d n;\n@@ -1050,12 +1054,6 @@ spin_chunks:\n \t\treturn 0;\n \n completed:\n-\tif (user_callback_handle_rxflow(wsi-\u003eprotocol-\u003ecallback,\n-\t\t\twsi, LWS_CALLBACK_COMPLETED_CLIENT_HTTP,\n-\t\t\twsi-\u003euser_space, NULL, 0)) {\n-\t\tlwsl_debug(\u0022%s: Completed call returned nonzero (mode %d)\u005cn\u0022, __func__, wsi-\u003emode);\n-\t\treturn -1;\n-\t}\n \n \tif (lws_http_transaction_completed_client(wsi)) {\n \t\tlwsl_notice(\u0022%s: transaction completed says -1\u005cn\u0022, __func__);\n@@ -1884,7 +1882,7 @@ drain:\n \t\t\tgoto close_and_handled;\n \t\t}\n \n-\t\tn \u003d lws_client_socket_service(context, wsi, pollfd);\n+\t\tn \u003d lws_client_socket_service(wsi, pollfd, NULL);\n \t\tif (n)\n \t\t\treturn 1;\n \t\tgoto handled;\ndiff --git a/minimal-examples/http-client/minimal-http-client-multi/minimal-http-client-multi.c b/minimal-examples/http-client/minimal-http-client-multi/minimal-http-client-multi.c\nindex 526d5cb..81a87ac 100644\n--- a/minimal-examples/http-client/minimal-http-client-multi/minimal-http-client-multi.c\n+++ b/minimal-examples/http-client/minimal-http-client-multi/minimal-http-client-multi.c\n@@ -12,6 +12,19 @@\n * Currently that takes the form of 8 individual simultaneous tcp and\n * tls connections, which happen concurrently. Notice that the ordering\n * of the returned payload may be intermingled for the various connections.\n+ *\n+ * By default the connections happen all together at the beginning and operate\n+ * concurrently, which is fast. However this is resource-intenstive, there are\n+ * 8 tcp connections, 8 tls tunnels on both the client and server. You can\n+ * instead opt to have the connections happen one after the other inside a\n+ * single tcp connection and tls tunnel, using HTTP/1.1 pipelining. To be\n+ * eligible to be pipelined on another existing connection to the same server,\n+ * the client connection must have the LCCSCF_PIPELINE flag on its\n+ * info.ssl_connection member (this is independent of whether the connection\n+ * is in ssl mode or not).\n+ *\n+ * Pipelined connections are slower (2.3s vs 1.6s for 8 connections), since the\n+ * transfers are serialized, but it is much less resource-intensive.\n */\n \n #include \u003clibwebsockets.h\u003e\n@@ -84,7 +97,7 @@ callback_http(struct lws *wsi, enum lws_callback_reasons reason,\n \t\tif (++completed \u003d\u003d COUNT) {\n \t\t\tlwsl_user(\u0022Done: failed: %d\u005cn\u0022, failed);\n \t\t\tinterrupted \u003d 1;\n-\t\t\t/* so we exit without the poll wait */\n+\t\t\t/* so we exit immediately */\n \t\t\tlws_cancel_service(lws_get_context(wsi));\n \t\t}\n \t\tbreak;\n@@ -97,12 +110,7 @@ callback_http(struct lws *wsi, enum lws_callback_reasons reason,\n }\n \n static const struct lws_protocols protocols[] \u003d {\n-\t{\n-\t\t\u0022http\u0022,\n-\t\tcallback_http,\n-\t\t0,\n-\t\t0,\n-\t},\n+\t{ \u0022http\u0022, callback_http, 0, 0, },\n \t{ NULL, NULL, 0, 0 }\n };\n \n@@ -159,7 +167,8 @@ int main(int argc, char **argv)\n \ti.path \u003d \u0022/\u0022;\n \ti.host \u003d i.address;\n \ti.origin \u003d i.address;\n-\ti.ssl_connection \u003d 1;\n+\ti.ssl_connection \u003d LCCSCF_PIPELINE /* enables http1.1 pipelining */ |\n+\t\t\t LCCSCF_USE_SSL;\n \ti.method \u003d \u0022GET\u0022;\n \n \ti.protocol \u003d protocols[0].name;\n","s":{"c":1745762042,"u": 29148}} ],"g": 3770,"chitpc": 0,"ehitpc": 0,"indexed":0 , "ab": 0, "si": 0, "db":0, "di":0, "sat":0, "lfc": "7d0a"}