[Libwebsockets] client pipelining and wsi context mixups

Andy Green andy at warmcat.com
Tue Jan 21 12:43:36 CET 2020

On 1/21/20 11:13 AM, Zevv wrote:

> I'm trying to run a multi-clients setup with libws and make use of
> pipelining/http keepalive where appropriate. I'm confused though be the way
> libws reuses existing lwis, and what the right order is to do things.

The http-client-multi has the best examples for this, for both h1 and h2...


... if something's not right that's also the best way to reproduce it 
since I also can run the same thing.

> This is what is happening:
> - I create one lws_context
> - I perform a HTTP client request using lws_client_connect_via_info with
>    LCCSCF_PIPELINE set. This generates a new wsi, let's call that A
> - The HTTP request is perfomed as expected. The events I handle:
>    * LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ for reading the data
>    * LWS_CALLBACK_COMPLETED_CLIENT_HTTP when the request is done
> So far so good, now the interesting part:
> - Before wsi A is cleaned up I start a second request with an url on the
>    same host. This gives me wsi B, which also holds the pointer to my
>    private data
> - lws figures out my intention and finds wsi A to send the request over
>    to make use of the existing connection to the server. Great!
> - Here things get confusing, now I get wsi callbacks for both wsi A and
>    B, but I'm not sure how to handle things:

This situation is very different depending if the connection is 
happening with h1 or h2... the http-client-multi example lets you try / 
shows you how to select between various modes with -p, -s, --h1 switches.

With h1 lws can just ignore any optimization (no PIPELINE flag / -p), it 
can queue your request clientside, or it can pipeline your request by 
sending headers ahead of time.  In the last two cases it's going to try 
to reuse the network connection and tls tunnel sequentially.

With h2 it can ignore it, queue it clientside, or open another stream on 
an existing network connection without any new network connection or tls 

Initially they're queued, because doing anything else requires extra 
knowledge about the server... does it actually support h2 for that, and 
does it support h1 keepalive for that... these are things we only find 
out when we have talked to the server already.  The choice about how to 
dispose of the queued client connections is made only after that then.

>         headers available in this wsi! I can 'fix' this with patch [1]
>         below, that makes sure I do receive headers, albeit in wsi A,
>         not in wsi B, which has my correct user data.

This is the point of confusion is it?

>    * A: LWS_CALLBACK_RECEIVE_CLIENT_HTTP. No problem, I don't need to
>         access my own data for this so I can just call
>         lws_http_client_read
>         and handle it.
> So, can anyone clarify the semantics behind this? I am problably misssing
> something obvious when setting up my requests, but I was not able to deduce
> from the docs or code what the right way is to handle this.

I think for pipelined h1 case, what's happening is although we're 
correctly doing the callback currently with the identity of the active 
queued child wsi, because in that case the children sequentially "reuse" 
the connection wsi, we reuse the connection wsi ah.

The wsi we are calling it with isn't wrong as it is, I think what might 
be missing is we also need to tag the child with the main connection wsi 
ah temporarily... how about this?

diff --git a/lib/roles/http/client/client-http.c 
index 9444e79a8..547c793f6 100644
--- a/lib/roles/http/client/client-http.c
+++ b/lib/roles/http/client/client-http.c
@@ -721,7 +721,7 @@ lws_client_interpret_server_handshake(struct lws *wsi)
         int n, port = 0, ssl = 0;
         int close_reason = LWS_CLOSE_STATUS_PROTOCOL_ERR;
         const char *prot, *ads = NULL, *path, *cce = NULL;
-       struct allocated_headers *ah;
+       struct allocated_headers *ah, *ah1;
         struct lws *w = lws_client_wsi_effective(wsi);
         struct lws *nwsi = lws_get_network_wsi(wsi);
         char *p, *q;
@@ -1016,10 +1016,12 @@ lws_client_interpret_server_handshake(struct lws 
                  * we seem to be good to go, give client last chance to 
                  * headers and OK it
+               ah1 = w->http.ah;
+               w->http.ah = ah;
                 if (w->protocol->callback(w,
                                             w->user_space, NULL, 0)) {
+                       w->http.ah = ah1;
                         cce = "HS: disallowed by client filter";
                         goto bail2;
@@ -1033,10 +1035,13 @@ lws_client_interpret_server_handshake(struct lws 
                 if (w->protocol->callback(w,
                                             w->user_space, NULL, 0)) {
+                       w->http.ah = ah1;
                         cce = "HS: disallowed at ESTABLISHED";
                         goto bail3;

+               w->http.ah = ah1;
                  * for pipelining, master needs to keep his ah... guys who
                  * queued on him can drop it now though.


> Thank you,
> Zevv
> [1] Changing the wsi context for LWS_CALLBACK_CLIENT_FILTER_PRE_ESTABLISH
> --- /lib/roles/http/client/client-http.c
> +++ /lib/roles/http/client/client-http.c
> @@ -1013,9 +1015,9 @@ lws_client_interpret_server_handshake(struct lws *wsi)
>                   * we seem to be good to go, give client last chance to check
>                   * headers and OK it
>                   */
> -               if (w->protocol->callback(w,
> +               if (w->protocol->callback(wsi,
>                                  LWS_CALLBACK_CLIENT_FILTER_PRE_ESTABLISH,
> -                                           w->user_space, NULL, 0)) {
> +                                           wsi->user_space, NULL, 0)) {
> _______________________________________________
> Libwebsockets mailing list
> Libwebsockets at ml.libwebsockets.org
> https://libwebsockets.org/mailman/listinfo/libwebsockets

More information about the Libwebsockets mailing list