[Libwebsockets] client chunk/unchunk

Joel Winarske joel.winarske at gmail.com
Mon Mar 6 20:37:58 CET 2017


Okay, so this makes a good deal more sense now :)

In the case of forwarding the coalesced chunks, do I need to append
anything for the chunking transfer encoding inline headers, or does that
happen magically?

On Mon, Mar 6, 2017 at 11:05 AM, Andy Green <andy at warmcat.com> wrote:

>
>
> On March 7, 2017 12:15:53 AM GMT+08:00, Joel Winarske <
> joel.winarske at gmail.com> wrote:
> >The chunk/unchunk is fine.  In my case the content-length was not equal
> >to
> >the chunk/unchunk read sizes.  This threw me off.  My problem lies in
> >trying to re-write these chunked/unchunked reads to the parent wsi.
> >
> >At the end of each read I call:
> >    lws_callback_on_writable(lws_get_parent(wsi))
> >
> >Below is the output pattern of using this method.  I see the parent
> >writable callback happen every other time.
> >
> >Do I need to coalesce the read data, or should I see the writable
> >callback
> >happen after each read, or do I ignore using lws_callback_on_writable()
> >and
> >just write it, since it's waiting for the response anyway?
>
> Maybe it's better to answer by explaining what's happening overall
>
>  - LWS_CALLBACK_RECEIVE_CLIENT_HTTP comes... it means "hey the remote
> server sent us some rx data, I didn't look at it yet"
>
>  - you don't have to deal with it (by lws_http_client_read()) right there
> in the callback.  It's just an indication something can be read.
>
>  - instead you can set a flag and ask for a callback on writeable on your
> outbound wsi.
>
>  - when the outbound wsi is writeable, and the flag set, you can call
> lws_http_client_read() from there
>
>  - with unchunked, you should get one LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ
> from that representing the whole rx.  With chunked, what was received may
> contain multiple (and partial...) chunking transfer encoding inline
> headers.  Lws strips these before giving it to you by making a callback to
> LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ for each whole or partial payload
> region in the buffer.
>
>  - since the writable callback is only good for one write each time, but
> in chunked case lws_http_client_read() may spatter you with many pieces
> reflecting how the chunking was sent, you should coalesce them before
> writing
>
> -Andy
>
> >
> >[2017/03/06 08:00:00:7171] INFO: http: (0301F760)
> >LWS_CALLBACK_ESTABLISHED_CLIENT_HTTP: 200
> >[2017/03/06 08:00:00:7181] INFO: lws_header_table_detach: wsi 0301F760:
> >ah
> >00EE43B8 (tsi=0, count = 2)
> >[2017/03/06 08:00:00:7191] INFO: lws_header_table_detach: wsi 0301F760:
> >ah
> >00EE43B8 (tsi=0, count = 1)
> >[2017/03/06 08:00:00:7201] NOTICE:
> >lws_client_interpret_server_handshake:
> >client connection up
> >[2017/03/06 08:00:00:7201] INFO: ssl buffered read
> >[2017/03/06 08:00:00:7211] INFO: chunk 32768
> >[2017/03/06 08:00:00:7221] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 701
> >[2017/03/06 08:00:00:7241] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7261] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7261] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7271] INFO: http: (0301F450)
> >LWS_CALLBACK_HTTP_WRITEABLE: v=02FFB770, ctx=00ED1CF0
> >[2017/03/06 08:00:00:7271] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7281] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7281] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7291] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7291] INFO: http: (0301F450)
> >LWS_CALLBACK_HTTP_WRITEABLE: v=02FFB770, ctx=00ED1CF0
> >[2017/03/06 08:00:00:7301] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7301] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7311] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7321] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7331] INFO: http: (0301F450)
> >LWS_CALLBACK_HTTP_WRITEABLE: v=02FFB770, ctx=00ED1CF0
> >[2017/03/06 08:00:00:7341] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7371] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7371] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7381] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7391] INFO: http: (0301F450)
> >LWS_CALLBACK_HTTP_WRITEABLE: v=02FFB770, ctx=00ED1CF0
> >[2017/03/06 08:00:00:7391] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7401] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7401] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7401] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7411] INFO: http: (0301F450)
> >LWS_CALLBACK_HTTP_WRITEABLE: v=02FFB770, ctx=00ED1CF0
> >[2017/03/06 08:00:00:7411] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7421] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7431] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7431] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7431] INFO: http: (0301F450)
> >LWS_CALLBACK_HTTP_WRITEABLE: v=02FFB770, ctx=00ED1CF0
> >[2017/03/06 08:00:00:7441] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7441] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7451] INFO: http: (0301F760)
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
> >[2017/03/06 08:00:00:7451] INFO: lws_read: read_ok, used 0
> >[2017/03/06 08:00:00:7461] NOTICE: lws_server_socket_service: wsi
> >0301F450
> >read 0
> >[2017/03/06 08:00:00:7501] INFO: lws_server_socket_service: read 0 len
> >[2017/03/06 08:00:00:7511] ERR: 169.254.225.161 - -
> >[06/Mar/2017:08:00:00
> >-0800] "POST /rest/send http/1.1" 200 8976 Mozilla/5.0 (Windows NT
> >10.0;
> >WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87
> >Safari/537.36
> >[2017/03/06 08:00:00:7521] INFO: lws_close_free_wsi: real
> >just_kill_connection: 0301F760 (sockfd 560)
> >[2017/03/06 08:00:00:7531] INFO: remove_wsi_socket_from_fds: removing
> >same
> >prot wsi 0301F760
> >[2017/03/06 08:00:00:7531] INFO: lws_close_free_wsi: shutting down
> >connection: 0301F450 (sock 500, state 3)
> >[2017/03/06 08:00:00:7541] INFO: lws_close_free_wsi: real
> >just_kill_connection: 0301F450 (sockfd 500)
> >[2017/03/06 08:00:00:7541] INFO: remove_wsi_socket_from_fds: removing
> >same
> >prot wsi 0301F450
> >[2017/03/06 08:00:00:7551] INFO: ah det due to close
> >[2017/03/06 08:00:00:7551] INFO: lws_header_table_detach: wsi 0301F450:
> >ah
> >00EE35A0 (tsi=0, count = 1)
> >[2017/03/06 08:00:00:7561] INFO: lws_header_table_detach: wsi 0301F450:
> >ah
> >00EE35A0 (tsi=0, count = 0)
> >[2017/03/06 08:00:00:7571] INFO: ah det due to close
> >[2017/03/06 08:00:00:7571] INFO: lws_header_table_detach: wsi 0301F760:
> >ah
> >00000000 (tsi=0, count = 0)
> >[2017/03/06 08:00:00:7581] INFO: lws_header_table_detach: wsi 0301F760:
> >ah
> >00000000 (tsi=0, count = 0)
> >
> >
> >
> >
> >
> >On Sun, Mar 5, 2017 at 8:43 PM Andy Green <andy at warmcat.com> wrote:
> >
> >>
> >>
> >> On 03/05/2017 11:55 PM, Joel Winarske wrote:
> >> > Hi Andy,
> >> >
> >> > I'm seeing a problem with the client.  lws_http_client_read()
> >returns
> >> > a value in both RECEIVE_CLIENT_HTTP callbacks.  I would expect it
> >> > would only return a value for chunked given the size.  See anything
> >> > suspicious here?
> >> >
> >> > [2017/03/05 07:51:40:9226] NOTICE: lws_ssl_client_connect2:
> >> > SSL_connect says 1
> >> > [2017/03/05 07:51:41:4546] INFO: lws_ensure_user_space: 031651B0
> >> > protocol 02E25628
> >> > [2017/03/05 07:51:41:4556] INFO: lws_ensure_user_space: 031651B0
> >> > protocol pss 11308, user_space=03173C10
> >> > [2017/03/05 07:51:41:4566] NOTICE:
> >> > lws_client_interpret_server_handshake: incoming content length 9506
> >> >     connection: keep-alive
> >> >     http/1.1  200 OK
> >> >     pragma: no-cache
> >> >     cache-control: no-cache
> >> >     content-length: 9506
> >> >     content-type: application/json; charset=utf-8
> >> >     date: Sun, 05 Mar 2017 15:51:38 GMT
> >> >     expires: -1
> >> >     server: Microsoft-IIS/8.5
> >> > [2017/03/05 07:51:41:4626] INFO: lws_header_table_detach: wsi
> >> > 031651B0: ah 00E53258 (tsi=0, count = 2)
> >> > [2017/03/05 07:51:41:4636] INFO: lws_header_table_detach: wsi
> >> > 031651B0: ah 00E53258 (tsi=0, count = 1)
> >> > [2017/03/05 07:51:41:4636] NOTICE:
> >> > lws_client_interpret_server_handshake: client connection up
> >> > [2017/03/05 07:51:41:4636] INFO: ssl buffered read
> >> > [2017/03/05 07:51:41:4646] INFO: http: (031651B0)
> >> > LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 4768
> >> > [2017/03/05 07:51:41:4646] INFO: http: (031651B0)
> >> > LWS_CALLBACK_RECEIVE_CLIENT_HTTP (unchunked) 4768
> >>
> >> How about only taking data from
> >LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ,
> >> for both chunked and unchunked?
> >>
> >> -Andy
> >>
> >> > [2017/03/05 07:51:41:4656] INFO: lws_close_free_wsi: shutting down
> >SSL
> >> > connection: 031651B0 (ssl 0317AD78, sock 560, state 6)
> >> > [2017/03/05 07:51:41:4666] INFO: lws_close_free_wsi: real
> >> > just_kill_connection: 031651B0 (sockfd 560)
> >> > [2017/03/05 07:51:41:4676] INFO: remove_wsi_socket_from_fds:
> >removing
> >> > same prot wsi 031651B0
> >> > [2017/03/05 07:51:41:4676] INFO: ah det due to close
> >> > [2017/03/05 07:51:41:4686] INFO: lws_header_table_detach: wsi
> >> > 031651B0: ah 00000000 (tsi=0, count = 1)
> >> > [2017/03/05 07:51:41:4686] INFO: lws_header_table_detach: wsi
> >> > 031651B0: ah 00000000 (tsi=0, count = 1)
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > Libwebsockets mailing list
> >> > Libwebsockets at ml.libwebsockets.org
> >> > https://libwebsockets.org/mailman/listinfo/libwebsockets
> >>
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://libwebsockets.org/pipermail/libwebsockets/attachments/20170306/60026242/attachment-0002.html>


More information about the Libwebsockets mailing list