[Libwebsockets] client chunk/unchunk

Andy Green andy at warmcat.com
Mon Mar 6 20:05:08 CET 2017



On March 7, 2017 12:15:53 AM GMT+08:00, Joel Winarske <joel.winarske at gmail.com> wrote:
>The chunk/unchunk is fine.  In my case the content-length was not equal
>to
>the chunk/unchunk read sizes.  This threw me off.  My problem lies in
>trying to re-write these chunked/unchunked reads to the parent wsi.
>
>At the end of each read I call:
>    lws_callback_on_writable(lws_get_parent(wsi))
>
>Below is the output pattern of using this method.  I see the parent
>writable callback happen every other time.
>
>Do I need to coalesce the read data, or should I see the writable
>callback
>happen after each read, or do I ignore using lws_callback_on_writable()
>and
>just write it, since it's waiting for the response anyway?

Maybe it's better to answer by explaining what's happening overall

 - LWS_CALLBACK_RECEIVE_CLIENT_HTTP comes... it means "hey the remote server sent us some rx data, I didn't look at it yet"

 - you don't have to deal with it (by lws_http_client_read()) right there in the callback.  It's just an indication something can be read.

 - instead you can set a flag and ask for a callback on writeable on your outbound wsi.

 - when the outbound wsi is writeable, and the flag set, you can call lws_http_client_read() from there

 - with unchunked, you should get one LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ from that representing the whole rx.  With chunked, what was received may contain multiple (and partial...) chunking transfer encoding inline headers.  Lws strips these before giving it to you by making a callback to LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ for each whole or partial payload region in the buffer.

 - since the writable callback is only good for one write each time, but in chunked case lws_http_client_read() may spatter you with many pieces reflecting how the chunking was sent, you should coalesce them before writing

-Andy

>
>[2017/03/06 08:00:00:7171] INFO: http: (0301F760)
>LWS_CALLBACK_ESTABLISHED_CLIENT_HTTP: 200
>[2017/03/06 08:00:00:7181] INFO: lws_header_table_detach: wsi 0301F760:
>ah
>00EE43B8 (tsi=0, count = 2)
>[2017/03/06 08:00:00:7191] INFO: lws_header_table_detach: wsi 0301F760:
>ah
>00EE43B8 (tsi=0, count = 1)
>[2017/03/06 08:00:00:7201] NOTICE:
>lws_client_interpret_server_handshake:
>client connection up
>[2017/03/06 08:00:00:7201] INFO: ssl buffered read
>[2017/03/06 08:00:00:7211] INFO: chunk 32768
>[2017/03/06 08:00:00:7221] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 701
>[2017/03/06 08:00:00:7241] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7261] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7261] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7271] INFO: http: (0301F450)
>LWS_CALLBACK_HTTP_WRITEABLE: v=02FFB770, ctx=00ED1CF0
>[2017/03/06 08:00:00:7271] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7281] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7281] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7291] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7291] INFO: http: (0301F450)
>LWS_CALLBACK_HTTP_WRITEABLE: v=02FFB770, ctx=00ED1CF0
>[2017/03/06 08:00:00:7301] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7301] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7311] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7321] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7331] INFO: http: (0301F450)
>LWS_CALLBACK_HTTP_WRITEABLE: v=02FFB770, ctx=00ED1CF0
>[2017/03/06 08:00:00:7341] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7371] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7371] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7381] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7391] INFO: http: (0301F450)
>LWS_CALLBACK_HTTP_WRITEABLE: v=02FFB770, ctx=00ED1CF0
>[2017/03/06 08:00:00:7391] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7401] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7401] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7401] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7411] INFO: http: (0301F450)
>LWS_CALLBACK_HTTP_WRITEABLE: v=02FFB770, ctx=00ED1CF0
>[2017/03/06 08:00:00:7411] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7421] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7431] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7431] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7431] INFO: http: (0301F450)
>LWS_CALLBACK_HTTP_WRITEABLE: v=02FFB770, ctx=00ED1CF0
>[2017/03/06 08:00:00:7441] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7441] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7451] INFO: http: (0301F760)
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 1409
>[2017/03/06 08:00:00:7451] INFO: lws_read: read_ok, used 0
>[2017/03/06 08:00:00:7461] NOTICE: lws_server_socket_service: wsi
>0301F450
>read 0
>[2017/03/06 08:00:00:7501] INFO: lws_server_socket_service: read 0 len
>[2017/03/06 08:00:00:7511] ERR: 169.254.225.161 - -
>[06/Mar/2017:08:00:00
>-0800] "POST /rest/send http/1.1" 200 8976 Mozilla/5.0 (Windows NT
>10.0;
>WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87
>Safari/537.36
>[2017/03/06 08:00:00:7521] INFO: lws_close_free_wsi: real
>just_kill_connection: 0301F760 (sockfd 560)
>[2017/03/06 08:00:00:7531] INFO: remove_wsi_socket_from_fds: removing
>same
>prot wsi 0301F760
>[2017/03/06 08:00:00:7531] INFO: lws_close_free_wsi: shutting down
>connection: 0301F450 (sock 500, state 3)
>[2017/03/06 08:00:00:7541] INFO: lws_close_free_wsi: real
>just_kill_connection: 0301F450 (sockfd 500)
>[2017/03/06 08:00:00:7541] INFO: remove_wsi_socket_from_fds: removing
>same
>prot wsi 0301F450
>[2017/03/06 08:00:00:7551] INFO: ah det due to close
>[2017/03/06 08:00:00:7551] INFO: lws_header_table_detach: wsi 0301F450:
>ah
>00EE35A0 (tsi=0, count = 1)
>[2017/03/06 08:00:00:7561] INFO: lws_header_table_detach: wsi 0301F450:
>ah
>00EE35A0 (tsi=0, count = 0)
>[2017/03/06 08:00:00:7571] INFO: ah det due to close
>[2017/03/06 08:00:00:7571] INFO: lws_header_table_detach: wsi 0301F760:
>ah
>00000000 (tsi=0, count = 0)
>[2017/03/06 08:00:00:7581] INFO: lws_header_table_detach: wsi 0301F760:
>ah
>00000000 (tsi=0, count = 0)
>
>
>
>
>
>On Sun, Mar 5, 2017 at 8:43 PM Andy Green <andy at warmcat.com> wrote:
>
>>
>>
>> On 03/05/2017 11:55 PM, Joel Winarske wrote:
>> > Hi Andy,
>> >
>> > I'm seeing a problem with the client.  lws_http_client_read()
>returns
>> > a value in both RECEIVE_CLIENT_HTTP callbacks.  I would expect it
>> > would only return a value for chunked given the size.  See anything
>> > suspicious here?
>> >
>> > [2017/03/05 07:51:40:9226] NOTICE: lws_ssl_client_connect2:
>> > SSL_connect says 1
>> > [2017/03/05 07:51:41:4546] INFO: lws_ensure_user_space: 031651B0
>> > protocol 02E25628
>> > [2017/03/05 07:51:41:4556] INFO: lws_ensure_user_space: 031651B0
>> > protocol pss 11308, user_space=03173C10
>> > [2017/03/05 07:51:41:4566] NOTICE:
>> > lws_client_interpret_server_handshake: incoming content length 9506
>> >     connection: keep-alive
>> >     http/1.1  200 OK
>> >     pragma: no-cache
>> >     cache-control: no-cache
>> >     content-length: 9506
>> >     content-type: application/json; charset=utf-8
>> >     date: Sun, 05 Mar 2017 15:51:38 GMT
>> >     expires: -1
>> >     server: Microsoft-IIS/8.5
>> > [2017/03/05 07:51:41:4626] INFO: lws_header_table_detach: wsi
>> > 031651B0: ah 00E53258 (tsi=0, count = 2)
>> > [2017/03/05 07:51:41:4636] INFO: lws_header_table_detach: wsi
>> > 031651B0: ah 00E53258 (tsi=0, count = 1)
>> > [2017/03/05 07:51:41:4636] NOTICE:
>> > lws_client_interpret_server_handshake: client connection up
>> > [2017/03/05 07:51:41:4636] INFO: ssl buffered read
>> > [2017/03/05 07:51:41:4646] INFO: http: (031651B0)
>> > LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ (chunked): 4768
>> > [2017/03/05 07:51:41:4646] INFO: http: (031651B0)
>> > LWS_CALLBACK_RECEIVE_CLIENT_HTTP (unchunked) 4768
>>
>> How about only taking data from
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ,
>> for both chunked and unchunked?
>>
>> -Andy
>>
>> > [2017/03/05 07:51:41:4656] INFO: lws_close_free_wsi: shutting down
>SSL
>> > connection: 031651B0 (ssl 0317AD78, sock 560, state 6)
>> > [2017/03/05 07:51:41:4666] INFO: lws_close_free_wsi: real
>> > just_kill_connection: 031651B0 (sockfd 560)
>> > [2017/03/05 07:51:41:4676] INFO: remove_wsi_socket_from_fds:
>removing
>> > same prot wsi 031651B0
>> > [2017/03/05 07:51:41:4676] INFO: ah det due to close
>> > [2017/03/05 07:51:41:4686] INFO: lws_header_table_detach: wsi
>> > 031651B0: ah 00000000 (tsi=0, count = 1)
>> > [2017/03/05 07:51:41:4686] INFO: lws_header_table_detach: wsi
>> > 031651B0: ah 00000000 (tsi=0, count = 1)
>> >
>> >
>> >
>> > _______________________________________________
>> > Libwebsockets mailing list
>> > Libwebsockets at ml.libwebsockets.org
>> > https://libwebsockets.org/mailman/listinfo/libwebsockets
>>
>>



More information about the Libwebsockets mailing list