[Libwebsockets] HTTP client receive behavior changed?

Andy Green andy at warmcat.com
Tue Apr 9 07:06:05 CEST 2019



On April 8, 2019 4:47:20 PM GMT+01:00, Kun Zhao <kunzhao77 at gmail.com> wrote:
>Is it possible a bug introduced in the master branch? My code has not

History tells us that's always possible.

>changed, using v3.1-stable everything works fine, data received in
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ. When switched to the master
>branch.
>LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ is never called (same url).

Could you try the minimal example client for this, and see if the same problem is coming?  On latest master.  If it's broken I'll try the same thing later.

-Andy

>On Fri, Apr 5, 2019 at 4:08 PM Andy Green <andy at warmcat.com> wrote:
>
>>
>>
>> On April 6, 2019 4:29:14 AM GMT+08:00, Kun Zhao <kunzhao77 at gmail.com>
>> wrote:
>> >The COMPLETED_CLIENT_HTTP is called before the last
>> >RECEIVE_CLIENT_HTTP.
>> >Here is the callback sequence.
>> >
>> >[2019/04/05 15:18:16:7170] INFO:
>lws_client_interpret_server_handshake:
>> >client connection up
>> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
>> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
>> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
>> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
>> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
>> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
>> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
>> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
>> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
>> >******** LWS_CALLBACK_COMPLETED_CLIENT_HTTP
>> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
>> >[2019/04/05 15:18:16:7187] INFO:
>lws_http_transaction_completed_client:
>> >wsi: 0x7fd6c8000b60, wsi_eff: 0x7fd6c8000b60 (http)
>> >[2019/04/05 15:18:16:7187] INFO:
>lws_http_transaction_completed_client:
>> >nothing pipelined waiting
>> >******** LWS_CALLBACK_CLOSED_CLIENT_HTTP
>> >[2019/04/05 15:18:22:0037] INFO: wsi 0x7fd6c8000b60: TIMEDOUT
>WAITING
>> >on 27
>> >(did hdr 0, ah 0x7fd6c8000e60, wl 0, pfd events 1) 1554495502 vs 5
>> >[2019/04/05 15:18:22:0037] INFO: __lws_close_free_wsi:
>0x7fd6c8000b60:
>> >caller: timeout
>> >
>> >
>> >On Fri, Apr 5, 2019 at 2:28 PM Kun Zhao <kunzhao77 at gmail.com> wrote:
>> >
>> >> Hi Andy,
>> >>
>> >> I updated libwebsockets from 3.0 to master branch today. I noticed
>a
>> >> behavior change in HTTP client. HTTP client used to receive data
>in
>> >> LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ callback. Now the
>> >> LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ is never called (I guess for
>> >> un-chunked data) and the data is provided in
>> >> LWS_CALLBACK_RECEIVE_CLIENT_HTTP.
>> >>
>> >> Just want to check with you if this is the correct behavior
>because
>> >of all
>> >> the minimal examples and test client are still read data in
>> >> LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ callback.
>>
>> Several minimal clients are built and run in CI, so if something like
>that
>> was true the CI build breaks.
>>
>> What actually happens is when http rx comes, the user code is
>notified
>> with LWS_CALLBACK_RECEIVE_CLIENT_HTTP. You can take the raw h1 data
>there,
>> but with h1, at the decision of the server (many cgis will enforce it
>for
>> their own convenience in being able to emit without knowing the
>> content-length beforehand) the payload rx may be chunked. Chunk
>headers are
>> then inserted at arbitrary intervals inline with the payload.
>>
>> Additionally, in many cases, there will be onward transfer to another
>> socket with some form of the incoming data... eg, proxying bulk data
>from
>> the h1 server to an onward ws link. When you want to consume the
>pending
>> data is then something the onward link writeability decides. (Lws
>supports
>> h1 / h2 <-> h1 reverse proxying natively using this stuff... in fact
>when
>> you visit libwebsockets.org git, it's transparently delivered to h1
>or h2
>> clients via an lws reverse proxy link from a gitohashi process that's
>> actually serving h1 on a unix domain socket on the same machine.)
>>
>> For those reasons the rx processing is split between the notification
>raw
>> rx http payload stuff is available, at which point you can choose to
>leave
>> it buffered by lws and turn off rx flow control and return to the
>event
>> loop until you can consume it, or you can immediately dechunk it and
>> consume it using the lws_http_client_read() api...
>>
>>
>>
>https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/http-client/minimal-http-client/minimal-http-client.c#n57-66
>>
>> ... which calls back LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ for each
>> dechunked block of data before returning.
>>
>> -Andy
>>
>> >> Thank,
>> >> Kun
>> >>
>>


More information about the Libwebsockets mailing list