[Libwebsockets] HTTP client receive behavior changed?

Kun Zhao kunzhao77 at gmail.com
Mon Apr 8 17:47:20 CEST 2019


Is it possible a bug introduced in the master branch? My code has not
changed, using v3.1-stable everything works fine, data received in
LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ. When switched to the master branch.
LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ is never called (same url).

On Fri, Apr 5, 2019 at 4:08 PM Andy Green <andy at warmcat.com> wrote:

>
>
> On April 6, 2019 4:29:14 AM GMT+08:00, Kun Zhao <kunzhao77 at gmail.com>
> wrote:
> >The COMPLETED_CLIENT_HTTP is called before the last
> >RECEIVE_CLIENT_HTTP.
> >Here is the callback sequence.
> >
> >[2019/04/05 15:18:16:7170] INFO: lws_client_interpret_server_handshake:
> >client connection up
> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
> >******** LWS_CALLBACK_COMPLETED_CLIENT_HTTP
> >******** LWS_CALLBACK_RECEIVE_CLIENT_HTTP
> >[2019/04/05 15:18:16:7187] INFO: lws_http_transaction_completed_client:
> >wsi: 0x7fd6c8000b60, wsi_eff: 0x7fd6c8000b60 (http)
> >[2019/04/05 15:18:16:7187] INFO: lws_http_transaction_completed_client:
> >nothing pipelined waiting
> >******** LWS_CALLBACK_CLOSED_CLIENT_HTTP
> >[2019/04/05 15:18:22:0037] INFO: wsi 0x7fd6c8000b60: TIMEDOUT WAITING
> >on 27
> >(did hdr 0, ah 0x7fd6c8000e60, wl 0, pfd events 1) 1554495502 vs 5
> >[2019/04/05 15:18:22:0037] INFO: __lws_close_free_wsi: 0x7fd6c8000b60:
> >caller: timeout
> >
> >
> >On Fri, Apr 5, 2019 at 2:28 PM Kun Zhao <kunzhao77 at gmail.com> wrote:
> >
> >> Hi Andy,
> >>
> >> I updated libwebsockets from 3.0 to master branch today. I noticed a
> >> behavior change in HTTP client. HTTP client used to receive data in
> >> LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ callback. Now the
> >> LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ is never called (I guess for
> >> un-chunked data) and the data is provided in
> >> LWS_CALLBACK_RECEIVE_CLIENT_HTTP.
> >>
> >> Just want to check with you if this is the correct behavior because
> >of all
> >> the minimal examples and test client are still read data in
> >> LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ callback.
>
> Several minimal clients are built and run in CI, so if something like that
> was true the CI build breaks.
>
> What actually happens is when http rx comes, the user code is notified
> with LWS_CALLBACK_RECEIVE_CLIENT_HTTP. You can take the raw h1 data there,
> but with h1, at the decision of the server (many cgis will enforce it for
> their own convenience in being able to emit without knowing the
> content-length beforehand) the payload rx may be chunked. Chunk headers are
> then inserted at arbitrary intervals inline with the payload.
>
> Additionally, in many cases, there will be onward transfer to another
> socket with some form of the incoming data... eg, proxying bulk data from
> the h1 server to an onward ws link. When you want to consume the pending
> data is then something the onward link writeability decides. (Lws supports
> h1 / h2 <-> h1 reverse proxying natively using this stuff... in fact when
> you visit libwebsockets.org git, it's transparently delivered to h1 or h2
> clients via an lws reverse proxy link from a gitohashi process that's
> actually serving h1 on a unix domain socket on the same machine.)
>
> For those reasons the rx processing is split between the notification raw
> rx http payload stuff is available, at which point you can choose to leave
> it buffered by lws and turn off rx flow control and return to the event
> loop until you can consume it, or you can immediately dechunk it and
> consume it using the lws_http_client_read() api...
>
>
> https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/http-client/minimal-http-client/minimal-http-client.c#n57-66
>
> ... which calls back LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ for each
> dechunked block of data before returning.
>
> -Andy
>
> >> Thank,
> >> Kun
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://libwebsockets.org/pipermail/libwebsockets/attachments/20190408/b56f8003/attachment.htm>


More information about the Libwebsockets mailing list