[Libwebsockets] Socket close during some HTTP gets

Michael Behrns-Miller moodboom at gmail.com
Thu Jan 19 03:52:32 CET 2017


Hey Andy, thanks for being so helpful.

I'm getting a similar result from the test client, see below.  No
LWS_CALLBACK_COMPLETED_CLIENT_HTTP, had to hit Ctrl-C to exit.  I have
pulled the latest v2.0-stable code.  Maybe I should switch up to v2.1?

Thanks,
Michael


Here is the installed version of libwebsockets, it should be v2.0 or higher:
2.1.0
m at case:~/development$ /usr/local/bin/libwebsockets-test-client
https://www.google.com/#q=energy+news+today
[2017/01/18 21:47:43:2250] NOTICE: libwebsockets test client - license
LGPL2.1+SLE
[2017/01/18 21:47:43:2251] NOTICE: (C) Copyright 2010-2016 Andy Green
<andy at warmcat.com>
[2017/01/18 21:47:43:2251] NOTICE:  Using SSL
[2017/01/18 21:47:43:2251] NOTICE:  Cert must validate correctly (use
-s to allow selfsigned)
[2017/01/18 21:47:43:2251] NOTICE:  Requiring peer cert hostname matches
[2017/01/18 21:47:43:2251] NOTICE: Initial logging level 7
[2017/01/18 21:47:43:2251] NOTICE: Libwebsockets version: 2.1.0
m at case-v2.0.0-203-g109d66c
[2017/01/18 21:47:43:2251] NOTICE: IPV6 not compiled in
[2017/01/18 21:47:43:2251] NOTICE: libev support not compiled in
[2017/01/18 21:47:43:2251] NOTICE: libuv support not compiled in
[2017/01/18 21:47:43:2251] NOTICE:  Threads: 1 each 1024 fds
[2017/01/18 21:47:43:2251] NOTICE:  mem: platform fd map:  8192 bytes
[2017/01/18 21:47:43:2251] NOTICE:  Compiled with OpenSSL support
[2017/01/18 21:47:43:2259] NOTICE: Creating Vhost 'default' port -1, 2
protocols, IPv6 off
[2017/01/18 21:47:43:2260] NOTICE:  mem: per-conn:          512 bytes
+ protocol rx buf
[2017/01/18 21:47:43:2260] NOTICE: using https mode (non-ws)
[2017/01/18 21:47:43:2260] NOTICE: http: connecting
[2017/01/18 21:47:43:2260] NOTICE: lws_protocol_init
[2017/01/18 21:47:43:5414] NOTICE:
lws_client_interpret_server_handshake: client connection up
[2017/01/18 21:47:43:5414] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5415] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5416] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5416] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5416] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5417] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5418] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5712] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5712] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5713] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5713] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5713] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5713] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5713] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5713] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5714] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5714] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5714] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5714] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5716] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5716] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5718] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5719] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5743] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5743] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5743] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5743] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5753] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5753] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5755] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5755] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5766] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5766] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5770] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5770] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5788] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5788] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5788] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:47:43:5788] NOTICE: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
[2017/01/18 21:48:04:0004] NOTICE: wsi 0x556085b4dfb0: TIMEDOUT
WAITING on 13 (did hdr 1, ah (nil), wl 0, pfd events 1)

On Wed, Jan 18, 2017 at 8:51 PM, Andy Green <andy at warmcat.com> wrote:
>
>
> On 01/19/2017 09:41 AM, Michael Behrns-Miller wrote:
>>
>> Thanks Andy.
>>
>> I did grab your latest code, my issue persists.  When i trace into the
>> handling, I find that in service.c ln ~553 :spin_chunks label,
>> wsi->chunked is 1.  I then have to catch my chunked http packets in
>
>
> It's possible the server you're talking to is just sending stuff with
> Content-encoding: chunked
>
> CGIs tend to do it because they have no idea about the total length the
> script will gente but want to send stuff as it becomes available.
>
>> LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ.  I'm afraid that I have not
>> separated the wsi data of my websockets and http properly, and the wss
>> chunked flag is set for my http traffic, maybe?
>
> If you increase the logging to eg 63, you should see the http headers...
> check and see if it is supposed to be chunked first... I assume it is.
>
> ws doesn't use HTTP chunked encoding... it's more likely your HTTP server
> actually is sending chunked.
>
> What happens if you use the test client, eg
>
> $ libwebsockets-test-client http://warmcat.com
>
> will dump the HTML... does your server sending chunked kill it?  If so can I
> try it from here?
>
> -Andy
>
>
>>
>> On Wed, Jan 18, 2017 at 8:34 PM, Andy Green <andy at warmcat.com> wrote:
>>>
>>>
>>> On 01/19/2017 06:25 AM, Michael Behrns-Miller wrote:
>>>>
>>>> Hello!
>>>>
>>>> I am working on interweaving an http get with a persistent wss client
>>>> connection back to a server.  I am using one context and one protocol.
>>>> lws always seems to go into "chunked' mode for the http get.  Some
>>>
>>>
>>> What do we mean by, 'chunked'... there is literally HTTP chunked mode,
>>> which
>>> has hex length headers before each chunk.  Or there is just "chunks" of
>>> data
>>> coming rather than the whole thing at once.
>>>>
>>>> sites work, and others drop the socket connection before finishing the
>>>> http get.
>>>>
>>>> To get it working, there is a lot to get right, and i am probably
>>>> getting some part of it wrong.
>>>>
>>>> My strategy was to use a standard wss client connection (that went
>>>> fine).  Then for the http get, I create a second
>>>> lws_client_connect_info and wsi for it, and attach it to the context
>>>> with lws_client_connect_via_info().
>>>
>>> Sounds good...
>>>
>>>> All my testing with local pages, big and small, seemed to work fine.
>>>> Other tests (in this case a simple google query) gave this -d15 level
>>>> debugging and failed to COMPLETE:
>>>>
>>>> [2017/01/18 16:42:09:3646] WARN: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
>>>> [2017/01/18 16:42:09:3646] WARN:
>>>> LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ: received 365 bytes
>>>> [2017/01/18 16:42:09:3646] WARN: HTTP_READ of 365 bytes begins:
>>>> on(a){try{var b=docu
>>>> [2017/01/18 16:42:09:3646] WARN: HTTP_READ of 365 bytes ends :
>>>> /(^|\\?|&)ei=/.test(
>>>> [2017/01/18 16:42:09:3646] WARN: LWS_CALLBACK_RECEIVE_CLIENT_HTTP
>>>> [2017/01/18 16:42:09:3646] INFO: lws_close_free_wsi: real
>>>> just_kill_connection: 0x5555557dfe90 (sockfd 7)
>>>> [2017/01/18 16:42:09:3646] INFO: remove_wsi_socket_from_fds: removing
>>>> same prot wsi 0x5555557dfe90
>>>> [2017/01/18 16:42:09:3646] INFO: remove_wsi_socket_from_fds:
>>>> wsi=0x5555557dfe90, sock=7, fds pos=2, end guy pos=3, endfd=0
>>>> [2017/01/18 16:42:09:3647] INFO: lws_header_table_detach: wsi
>>>> 0x5555557dfe90: ah (nil) (tsi=0, count = 0)
>>>>
>>>> Some of the http gets do not log any errors, but never finish.  In
>>>> debugging the code, I usually find that the call is perceived as
>>>> chunked for some reason, and I guess the final chunked marker is never
>>>> hit.
>>>>
>>>> Sorry Andy, that's a small bit of the picture I know.  Any advice?
>>>> Even if it is just something to try.  Appreciate it.
>>>
>>> Make sure you're using latest master lws first I think.  I fixed some
>>> bugs
>>> around client HTTP last week.
>>>
>>> There's nothing wrong with the overall approach you can have any mixture
>>> of
>>> client connections as much as you like.
>>>
>>> -Andy
>>>>
>>>>
>>>> Michael
>>>> _______________________________________________
>>>> Libwebsockets mailing list
>>>> Libwebsockets at ml.libwebsockets.org
>>>> https://libwebsockets.org/mailman/listinfo/libwebsockets
>>>
>>>
>



More information about the Libwebsockets mailing list