[Libwebsockets] Is there a way to reuse a lws_context without destroying it

Andy Green andy at warmcat.com
Fri Jun 16 01:04:20 CEST 2017



On 06/16/2017 03:57 AM, Ash 20001 wrote:
> I tried to reproduce it with libwebsocket-test-server and test-client 
> with 2 servers on separate ports but could not with the example.
> 
> 
> In my application I have been able to reproduce it on the same port 
> itself and when I reconnected using the same context I see this:
> 
> 
> 140  :  2017/6/15 12:53:09.397DBG  :  [26988] lws: 
> insert_wsi_socket_into_fds: 0x7f5c2c01fa40: tsi=0, sock=13, pos-in-fds=1
> sigReconnectHandler: caught signal 3... reconnecting
> 477  :  2017/6/15 12:53:24.708DBG  :  [27038] lws: 
> insert_wsi_socket_into_fds: 0x7f5c2c01fbe0: tsi=0, sock=14, pos-in-fds=2
> sigReconnectHandler: caught signal 3... reconnecting
> 627  :  2017/6/15 12:53:31.352DBG  :  [27074] lws: 
> insert_wsi_socket_into_fds: 0x7f5c2400fe20: tsi=0, sock=13, pos-in-fds=2
> sigReconnectHandler: caught signal 3... reconnecting
> 1115 :  2017/6/15 12:53:37.216DBG  :  [27109] lws: 
> insert_wsi_socket_into_fds: 0x7f5c24000e90: tsi=0, sock=15, pos-in-fds=3
> sigReconnectHandler: caught signal 3... reconnecting
> 1270 :  2017/6/15 12:53:42.761DBG  :  [27145] lws: 
> insert_wsi_socket_into_fds: 0x7f5c1c0014a0: tsi=0, sock=13, pos-in-fds=3
> sigReconnectHandler: caught signal 3... reconnecting
> 1783 :  2017/6/15 12:53:48.660DBG  :  [27181] lws: 
> insert_wsi_socket_into_fds: 0x7f5c1c002700: tsi=0, sock=16, pos-in-fds=4
> 1926 :  2017/6/15 12:53:52.921DBG  :  [27209] lws: 
> insert_wsi_socket_into_fds: 0x7f5c1c000ca0: tsi=0, sock=13, pos-in-fds=4
> sigReconnectHandler: caught signal 3... reconnecting
> 2411 :  2017/6/15 12:53:54.316DBG  :  [27241] lws: 
> insert_wsi_socket_into_fds: 0x7f5c1c001350: tsi=0, sock=17, pos-in-fds=5
> 
> To me that means that sockets aren't getting cleaned up. In addition I 

They might not even be being closed depending on what changes you made.

On Linux, if you know your PID

$ ps fax | grep test-cli
11879 pts/2    R+     0:41  |   |   \_ libwebsockets-test-client localhost

You can externally look at what fds the process has open

$ ls -l /proc/11879/fd
total 0
lrwx------. 1 agreen agreen 64 Jun 16 06:57 0 -> /dev/pts/2
lrwx------. 1 agreen agreen 64 Jun 16 06:57 1 -> /dev/pts/2
lrwx------. 1 agreen agreen 64 Jun 16 06:56 2 -> /dev/pts/2
lr-x------. 1 agreen agreen 64 Jun 16 06:57 3 -> /dev/urandom
lr-x------. 1 agreen agreen 64 Jun 16 06:57 4 -> pipe:[795470]
l-wx------. 1 agreen agreen 64 Jun 16 06:57 5 -> pipe:[795470]
lrwx------. 1 agreen agreen 64 Jun 16 06:57 6 -> socket:[795479]
lrwx------. 1 agreen agreen 64 Jun 16 06:57 7 -> socket:[796963]

That's after leaving the client running for a few minutes on unchanged 
master test-client.  So it is closing the old sockets fine.

If it's not Linux, actually the reuse-of-lowest-fd strategy is not 
mandatory, open can monotonically increment the fd and wrap to reuse 
older ones on other platforms (ie, lwip on ESP32 does this).

> have found cases in wireshark when after I have connected with one port, 
> another SYN starts from a different port within the libwebsockets code. 
> I don't understand this.

If you can show the test apps doing it, I will worry about it.

> One question I have is that if I reconnect using the same context on a 
> different thread each time, would that cause problems?

You should only do this stuff from the same thread that is doing the 
service.

-Andy

> 
> 
> 
> ------------------------------------------------------------------------
> *From:* Andy Green <andy at warmcat.com>
> *Sent:* Thursday, June 15, 2017 1:39 AM
> *To:* Ash 20001; libwebsockets at ml.libwebsockets.org
> *Subject:* Re: [Libwebsockets] Is there a way to reuse a lws_context 
> without destroying it
> 
> 
> On 06/15/2017 04:24 PM, Ash 20001 wrote:
>> It looks like the second time I connect to a different port (with same 
>> context), I see some packets being sent on the first port (which is 
>> supposed to be closed). It's almost as if there are two sockets sitting 
> 
> Mm doubt it.
> 
> It's normal for some (zero payload length) tcp stuff housekeeping to fly
> around.
> 
>> around, and one is trying to connect back to the first port and the 
>> other is also connecting to the second port. This seems to be messing up 
>> my callbacks. If this is the problem, is there any way to clean up this 
>> old socket state without explicitly called destroy_context?
> 
> What I think would be good is find some way for me to reproduce this
> using the test server etc.
> 
> Like run 2 x test servers on different ports, and add a small hack on
> test client so that it switches between the two servers each time.  It
> should be fairly easy.
> 
> If that shows the problem, well, I guess it's my problem.  If it works,
> then the problem is tied up with your code.  It might sounds like "oh
> but that's not moving me forward" but actually either way you will learn
> something good for where the problem is (and get it solved by me if I
> can reproduce it with your changes to my pieces).
> 
> -Andy
> 
>> 
>> 
>> ------------------------------------------------------------------------
>> *From:* Andy Green <andy at warmcat.com>
>> *Sent:* Thursday, June 15, 2017 12:07 AM
>> *To:* Ash 20001; libwebsockets at ml.libwebsockets.org
>> *Subject:* Re: [Libwebsockets] Is there a way to reuse a lws_context 
>> without destroying it
>> 
>> 
>> On 06/15/2017 02:43 PM, Ash 20001 wrote:
>>>  1. lws 2.2 made no difference
>>>  2. i took a wireshark capture and found out the server indeed was
>>>     sending a FIN to the client. But the same server if I don't reuse
>>>     the context, it works, no spurious FIN. The server is running a java
>>>     version of websockets.  Any idea why context reuse could be screwing
>>>     up the server? i assume some state somewhere in the context. 
>>>
>> 
>> As I said look at what is sent with, eg, tcpdump -s0 -X.
>> 
>> If state in lws context decides it, but the remote side executes it, it
>> can only be that the remote side 'knew' the state by something lws sent
>> to it.
>> 
>> Look at what if anything is different about what you send to the host
>> the first and second time.
>> 
>> -Andy
>> 
>>> 
>>> 
>>> 
>>> ------------------------------------------------------------------------
>>> *From:* Andy Green <andy at warmcat.com>
>>> *Sent:* Wednesday, June 14, 2017 6:17 PM
>>> *To:* Ash 20001; libwebsockets at ml.libwebsockets.org
>>> *Subject:* Re: [Libwebsockets] Is there a way to reuse a lws_context 
>>> without destroying it
>>> 
>>> 
>>> On June 15, 2017 9:08:26 AM GMT+08:00, Ash 20001 <ash20001 at hotmail.com> 
>>> wrote:
>>>>I got the logs with the DEBUG flag on, does this mean the server closed
>>>>the connection?
>>>>
>>>>1003 :  2017/6/14 18:05:19.590DBG  :  [5708] lws:
>>>>lws_header_table_attach: wsi 0x7fddf002de00: ah 0x7fddf0009f70: count 1
>>>>(on exit)
>>>>1004 :  2017/6/14 18:05:19.590DBG  :  [5708] lws: lws_client_connect:
>>>>direct conn
>>>>1005 :  2017/6/14 18:05:19.590DBG  :  [5708] lws: lws_client_connect_2
>>>>1006 :  2017/6/14 18:05:19.590DBG  :  [5708] lws: lws_client_connect_2:
>>>>address 105.145.65.86
>>>>1022 :  2017/6/14 18:05:19.593DBG  :  [5708] lws:
>>>>insert_wsi_socket_into_fds: 0x7fddf002de00: tsi=0, sock=14,
>>>>pos-in-fds=2
>>>>1023 :  2017/6/14 18:05:19.593DBG  :  [5708] lws:
>>>>LWS_CALLBACK_WSI_CREATE
>>>>1024 :  2017/6/14 18:05:19.593DBG  :  [5708] lws: nonblocking connect
>>>>retry
>>>>1025 :  2017/6/14 18:05:19.593DBG  :  [5708] lws: wsc loop starting
>>>>1026 :  2017/6/14 18:05:19.593DBG  :  [5708] lws: service_fd: closing
>>>>due to 0 length read
>>> 
>>> This is how lws learns the peer closed the connection.
>>> 
>>> You can use tcpdump or so to look at the packets that were exchanged if 
>>> it's not SSL.
>>> 
>>> -Andy
>>> 
>>>>1027 :  2017/6/14 18:05:19.593DBG  :  [5708] lws: Close and handled
>>>>1028 :  2017/6/14 18:05:19.593DBG  :  [5708] lws: lws_close_free_wsi:
>>>>shutting down connection: 0x7fddf001f910
>>>>1029 :  2017/6/14 18:05:19.593DBG  :  [5708] lws: service_fd: closing
>>>>due to 0 length read
>>>>1030 :  2017/6/14 18:05:19.593DBG  :  [5708] lws: Close and handled
>>>>1031 :  2017/6/14 18:05:19.593DBG  :  [5708] lws: lws_close_free_wsi:
>>>>real just_kill_connection: 0x7fddf001f910
>>>>1032 :  2017/6/14 18:05:19.593DBG  :  [5708] lws:
>>>>remove_wsi_socket_from_fds: wsi=0x7fddf001f910, sock=13, fds pos=1, end
>>>>guy pos=3, endfd=0
>>>>1033 :  2017/6/14 18:05:19.593DBG  :  [5708] lws: calling back CLOSED
>>>>1034 :  2017/6/14 18:05:19.593DBG  :  [5708] lws: LWS_CALLBACK_CLOSED
>>>>
>>>>
>>>>
>>>>
>>>>________________________________
>>>>From: Andy Green <andy at warmcat.com>
>>>>Sent: Wednesday, June 14, 2017 3:53 PM
>>>>To: Ash 20001; libwebsockets at ml.libwebsockets.org
>>>>Subject: Re: [Libwebsockets] Is there a way to reuse a lws_context
>>>>without destroying it
>>>>
>>>>
>>>>
>>>>On June 15, 2017 6:45:33 AM GMT+08:00, Ash 20001 <ash20001 at hotmail.com>
>>>>wrote:
>>>>>1.  you are right, the test-client does handle closed/connection error
>>>>>callbacks, but the code itself does not initiate a disconnect from the
>>>>>server but rather it handles cases where the server dies...not sure if
>>>>>that would make a difference. in my application the client is
>>>>>disconnecting.
>>>>
>>>>No test-client also does initiate a disconnect from the server.  The
>>>>connection gets a random 'lifetime' when it is made, after sending that
>>>>many messages it closes the connection and after the ratelimit period
>>>>the outer loop will create a new one.
>>>>
>>>>>2.  I can try a later websockets however is there any logging I can
>>>>put
>>>>>into libwebsockets to know why after connection i get a
>>>>callback_closed
>>>>>event after some time randomly? The server is not disconnecting from
>>>>>what I can see.
>>>>
>>>>As I said
>>>>
>>>>>>In later lws in CONNECTION_ERROR, *in may point to a string
>>>>describing
>>>>>>the problem.
>>>>
>>>>If lws did something that failed and initiates the close, there are
>>>>like 20 different reason strings to help debug it.
>>>>
>>>>If the peer closes the socket though, 'in' will be NULL.
>>>>
>>>>You can turn up the logging to get additional clues, build in DEBUG
>>>>mode at cmake (see README.build.md)  and set the logging level high (eg
>>>>-d 65535 on the test client).
>>>>
>>>>-Andy
>>>>
>>>>>
>>>>>
>>>>>________________________________
>>>>>From: Andy Green <andy at warmcat.com>
>>>>>Sent: Wednesday, June 14, 2017 3:36 PM
>>>>>To: Ash 20001; libwebsockets at ml.libwebsockets.org
>>>>>Subject: Re: [Libwebsockets] Is there a way to reuse a lws_context
>>>>>without destroying it
>>>>>
>>>>>
>>>>>
>>>>>On June 15, 2017 1:29:46 AM GMT+08:00, Ash 20001
>>>><ash20001 at hotmail.com>
>>>>>wrote:
>>>>>>sorry i forgot to mention that on reconnect i am connecting to a
>>>>>>different port and potentially even different server ip. Is that why
>>>>>it
>>>>>>won't work with the same context?
>>>>>
>>>>>Doubt it, every connect starts fresh.
>>>>>
>>>>>>________________________________
>>>>>>From: Ash 20001 <ash20001 at hotmail.com>
>>>>>>Sent: Wednesday, June 14, 2017 8:47 AM
>>>>>>To: Andy Green
>>>>>>Subject: Re: [Libwebsockets] Is there a way to reuse a lws_context
>>>>>>without destroying it
>>>>>>
>>>>>>
>>>>>>I am using 1.7 websockets. When I try it again, I start immediately
>>>>>
>>>>>1.7 is too old... please see if you still have a problem on master
>>>>lws.
>>>>>
>>>>>>getting callback_closed messages after lws_client_connect_via_info is
>>>>>>called. I have no idea why I am getting these messages but if I
>>>>>destroy
>>>>>>the context and then call lws_Create_Context again, it works.
>>>>>
>>>>>In later lws in CONNECTION_ERROR, *in may point to a string describing
>>>>>the problem.
>>>>>
>>>>>>
>>>>>>In libwebsockets-test-client, it doesn't seem to handle callback
>>>>>closed
>>>>>>or callback connection error. Normally during these events I exit out
>>>>>
>>>>>It does handle them, or it doesn't know it should retry (mymemory only
>>>>>goesback to v2.0-ish though).
>>>>>
>>>>>>of the main lws_Service loop, and then later I try to call
>>>>>>connect_via_info again, restart the service loop, and then run into
>>>>>the
>>>>>>issue above. What should I do to prevent this?
>>>>>
>>>>>See if it still happens on later lws.
>>>>>
>>>>>>Yes openssl is being trashed and messing around with other threads
>>>>>>using curl to an https source for instance so I am trying to
>>>>>workaround
>>>>>>it because I cannot change the openssl version.
>>>>>
>>>>>You're going to have to change something...
>>>>>
>>>>>-Andy
>>>>>
>>>>>>
>>>>>>________________________________
>>>>>>From: Andy Green <andy at warmcat.com>
>>>>>>Sent: Wednesday, June 14, 2017 12:29 AM
>>>>>>To: libwebsockets at ml.libwebsockets.org; Ash 20001;
>>>>>>libwebsockets at ml.libwebsockets.org
>>>>>>Subject: Re: [Libwebsockets] Is there a way to reuse a lws_context
>>>>>>without destroying it
>>>>>>
>>>>>>
>>>>>>
>>>>>>On June 14, 2017 2:35:24 PM GMT+08:00, Ash 20001
>>>>><ash20001 at hotmail.com>
>>>>>>wrote:
>>>>>>>Is there a way to call lws_create_context just once for program
>>>>>>>lifetime as a client, and use that context for many connections (ie.
>>>>>>>connect to a server, disconnect, repeat)? I have tried but it
>>>>doesn't
>>>>>>>seem to work on the second connection.
>>>>>>
>>>>>>Yes it should be fine.
>>>>>>
>>>>>>Libwebsockets-test-client does exactly this in its normal operation.
>>>>>>
>>>>>>What exactly happens (and does it happen on lws master?)
>>>>>>
>>>>>>>The reason I ask is because I am having SSL issues with other
>>>>threads
>>>>>>>that I have no control over in a process which use ssl functions in
>>>>>>>CURL. lws_destroy_context globally destructs SSL state and causes
>>>>>>>issues.  My openssl version is rather old 0.9.8 and cannot be
>>>>>changed.
>>>>>>
>>>>>>Yeah.... we have seen it before.  Later openssl fixed it.
>>>>>>
>>>>>>It will trash openssl for the whole process.  Is that why you can't
>>>>>get
>>>>>>a second client connection in lws?
>>>>>>
>>>>>>-Andy
>>>>>>
>>>>>>>
>>>>>>>Thanks!
>>>>>>>
>>>>>>>(Sorry if I sent two emails, not sure if the first one went through
>>>>>>>because I was trying to subscribe to the mailing list).



More information about the Libwebsockets mailing list