[Libwebsockets] Atomic Frame Size

Andy Green andy at warmcat.com
Mon Jan 26 05:13:29 CET 2015



On 11 January 2015 17:15:56 GMT+08:00, Duy Lan Nguyen <ndlan2k at gmail.com> wrote:
>Hi Andy,
>
>I've implemented item 3). I can contribute it to lws if you want.
>
>Do you have any news on items 1) and 2)?
>
>Thanks,
>Lan
>
>
>On Tue, Dec 23, 2014 at 7:26 PM, Duy Lan Nguyen <ndlan2k at gmail.com>
>wrote:
>
>> In 1) and 2), I mean the server goes down suddenly
>>
>> On Tue, Dec 23, 2014 at 7:17 PM, Duy Lan Nguyen <ndlan2k at gmail.com>
>wrote:
>>
>>> Hi Andy,
>>>
>>> I have some questions if you could help
>>> 1) I'm trying to let client handle gracefully when the server shuts
>down.
>>> With lots of traffic going back and forth (each of client and server
>has 2
>>> threads, one for sending and one for receiving), LWS_CALLBACK_CLOSED
>>> sometimes isn't caught by polling, and its callback routine (in the
>>> switch-case) isn't called. Is it possible to make
>LWS_CALLBACK_CLOSED
>>> never be dropped?

LWS never drops any close message AFAIK, but the OS may never become aware that the connection is dead.  It doesn't know if the reason everything is quiet is because everything is quiet, or because somewhere a network cable fell out.

You need to use keepalives of some sort (tcp, or PING / PONG in lws) to detect that.

>>> 2) And after the connection is already shutdown for a while (client
>>> doesn't know it as LWS_CALLBACK_CLOSED is dropped), calling
>>> libwebsocket_context_destroy() is hanged. Is it possible to avoid it
>>> hanging?

Sorry where does it hang in there?

>>> 3) We want to build a single-file client application, so the app
>needs to
>>> take the server CA cert from a memory blob instead from a file
>(info.
>>> ssl_ca_filepath). Can this be done?

If you want to send a patch I'm happy to look at it.

-Andy

>>> Many thanks,
>>> Lan
>>>
>>>
>>>
>>> On Sat, Dec 13, 2014 at 10:50 AM, Andy Green <andy at warmcat.com>
>wrote:
>>>
>>>>
>>>>
>>>> On 13 December 2014 23:42:18 GMT+08:00, Duy Lan Nguyen <
>>>> ndlan2k at gmail.com> wrote:
>>>> >Hi Andy,
>>>> >
>>>> >Just to make sure, the sample codes for the strategy you mentioned
>>>> >(wait
>>>> >for the WRITEABLE callback and the use a do { } while() to stuff
>the
>>>> >pipe
>>>> >with a frame of total length below "the number" until after the
>last
>>>> >one,
>>>> >it has no more POLLOUT for you) are in test-server.c,
>>>> >in LWS_CALLBACK_HTTP_WRITEABLE and LWS_CALLBACK_SERVER_WRITEABLE,
>where
>>>> >you
>>>> >used
>>>> >
>>>> >if (lws_partial_buffered(wsi) || lws_send_pipe_choked(wsi)) {
>>>> >libwebsocket_callback_on_writable(context, wsi);
>>>> >break;
>>>> >}
>>>> >
>>>> >to check if "it has no more POLLOUT for you". Is that right?
>>>>
>>>> Yes.  That way you can be sure you gave the kernel as much as it
>could
>>>> handle for that connection each time it had indicated you could
>give the
>>>> connection anything.
>>>>
>>>> -Andy
>>>>
>>>> >Many thanks,
>>>> >Lan
>>>> >
>>>> >
>>>> >
>>>> >On Mon, Nov 24, 2014 at 2:45 PM, Andy Green <andy at warmcat.com>
>wrote:
>>>> >>
>>>> >>
>>>> >>
>>>> >> On 22 November 2014 16:42:32 GMT+08:00, Duy Lan Nguyen
>>>> ><ndlan2k at gmail.com>
>>>> >> wrote:
>>>> >>
>>>> >> I am replying on the list.
>>>> >>
>>>> >> >Hi Andy,
>>>> >> >
>>>> >> >I'll need to chop 1 MB message to atomic frames to send and
>receive.
>>>> >It
>>>> >>
>>>> >> What is an "atomic frame"... if you mean a logical frame that is
>>>> >split
>>>> >> into fragments of a larger logical message and sent like that,
>yes
>>>> >you
>>>> >> should do that.
>>>> >>
>>>> >> >seems that the default atomic frame size is 4KB which is
>>>> >> >LWS_MAX_SOCKET_IO_BUF.
>>>> >> >
>>>> >> >Why should it be 4 KB? Or what should be the best atomic frame
>size,
>>>> >> >for
>>>> >> >example for 1 MB message?
>>>> >>
>>>> >> Well it depends on the server and the server load.
>>>> >>
>>>> >> We avoid blocking by waiting for a POLLOUT indication.  But that
>only
>>>> >> tells us we could send "something", it doesn't say how much we
>could
>>>> >send
>>>> >> without blocking.  If we get POLLOUT and try to send 10MB at
>once, we
>>>> >will
>>>> >> get a partial send.
>>>> >>
>>>> >> The reason is the kernel is not willing to allocate 10MB buffer
>on
>>>> >kernel
>>>> >> side.  He did give us POLLOUT though, so he is willing to take
>>>> >something.
>>>> >>
>>>> >> Empirically at least with linux on typical ethernet and WLAN
>setups,
>>>> >there
>>>> >> is "a number" below which we don't get a POLLOUT indication.  It
>>>> >seems to
>>>> >> be around an ethernet frame size, eg, 1500 even if the box is
>heavily
>>>> >> loaded (at least, on my boxes).  If the box is less loaded, it
>will
>>>> >accept
>>>> >> more.
>>>> >>
>>>> >> So one strategy is shown in the test server code, wait for the
>>>> >WRITEABLE
>>>> >> callback and the use a do { } while() to stuff the pipe with a
>frame
>>>> >of
>>>> >> total length below "the number" until after the last one, it has
>no
>>>> >more
>>>> >> POLLOUT for you.  Then if you still have more, ask for another
>>>> >WRITEABLE.
>>>> >> That way you should never block but also keep the kernel side as
>>>> >stuffed as
>>>> >> possible, and take advantage of packet coalesce if possible.
>>>> >>
>>>> >> Another strategy if you have a lot of memory is just rely on the
>>>> >partial
>>>> >> send buffering in lws.  Do ask to send 1MB, when the kernel just
>>>> >takes 2K
>>>> >> or whatever lws will copy the rest into a buffer and take care
>of
>>>> >> prioritizing the send of the remaining buffered data before that
>>>> >connection
>>>> >> will get another WRITABLE callback.  But 1) it's expensive
>allocating
>>>> >and
>>>> >> copying around the buffer, and 2) when you use big frames you
>block
>>>> >the
>>>> >> link for control frames like PING for long periods or from
>sending
>>>> >anything
>>>> >> else, so your latency can be hurt.
>>>> >>
>>>> >> -Andy
>>>> >>
>>>> >> >Many thanks,
>>>> >> >Lan
>>>> >>
>>>> >>
>>>>
>>>>
>>>
>>




More information about the Libwebsockets mailing list