[Libwebsockets] Atomic Frame Size

Duy Lan Nguyen ndlan2k at gmail.com
Mon Mar 23 08:57:30 CET 2015


Hi Andy,

I still have the problem of libwebsocket_context_destroy() hanging, but I
still don't understand its cause. It happens as follows in my Windows
machine.

In a client-server connection, the server is suddenly shut down. The client
doesn't realize it after a while, and finally closes down itself. When the
client calls libwebsocket_context_destroy(), it hangs in the following loop
inside libwebsocket_context_destroy()  as wsi is never NULL

for (n = 0; n < context->fds_count; n++) {
struct libwebsocket *wsi =
context->lws_lookup[context->fds[n].fd];
if (!wsi)
continue;
libwebsocket_close_and_free_session(context,
wsi, LWS_CLOSE_STATUS_NOSTATUS /* no protocol close */);
n--;
}

This happens as inside libwebsocket_close_and_free_session(), the code
keeps going into

case WSI_STATE_FLUSHING_STORED_SEND_BEFORE_CLOSE:
if (wsi->truncated_send_len) {
libwebsocket_callback_on_writable(context, wsi);
return;
}

Did I do something wrong here or could you describe a solution to this
hanging problem?

Thanks,
Lan


On Sun, Jan 25, 2015 at 8:13 PM, Andy Green <andy at warmcat.com> wrote:

>
>
> On 11 January 2015 17:15:56 GMT+08:00, Duy Lan Nguyen <ndlan2k at gmail.com>
> wrote:
> >Hi Andy,
> >
> >I've implemented item 3). I can contribute it to lws if you want.
> >
> >Do you have any news on items 1) and 2)?
> >
> >Thanks,
> >Lan
> >
> >
> >On Tue, Dec 23, 2014 at 7:26 PM, Duy Lan Nguyen <ndlan2k at gmail.com>
> >wrote:
> >
> >> In 1) and 2), I mean the server goes down suddenly
> >>
> >> On Tue, Dec 23, 2014 at 7:17 PM, Duy Lan Nguyen <ndlan2k at gmail.com>
> >wrote:
> >>
> >>> Hi Andy,
> >>>
> >>> I have some questions if you could help
> >>> 1) I'm trying to let client handle gracefully when the server shuts
> >down.
> >>> With lots of traffic going back and forth (each of client and server
> >has 2
> >>> threads, one for sending and one for receiving), LWS_CALLBACK_CLOSED
> >>> sometimes isn't caught by polling, and its callback routine (in the
> >>> switch-case) isn't called. Is it possible to make
> >LWS_CALLBACK_CLOSED
> >>> never be dropped?
>
> LWS never drops any close message AFAIK, but the OS may never become aware
> that the connection is dead.  It doesn't know if the reason everything is
> quiet is because everything is quiet, or because somewhere a network cable
> fell out.
>
> You need to use keepalives of some sort (tcp, or PING / PONG in lws) to
> detect that.
>
> >>> 2) And after the connection is already shutdown for a while (client
> >>> doesn't know it as LWS_CALLBACK_CLOSED is dropped), calling
> >>> libwebsocket_context_destroy() is hanged. Is it possible to avoid it
> >>> hanging?
>
> Sorry where does it hang in there?
>
> >>> 3) We want to build a single-file client application, so the app
> >needs to
> >>> take the server CA cert from a memory blob instead from a file
> >(info.
> >>> ssl_ca_filepath). Can this be done?
>
> If you want to send a patch I'm happy to look at it.
>
> -Andy
>
> >>> Many thanks,
> >>> Lan
> >>>
> >>>
> >>>
> >>> On Sat, Dec 13, 2014 at 10:50 AM, Andy Green <andy at warmcat.com>
> >wrote:
> >>>
> >>>>
> >>>>
> >>>> On 13 December 2014 23:42:18 GMT+08:00, Duy Lan Nguyen <
> >>>> ndlan2k at gmail.com> wrote:
> >>>> >Hi Andy,
> >>>> >
> >>>> >Just to make sure, the sample codes for the strategy you mentioned
> >>>> >(wait
> >>>> >for the WRITEABLE callback and the use a do { } while() to stuff
> >the
> >>>> >pipe
> >>>> >with a frame of total length below "the number" until after the
> >last
> >>>> >one,
> >>>> >it has no more POLLOUT for you) are in test-server.c,
> >>>> >in LWS_CALLBACK_HTTP_WRITEABLE and LWS_CALLBACK_SERVER_WRITEABLE,
> >where
> >>>> >you
> >>>> >used
> >>>> >
> >>>> >if (lws_partial_buffered(wsi) || lws_send_pipe_choked(wsi)) {
> >>>> >libwebsocket_callback_on_writable(context, wsi);
> >>>> >break;
> >>>> >}
> >>>> >
> >>>> >to check if "it has no more POLLOUT for you". Is that right?
> >>>>
> >>>> Yes.  That way you can be sure you gave the kernel as much as it
> >could
> >>>> handle for that connection each time it had indicated you could
> >give the
> >>>> connection anything.
> >>>>
> >>>> -Andy
> >>>>
> >>>> >Many thanks,
> >>>> >Lan
> >>>> >
> >>>> >
> >>>> >
> >>>> >On Mon, Nov 24, 2014 at 2:45 PM, Andy Green <andy at warmcat.com>
> >wrote:
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> On 22 November 2014 16:42:32 GMT+08:00, Duy Lan Nguyen
> >>>> ><ndlan2k at gmail.com>
> >>>> >> wrote:
> >>>> >>
> >>>> >> I am replying on the list.
> >>>> >>
> >>>> >> >Hi Andy,
> >>>> >> >
> >>>> >> >I'll need to chop 1 MB message to atomic frames to send and
> >receive.
> >>>> >It
> >>>> >>
> >>>> >> What is an "atomic frame"... if you mean a logical frame that is
> >>>> >split
> >>>> >> into fragments of a larger logical message and sent like that,
> >yes
> >>>> >you
> >>>> >> should do that.
> >>>> >>
> >>>> >> >seems that the default atomic frame size is 4KB which is
> >>>> >> >LWS_MAX_SOCKET_IO_BUF.
> >>>> >> >
> >>>> >> >Why should it be 4 KB? Or what should be the best atomic frame
> >size,
> >>>> >> >for
> >>>> >> >example for 1 MB message?
> >>>> >>
> >>>> >> Well it depends on the server and the server load.
> >>>> >>
> >>>> >> We avoid blocking by waiting for a POLLOUT indication.  But that
> >only
> >>>> >> tells us we could send "something", it doesn't say how much we
> >could
> >>>> >send
> >>>> >> without blocking.  If we get POLLOUT and try to send 10MB at
> >once, we
> >>>> >will
> >>>> >> get a partial send.
> >>>> >>
> >>>> >> The reason is the kernel is not willing to allocate 10MB buffer
> >on
> >>>> >kernel
> >>>> >> side.  He did give us POLLOUT though, so he is willing to take
> >>>> >something.
> >>>> >>
> >>>> >> Empirically at least with linux on typical ethernet and WLAN
> >setups,
> >>>> >there
> >>>> >> is "a number" below which we don't get a POLLOUT indication.  It
> >>>> >seems to
> >>>> >> be around an ethernet frame size, eg, 1500 even if the box is
> >heavily
> >>>> >> loaded (at least, on my boxes).  If the box is less loaded, it
> >will
> >>>> >accept
> >>>> >> more.
> >>>> >>
> >>>> >> So one strategy is shown in the test server code, wait for the
> >>>> >WRITEABLE
> >>>> >> callback and the use a do { } while() to stuff the pipe with a
> >frame
> >>>> >of
> >>>> >> total length below "the number" until after the last one, it has
> >no
> >>>> >more
> >>>> >> POLLOUT for you.  Then if you still have more, ask for another
> >>>> >WRITEABLE.
> >>>> >> That way you should never block but also keep the kernel side as
> >>>> >stuffed as
> >>>> >> possible, and take advantage of packet coalesce if possible.
> >>>> >>
> >>>> >> Another strategy if you have a lot of memory is just rely on the
> >>>> >partial
> >>>> >> send buffering in lws.  Do ask to send 1MB, when the kernel just
> >>>> >takes 2K
> >>>> >> or whatever lws will copy the rest into a buffer and take care
> >of
> >>>> >> prioritizing the send of the remaining buffered data before that
> >>>> >connection
> >>>> >> will get another WRITABLE callback.  But 1) it's expensive
> >allocating
> >>>> >and
> >>>> >> copying around the buffer, and 2) when you use big frames you
> >block
> >>>> >the
> >>>> >> link for control frames like PING for long periods or from
> >sending
> >>>> >anything
> >>>> >> else, so your latency can be hurt.
> >>>> >>
> >>>> >> -Andy
> >>>> >>
> >>>> >> >Many thanks,
> >>>> >> >Lan
> >>>> >>
> >>>> >>
> >>>>
> >>>>
> >>>
> >>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libwebsockets.org/pipermail/libwebsockets/attachments/20150323/a59d7e63/attachment.html>


More information about the Libwebsockets mailing list