[Libwebsockets] Atomic Frame Size

Duy Lan Nguyen ndlan2k at gmail.com
Wed Apr 1 17:50:32 CEST 2015


Turn out that just removing #include stdint.h is enough and lws still works
fine.

I made another change to allow using cert blobs instead of using cert
files, with cyassl. I can provide that as well?

Should I email changes to you or push it to git?

Thanks

On Sat, Mar 28, 2015 at 5:34 AM, Andy Green <andy at warmcat.com> wrote:

>
>
> On 28 March 2015 19:09:53 GMT+08:00, Duy Lan Nguyen <ndlan2k at gmail.com>
> wrote:
> >Hi Andy,
> >
> >Would it be possible to make this change?
>
> Sure, send a patch.
>
> The only windows this gets built on, an autobuilder, it builds already.
>
> If your patch makes it build for you and doesn't break the autobuilder
> then fine...
>
> -Andy
>
> >We'd greatly appreciate!!!
> >
> >On Thu, Mar 26, 2015 at 10:17 PM, Duy Lan Nguyen <ndlan2k at gmail.com>
> >wrote:
> >
> >> We need to use Visual Studio 2008 (VS 9) with libwebsockets to be
> >> compatible with our Driver DK. It was ok with previous lws but the
> >latest
> >> lws doesn't compile as libwebsockets.h includes stdint.h that's not
> >yet
> >> available in VS 9. Could you please fix that so that lws will work
> >with VS
> >> 9?
> >>
> >> Thanks!!
> >>
> >> On Wed, Mar 25, 2015 at 1:43 PM, Duy Lan Nguyen <ndlan2k at gmail.com>
> >wrote:
> >>
> >>> Awesome. Thanks
> >>>
> >>>
> >>>
> >>> On Tue, Mar 24, 2015 at 10:55 AM, Andy Green <andy at warmcat.com>
> >wrote:
> >>>
> >>>>
> >>>>
> >>>> On 23 March 2015 22:28:45 GMT+08:00, Duy Lan Nguyen
> ><ndlan2k at gmail.com>
> >>>> wrote:
> >>>> >Yes, it's SSL connection
> >>>>
> >>>> I just pushed something that should solve it.
> >>>>
> >>>> -Andy
> >>>>
> >>>> >-----Original Message-----
> >>>> >From: "Andy Green" <andy at warmcat.com>
> >>>> >Sent: ‎3/‎23/‎2015 4:02 AM
> >>>> >To: "Duy Lan Nguyen" <ndlan2k at gmail.com>
> >>>> >Cc: "libwebsockets at ml.libwebsockets.org"
> >>>> ><libwebsockets at ml.libwebsockets.org>
> >>>> >Subject: Re: Atomic Frame Size
> >>>> >
> >>>> >
> >>>> >
> >>>> >On 23 March 2015 15:57:30 GMT+08:00, Duy Lan Nguyen
> ><ndlan2k at gmail.com>
> >>>> >wrote:
> >>>> >>Hi Andy,
> >>>> >>
> >>>> >>I still have the problem of libwebsocket_context_destroy()
> >hanging,
> >>>> >but
> >>>> >>I
> >>>> >>still don't understand its cause. It happens as follows in my
> >Windows
> >>>> >>machine.
> >>>> >>
> >>>> >>In a client-server connection, the server is suddenly shut down.
> >The
> >>>> >>client
> >>>> >>doesn't realize it after a while, and finally closes down itself.
> >When
> >>>> >>the
> >>>> >>client calls libwebsocket_context_destroy(), it hangs in the
> >following
> >>>> >>loop
> >>>> >>inside libwebsocket_context_destroy()  as wsi is never NULL
> >>>> >>
> >>>> >>for (n = 0; n < context->fds_count; n++) {
> >>>> >>struct libwebsocket *wsi =
> >>>> >>context->lws_lookup[context->fds[n].fd];
> >>>> >>if (!wsi)
> >>>> >>continue;
> >>>> >>libwebsocket_close_and_free_session(context,
> >>>> >>wsi, LWS_CLOSE_STATUS_NOSTATUS /* no protocol close */);
> >>>> >>n--;
> >>>> >>}
> >>>> >>
> >>>> >>This happens as inside libwebsocket_close_and_free_session(), the
> >code
> >>>> >>keeps going into
> >>>> >>
> >>>> >>case WSI_STATE_FLUSHING_STORED_SEND_BEFORE_CLOSE:
> >>>> >>if (wsi->truncated_send_len) {
> >>>> >>libwebsocket_callback_on_writable(context, wsi);
> >>>> >>return;
> >>>> >>}
> >>>> >>
> >>>> >>Did I do something wrong here or could you describe a solution to
> >this
> >>>> >
> >>>> >Probably nothing more wrong than using windows.
> >>>> >
> >>>> >>hanging problem?
> >>>> >
> >>>> >I guess there's a case where he has something he wants to send
> >before
> >>>> >dying, but he can't send it.  Is it an SSL connection?
> >>>> >
> >>>> >-Andy
> >>>> >
> >>>> >>Thanks,
> >>>> >>Lan
> >>>> >>
> >>>> >>
> >>>> >>On Sun, Jan 25, 2015 at 8:13 PM, Andy Green <andy at warmcat.com>
> >wrote:
> >>>> >>
> >>>> >>>
> >>>> >>>
> >>>> >>> On 11 January 2015 17:15:56 GMT+08:00, Duy Lan Nguyen
> >>>> >><ndlan2k at gmail.com>
> >>>> >>> wrote:
> >>>> >>> >Hi Andy,
> >>>> >>> >
> >>>> >>> >I've implemented item 3). I can contribute it to lws if you
> >want.
> >>>> >>> >
> >>>> >>> >Do you have any news on items 1) and 2)?
> >>>> >>> >
> >>>> >>> >Thanks,
> >>>> >>> >Lan
> >>>> >>> >
> >>>> >>> >
> >>>> >>> >On Tue, Dec 23, 2014 at 7:26 PM, Duy Lan Nguyen
> ><ndlan2k at gmail.com>
> >>>> >>> >wrote:
> >>>> >>> >
> >>>> >>> >> In 1) and 2), I mean the server goes down suddenly
> >>>> >>> >>
> >>>> >>> >> On Tue, Dec 23, 2014 at 7:17 PM, Duy Lan Nguyen
> >>>> >><ndlan2k at gmail.com>
> >>>> >>> >wrote:
> >>>> >>> >>
> >>>> >>> >>> Hi Andy,
> >>>> >>> >>>
> >>>> >>> >>> I have some questions if you could help
> >>>> >>> >>> 1) I'm trying to let client handle gracefully when the
> >server
> >>>> >>shuts
> >>>> >>> >down.
> >>>> >>> >>> With lots of traffic going back and forth (each of client
> >and
> >>>> >>server
> >>>> >>> >has 2
> >>>> >>> >>> threads, one for sending and one for receiving),
> >>>> >>LWS_CALLBACK_CLOSED
> >>>> >>> >>> sometimes isn't caught by polling, and its callback routine
> >(in
> >>>> >>the
> >>>> >>> >>> switch-case) isn't called. Is it possible to make
> >>>> >>> >LWS_CALLBACK_CLOSED
> >>>> >>> >>> never be dropped?
> >>>> >>>
> >>>> >>> LWS never drops any close message AFAIK, but the OS may never
> >become
> >>>> >>aware
> >>>> >>> that the connection is dead.  It doesn't know if the reason
> >>>> >>everything is
> >>>> >>> quiet is because everything is quiet, or because somewhere a
> >network
> >>>> >>cable
> >>>> >>> fell out.
> >>>> >>>
> >>>> >>> You need to use keepalives of some sort (tcp, or PING / PONG in
> >lws)
> >>>> >>to
> >>>> >>> detect that.
> >>>> >>>
> >>>> >>> >>> 2) And after the connection is already shutdown for a while
> >>>> >>(client
> >>>> >>> >>> doesn't know it as LWS_CALLBACK_CLOSED is dropped), calling
> >>>> >>> >>> libwebsocket_context_destroy() is hanged. Is it possible to
> >>>> >avoid
> >>>> >>it
> >>>> >>> >>> hanging?
> >>>> >>>
> >>>> >>> Sorry where does it hang in there?
> >>>> >>>
> >>>> >>> >>> 3) We want to build a single-file client application, so
> >the app
> >>>> >>> >needs to
> >>>> >>> >>> take the server CA cert from a memory blob instead from a
> >file
> >>>> >>> >(info.
> >>>> >>> >>> ssl_ca_filepath). Can this be done?
> >>>> >>>
> >>>> >>> If you want to send a patch I'm happy to look at it.
> >>>> >>>
> >>>> >>> -Andy
> >>>> >>>
> >>>> >>> >>> Many thanks,
> >>>> >>> >>> Lan
> >>>> >>> >>>
> >>>> >>> >>>
> >>>> >>> >>>
> >>>> >>> >>> On Sat, Dec 13, 2014 at 10:50 AM, Andy Green
> ><andy at warmcat.com>
> >>>> >>> >wrote:
> >>>> >>> >>>
> >>>> >>> >>>>
> >>>> >>> >>>>
> >>>> >>> >>>> On 13 December 2014 23:42:18 GMT+08:00, Duy Lan Nguyen <
> >>>> >>> >>>> ndlan2k at gmail.com> wrote:
> >>>> >>> >>>> >Hi Andy,
> >>>> >>> >>>> >
> >>>> >>> >>>> >Just to make sure, the sample codes for the strategy you
> >>>> >>mentioned
> >>>> >>> >>>> >(wait
> >>>> >>> >>>> >for the WRITEABLE callback and the use a do { } while()
> >to
> >>>> >>stuff
> >>>> >>> >the
> >>>> >>> >>>> >pipe
> >>>> >>> >>>> >with a frame of total length below "the number" until
> >after
> >>>> >the
> >>>> >>> >last
> >>>> >>> >>>> >one,
> >>>> >>> >>>> >it has no more POLLOUT for you) are in test-server.c,
> >>>> >>> >>>> >in LWS_CALLBACK_HTTP_WRITEABLE and
> >>>> >>LWS_CALLBACK_SERVER_WRITEABLE,
> >>>> >>> >where
> >>>> >>> >>>> >you
> >>>> >>> >>>> >used
> >>>> >>> >>>> >
> >>>> >>> >>>> >if (lws_partial_buffered(wsi) ||
> >lws_send_pipe_choked(wsi)) {
> >>>> >>> >>>> >libwebsocket_callback_on_writable(context, wsi);
> >>>> >>> >>>> >break;
> >>>> >>> >>>> >}
> >>>> >>> >>>> >
> >>>> >>> >>>> >to check if "it has no more POLLOUT for you". Is that
> >right?
> >>>> >>> >>>>
> >>>> >>> >>>> Yes.  That way you can be sure you gave the kernel as much
> >as
> >>>> >it
> >>>> >>> >could
> >>>> >>> >>>> handle for that connection each time it had indicated you
> >could
> >>>> >>> >give the
> >>>> >>> >>>> connection anything.
> >>>> >>> >>>>
> >>>> >>> >>>> -Andy
> >>>> >>> >>>>
> >>>> >>> >>>> >Many thanks,
> >>>> >>> >>>> >Lan
> >>>> >>> >>>> >
> >>>> >>> >>>> >
> >>>> >>> >>>> >
> >>>> >>> >>>> >On Mon, Nov 24, 2014 at 2:45 PM, Andy Green
> ><andy at warmcat.com>
> >>>> >>> >wrote:
> >>>> >>> >>>> >>
> >>>> >>> >>>> >>
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> On 22 November 2014 16:42:32 GMT+08:00, Duy Lan Nguyen
> >>>> >>> >>>> ><ndlan2k at gmail.com>
> >>>> >>> >>>> >> wrote:
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> I am replying on the list.
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> >Hi Andy,
> >>>> >>> >>>> >> >
> >>>> >>> >>>> >> >I'll need to chop 1 MB message to atomic frames to
> >send and
> >>>> >>> >receive.
> >>>> >>> >>>> >It
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> What is an "atomic frame"... if you mean a logical
> >frame
> >>>> >that
> >>>> >>is
> >>>> >>> >>>> >split
> >>>> >>> >>>> >> into fragments of a larger logical message and sent
> >like
> >>>> >>that,
> >>>> >>> >yes
> >>>> >>> >>>> >you
> >>>> >>> >>>> >> should do that.
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> >seems that the default atomic frame size is 4KB which
> >is
> >>>> >>> >>>> >> >LWS_MAX_SOCKET_IO_BUF.
> >>>> >>> >>>> >> >
> >>>> >>> >>>> >> >Why should it be 4 KB? Or what should be the best
> >atomic
> >>>> >>frame
> >>>> >>> >size,
> >>>> >>> >>>> >> >for
> >>>> >>> >>>> >> >example for 1 MB message?
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> Well it depends on the server and the server load.
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> We avoid blocking by waiting for a POLLOUT indication.
> >But
> >>>> >>that
> >>>> >>> >only
> >>>> >>> >>>> >> tells us we could send "something", it doesn't say how
> >much
> >>>> >>we
> >>>> >>> >could
> >>>> >>> >>>> >send
> >>>> >>> >>>> >> without blocking.  If we get POLLOUT and try to send
> >10MB at
> >>>> >>> >once, we
> >>>> >>> >>>> >will
> >>>> >>> >>>> >> get a partial send.
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> The reason is the kernel is not willing to allocate
> >10MB
> >>>> >>buffer
> >>>> >>> >on
> >>>> >>> >>>> >kernel
> >>>> >>> >>>> >> side.  He did give us POLLOUT though, so he is willing
> >to
> >>>> >>take
> >>>> >>> >>>> >something.
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> Empirically at least with linux on typical ethernet and
> >WLAN
> >>>> >>> >setups,
> >>>> >>> >>>> >there
> >>>> >>> >>>> >> is "a number" below which we don't get a POLLOUT
> >indication.
> >>>> >
> >>>> >>It
> >>>> >>> >>>> >seems to
> >>>> >>> >>>> >> be around an ethernet frame size, eg, 1500 even if the
> >box
> >>>> >is
> >>>> >>> >heavily
> >>>> >>> >>>> >> loaded (at least, on my boxes).  If the box is less
> >loaded,
> >>>> >>it
> >>>> >>> >will
> >>>> >>> >>>> >accept
> >>>> >>> >>>> >> more.
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> So one strategy is shown in the test server code, wait
> >for
> >>>> >>the
> >>>> >>> >>>> >WRITEABLE
> >>>> >>> >>>> >> callback and the use a do { } while() to stuff the pipe
> >with
> >>>> >>a
> >>>> >>> >frame
> >>>> >>> >>>> >of
> >>>> >>> >>>> >> total length below "the number" until after the last
> >one, it
> >>>> >>has
> >>>> >>> >no
> >>>> >>> >>>> >more
> >>>> >>> >>>> >> POLLOUT for you.  Then if you still have more, ask for
> >>>> >>another
> >>>> >>> >>>> >WRITEABLE.
> >>>> >>> >>>> >> That way you should never block but also keep the
> >kernel
> >>>> >side
> >>>> >>as
> >>>> >>> >>>> >stuffed as
> >>>> >>> >>>> >> possible, and take advantage of packet coalesce if
> >possible.
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> Another strategy if you have a lot of memory is just
> >rely on
> >>>> >>the
> >>>> >>> >>>> >partial
> >>>> >>> >>>> >> send buffering in lws.  Do ask to send 1MB, when the
> >kernel
> >>>> >>just
> >>>> >>> >>>> >takes 2K
> >>>> >>> >>>> >> or whatever lws will copy the rest into a buffer and
> >take
> >>>> >>care
> >>>> >>> >of
> >>>> >>> >>>> >> prioritizing the send of the remaining buffered data
> >before
> >>>> >>that
> >>>> >>> >>>> >connection
> >>>> >>> >>>> >> will get another WRITABLE callback.  But 1) it's
> >expensive
> >>>> >>> >allocating
> >>>> >>> >>>> >and
> >>>> >>> >>>> >> copying around the buffer, and 2) when you use big
> >frames
> >>>> >you
> >>>> >>> >block
> >>>> >>> >>>> >the
> >>>> >>> >>>> >> link for control frames like PING for long periods or
> >from
> >>>> >>> >sending
> >>>> >>> >>>> >anything
> >>>> >>> >>>> >> else, so your latency can be hurt.
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> -Andy
> >>>> >>> >>>> >>
> >>>> >>> >>>> >> >Many thanks,
> >>>> >>> >>>> >> >Lan
> >>>> >>> >>>> >>
> >>>> >>> >>>> >>
> >>>> >>> >>>>
> >>>> >>> >>>>
> >>>> >>> >>>
> >>>> >>> >>
> >>>> >>>
> >>>> >>>
> >>>>
> >>>>
> >>>
> >>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libwebsockets.org/pipermail/libwebsockets/attachments/20150401/5410e995/attachment.html>


More information about the Libwebsockets mailing list