[Libwebsockets] Atomic Frame Size

Andy Green andy at warmcat.com
Sat Dec 13 19:50:37 CET 2014



On 13 December 2014 23:42:18 GMT+08:00, Duy Lan Nguyen <ndlan2k at gmail.com> wrote:
>Hi Andy,
>
>Just to make sure, the sample codes for the strategy you mentioned
>(wait
>for the WRITEABLE callback and the use a do { } while() to stuff the
>pipe
>with a frame of total length below "the number" until after the last
>one,
>it has no more POLLOUT for you) are in test-server.c,
>in LWS_CALLBACK_HTTP_WRITEABLE and LWS_CALLBACK_SERVER_WRITEABLE, where
>you
>used
>
>if (lws_partial_buffered(wsi) || lws_send_pipe_choked(wsi)) {
>libwebsocket_callback_on_writable(context, wsi);
>break;
>}
>
>to check if "it has no more POLLOUT for you". Is that right?

Yes.  That way you can be sure you gave the kernel as much as it could handle for that connection each time it had indicated you could give the connection anything.

-Andy

>Many thanks,
>Lan
>
>
>
>On Mon, Nov 24, 2014 at 2:45 PM, Andy Green <andy at warmcat.com> wrote:
>>
>>
>>
>> On 22 November 2014 16:42:32 GMT+08:00, Duy Lan Nguyen
><ndlan2k at gmail.com>
>> wrote:
>>
>> I am replying on the list.
>>
>> >Hi Andy,
>> >
>> >I'll need to chop 1 MB message to atomic frames to send and receive.
>It
>>
>> What is an "atomic frame"... if you mean a logical frame that is
>split
>> into fragments of a larger logical message and sent like that, yes
>you
>> should do that.
>>
>> >seems that the default atomic frame size is 4KB which is
>> >LWS_MAX_SOCKET_IO_BUF.
>> >
>> >Why should it be 4 KB? Or what should be the best atomic frame size,
>> >for
>> >example for 1 MB message?
>>
>> Well it depends on the server and the server load.
>>
>> We avoid blocking by waiting for a POLLOUT indication.  But that only
>> tells us we could send "something", it doesn't say how much we could
>send
>> without blocking.  If we get POLLOUT and try to send 10MB at once, we
>will
>> get a partial send.
>>
>> The reason is the kernel is not willing to allocate 10MB buffer on
>kernel
>> side.  He did give us POLLOUT though, so he is willing to take
>something.
>>
>> Empirically at least with linux on typical ethernet and WLAN setups,
>there
>> is "a number" below which we don't get a POLLOUT indication.  It
>seems to
>> be around an ethernet frame size, eg, 1500 even if the box is heavily
>> loaded (at least, on my boxes).  If the box is less loaded, it will
>accept
>> more.
>>
>> So one strategy is shown in the test server code, wait for the
>WRITEABLE
>> callback and the use a do { } while() to stuff the pipe with a frame
>of
>> total length below "the number" until after the last one, it has no
>more
>> POLLOUT for you.  Then if you still have more, ask for another
>WRITEABLE.
>> That way you should never block but also keep the kernel side as
>stuffed as
>> possible, and take advantage of packet coalesce if possible.
>>
>> Another strategy if you have a lot of memory is just rely on the
>partial
>> send buffering in lws.  Do ask to send 1MB, when the kernel just
>takes 2K
>> or whatever lws will copy the rest into a buffer and take care of
>> prioritizing the send of the remaining buffered data before that
>connection
>> will get another WRITABLE callback.  But 1) it's expensive allocating
>and
>> copying around the buffer, and 2) when you use big frames you block
>the
>> link for control frames like PING for long periods or from sending
>anything
>> else, so your latency can be hurt.
>>
>> -Andy
>>
>> >Many thanks,
>> >Lan
>>
>>




More information about the Libwebsockets mailing list