[Libwebsockets] Atomic Frame Size

Andy Green andy at warmcat.com
Mon Nov 24 23:45:25 CET 2014



On 22 November 2014 16:42:32 GMT+08:00, Duy Lan Nguyen <ndlan2k at gmail.com> wrote:

I am replying on the list.

>Hi Andy,
>
>I'll need to chop 1 MB message to atomic frames to send and receive. It

What is an "atomic frame"... if you mean a logical frame that is split into fragments of a larger logical message and sent like that, yes you should do that.

>seems that the default atomic frame size is 4KB which is
>LWS_MAX_SOCKET_IO_BUF.
>
>Why should it be 4 KB? Or what should be the best atomic frame size,
>for
>example for 1 MB message?

Well it depends on the server and the server load.

We avoid blocking by waiting for a POLLOUT indication.  But that only tells us we could send "something", it doesn't say how much we could send without blocking.  If we get POLLOUT and try to send 10MB at once, we will get a partial send.

The reason is the kernel is not willing to allocate 10MB buffer on kernel side.  He did give us POLLOUT though, so he is willing to take something.

Empirically at least with linux on typical ethernet and WLAN setups, there is "a number" below which we don't get a POLLOUT indication.  It seems to be around an ethernet frame size, eg, 1500 even if the box is heavily loaded (at least, on my boxes).  If the box is less loaded, it will accept more.

So one strategy is shown in the test server code, wait for the WRITEABLE callback and the use a do { } while() to stuff the pipe with a frame of total length below "the number" until after the last one, it has no more POLLOUT for you.  Then if you still have more, ask for another WRITEABLE.  That way you should never block but also keep the kernel side as stuffed as possible, and take advantage of packet coalesce if possible.

Another strategy if you have a lot of memory is just rely on the partial send buffering in lws.  Do ask to send 1MB, when the kernel just takes 2K or whatever lws will copy the rest into a buffer and take care of prioritizing the send of the remaining buffered data before that connection will get another WRITABLE callback.  But 1) it's expensive allocating and copying around the buffer, and 2) when you use big frames you block the link for control frames like PING for long periods or from sending anything else, so your latency can be hurt.

-Andy

>Many thanks,
>Lan




More information about the Libwebsockets mailing list