[Libwebsockets] Packet size question

Andy Green andy at warmcat.com
Wed May 20 14:22:07 CEST 2020

On 5/20/20 1:13 PM, Brice Hamon wrote:
> Before we make modifications, what is the best approach?
> To chop the message in MTU chunks and call lws_write() for each packet 
> after we get a WRITE notification, or call lws_write() in a loop until 
> we get an error like TCP, then go back to the epoll loop for the remaining?

If at all possible, do not create / read / produce / store the output 
until it's able to send it, and then only the part that you are going to 

The best way to deal with it is ask for a new writeable callback for 
each chunk.  That will adapt to all kinds of situations cleanly like 
many connections sending, sending as h2 streams, ws-over-h2 etc.

On a particular machine under its normal conditions you might find it's 
no problem to send larger chunks up to several KB.  If it does 
occasionally meet a partial send, lws will conceal it (by stopping you 
seeing any more writeable callbacks until it has cleared it in the 
background).  You want some sweet spot between throughput and 
probability of having the partials using heap, whatever that is for your 
particular situation.


> Thank you.
> Brice.
> On Tue, May 19, 2020 at 5:53 PM Brice Hamon <normandviking at gmail.com 
> <mailto:normandviking at gmail.com>> wrote:
>     Hi Andy,
>     Ok got it. Thank you.
>     Yes it is withing ws.
>     So I'll chop the big buffers into MTU size packets and lws_write()
>     them, first n-1 with the FIN flag, and the last one without.
>     I'll report.
>     Thank you
>     Brice.
>     On Tue, May 19, 2020 at 5:33 PM Andy Green <andy at warmcat.com
>     <mailto:andy at warmcat.com>> wrote:
>         On 5/19/20 9:52 PM, Brice Hamon wrote:
>          > Hi guys,
>          >
>          > I was wondering about payload size in lws.
>          >
>          > Is there a maximum size pre-defined, or I can allocate a big
>         buffer and
>          > use lws_write() to let lws manage the fragmentation?
>          >
>          > So far we are hitting around a 273872 chars limit somewhere.
>         No, should should definitely chop it up yourself into something
>         reasonable.  lws doesn't have any fixed limit but if your
>         connection
>         gets choked, space will appear in the tcp window as things are
>         acked, if
>         the connection is relatively slow that will manifest itself as
>         the tcp
>         stack being willing to send "a bit more" measured in mtu-sized
>         packets.
>         Exactly how much it will accept depends on the platform, memory
>         pressure
>         and the state of the tcp window, but typically somewhere between
>         1 and 2
>         mtu is a good bet, so chunks of 1.4 .. 2.8KB.
>         If you try to send giant buffers it all goes on the heap and
>         it's very
>         inefficient in that for every send(), the whole of the giant
>         buffer is
>         copied into kernel space each time, even if it ends up the
>         network stack
>         only used 2KB of it.
>         If it's ws protocol you can mark the writes after the first as not
>         having a FIN, so it's a continuation of an existing ws message
>         in its
>         own frame.
>         If it's eg http body you can just keep sending the next part.
>         -Andy
>          > Thank you,
>          > Brice.
>          >
>          >
>          >
>          > _______________________________________________
>          > Libwebsockets mailing list
>          > Libwebsockets at ml.libwebsockets.org
>         <mailto:Libwebsockets at ml.libwebsockets.org>
>          > https://libwebsockets.org/mailman/listinfo/libwebsockets
>          >

More information about the Libwebsockets mailing list