[Libwebsockets] Packet size question

Brice Hamon normandviking at gmail.com
Wed May 20 19:21:34 CEST 2020


Hi Andy,

We've implemented the message fragmentation to regular MTU size - LWS_PRE
so 1484.

We chopped the big messages in chunks of 1484 and send it with proto
= LWS_WRITE_BINARY | LWS_WRITE_NO_FIN.
Then the last one with only  proto = LWS_WRITE_BINARY.
What happens next is that we are getting a LWS_CALLBACK_CLOSED and the
connection gets closed, which is not good for our application as we are not
over http.

If we send all packets with only  LWS_WRITE_BINARY, all works well, and we
can send huge buffers across.

So I don't know if our implementation is correct but so far sending the big
message in small chunks allowed us to send big messages,
we just don't use the  LWS_WRITE_NO_FIN flag.

Thank you
Brice.








On Wed, May 20, 2020 at 8:33 AM Brice Hamon <normandviking at gmail.com> wrote:

> Thanks Andy.
> We will go the safer route and ask for write readiness before actually
> calling lws_write().
> Thanks again,
> Brice.
>
>
> On Wed, May 20, 2020 at 8:22 AM Andy Green <andy at warmcat.com> wrote:
>
>>
>>
>> On 5/20/20 1:13 PM, Brice Hamon wrote:
>> > Before we make modifications, what is the best approach?
>> > To chop the message in MTU chunks and call lws_write() for each packet
>> > after we get a WRITE notification, or call lws_write() in a loop until
>> > we get an error like TCP, then go back to the epoll loop for the
>> remaining?
>>
>> If at all possible, do not create / read / produce / store the output
>> until it's able to send it, and then only the part that you are going to
>> send.
>>
>> The best way to deal with it is ask for a new writeable callback for
>> each chunk.  That will adapt to all kinds of situations cleanly like
>> many connections sending, sending as h2 streams, ws-over-h2 etc.
>>
>> On a particular machine under its normal conditions you might find it's
>> no problem to send larger chunks up to several KB.  If it does
>> occasionally meet a partial send, lws will conceal it (by stopping you
>> seeing any more writeable callbacks until it has cleared it in the
>> background).  You want some sweet spot between throughput and
>> probability of having the partials using heap, whatever that is for your
>> particular situation.
>>
>> -Andy
>>
>> > Thank you.
>> > Brice.
>> >
>> > On Tue, May 19, 2020 at 5:53 PM Brice Hamon <normandviking at gmail.com
>> > <mailto:normandviking at gmail.com>> wrote:
>> >
>> >     Hi Andy,
>> >
>> >     Ok got it. Thank you.
>> >
>> >     Yes it is withing ws.
>> >     So I'll chop the big buffers into MTU size packets and lws_write()
>> >     them, first n-1 with the FIN flag, and the last one without.
>> >     I'll report.
>> >     Thank you
>> >     Brice.
>> >
>> >
>> >
>> >
>> >     On Tue, May 19, 2020 at 5:33 PM Andy Green <andy at warmcat.com
>> >     <mailto:andy at warmcat.com>> wrote:
>> >
>> >
>> >
>> >         On 5/19/20 9:52 PM, Brice Hamon wrote:
>> >          > Hi guys,
>> >          >
>> >          > I was wondering about payload size in lws.
>> >          >
>> >          > Is there a maximum size pre-defined, or I can allocate a big
>> >         buffer and
>> >          > use lws_write() to let lws manage the fragmentation?
>> >          >
>> >          > So far we are hitting around a 273872 chars limit somewhere.
>> >
>> >         No, should should definitely chop it up yourself into something
>> >         reasonable.  lws doesn't have any fixed limit but if your
>> >         connection
>> >         gets choked, space will appear in the tcp window as things are
>> >         acked, if
>> >         the connection is relatively slow that will manifest itself as
>> >         the tcp
>> >         stack being willing to send "a bit more" measured in mtu-sized
>> >         packets.
>> >         Exactly how much it will accept depends on the platform, memory
>> >         pressure
>> >         and the state of the tcp window, but typically somewhere between
>> >         1 and 2
>> >         mtu is a good bet, so chunks of 1.4 .. 2.8KB.
>> >
>> >         If you try to send giant buffers it all goes on the heap and
>> >         it's very
>> >         inefficient in that for every send(), the whole of the giant
>> >         buffer is
>> >         copied into kernel space each time, even if it ends up the
>> >         network stack
>> >         only used 2KB of it.
>> >
>> >         If it's ws protocol you can mark the writes after the first as
>> not
>> >         having a FIN, so it's a continuation of an existing ws message
>> >         in its
>> >         own frame.
>> >
>> >         If it's eg http body you can just keep sending the next part.
>> >
>> >         -Andy
>> >
>> >          > Thank you,
>> >          > Brice.
>> >          >
>> >          >
>> >          >
>> >          > _______________________________________________
>> >          > Libwebsockets mailing list
>> >          > Libwebsockets at ml.libwebsockets.org
>> >         <mailto:Libwebsockets at ml.libwebsockets.org>
>> >          > https://libwebsockets.org/mailman/listinfo/libwebsockets
>> >          >
>> >
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://libwebsockets.org/pipermail/libwebsockets/attachments/20200520/b37f3b5b/attachment.htm>


More information about the Libwebsockets mailing list