[Libwebsockets] Packet size question

Brice Hamon normandviking at gmail.com
Wed May 20 21:11:39 CEST 2020


Thank you Andy,

I used that function in your example and it works perfectly.
So that limitation is now lifted. Cool.

Thank you,
Brice.


On Wed, May 20, 2020 at 2:00 PM Andy Green <andy at warmcat.com> wrote:

>
>
> On 5/20/20 6:21 PM, Brice Hamon wrote:
> > Hi Andy,
> >
> > We've implemented the message fragmentation to regular MTU size
> > - LWS_PRE so 1484.
> >
> > We chopped the big messages in chunks of 1484 and send it with proto
> > = LWS_WRITE_BINARY | LWS_WRITE_NO_FIN.
> > Then the last one with only proto = LWS_WRITE_BINARY.
> > What happens next is that we are getting a LWS_CALLBACK_CLOSED and the
> > connection gets closed, which is not good for our application as we are
> > not over http.
>
> That's not how ws works, it needs a BINARY on the first and CONTINUATION
> on the subsequent ones.
>
> There is a helper api here
>
>
> https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-write.h#n221-246
>
> that makes it easier to form the right flags.
>
> -Andy
>
> > If we send all packets with only LWS_WRITE_BINARY, all works well, and
> > we can send huge buffers across.
> >
> > So I don't know if our implementation is correct but so far sending the
> > big message in small chunks allowed us to send big messages,
> > we just don't use the LWS_WRITE_NO_FIN flag.
> >
> > Thank you
> > Brice.
> >
> >
> >
> >
> >
> >
> >
> >
> > On Wed, May 20, 2020 at 8:33 AM Brice Hamon <normandviking at gmail.com
> > <mailto:normandviking at gmail.com>> wrote:
> >
> >     Thanks Andy.
> >     We will go the safer route and ask for write readiness before
> >     actually calling lws_write().
> >     Thanks again,
> >     Brice.
> >
> >
> >     On Wed, May 20, 2020 at 8:22 AM Andy Green <andy at warmcat.com
> >     <mailto:andy at warmcat.com>> wrote:
> >
> >
> >
> >         On 5/20/20 1:13 PM, Brice Hamon wrote:
> >          > Before we make modifications, what is the best approach?
> >          > To chop the message in MTU chunks and call lws_write() for
> >         each packet
> >          > after we get a WRITE notification, or call lws_write() in a
> >         loop until
> >          > we get an error like TCP, then go back to the epoll loop for
> >         the remaining?
> >
> >         If at all possible, do not create / read / produce / store the
> >         output
> >         until it's able to send it, and then only the part that you are
> >         going to
> >         send.
> >
> >         The best way to deal with it is ask for a new writeable callback
> >         for
> >         each chunk.  That will adapt to all kinds of situations cleanly
> >         like
> >         many connections sending, sending as h2 streams, ws-over-h2 etc.
> >
> >         On a particular machine under its normal conditions you might
> >         find it's
> >         no problem to send larger chunks up to several KB.  If it does
> >         occasionally meet a partial send, lws will conceal it (by
> >         stopping you
> >         seeing any more writeable callbacks until it has cleared it in
> the
> >         background).  You want some sweet spot between throughput and
> >         probability of having the partials using heap, whatever that is
> >         for your
> >         particular situation.
> >
> >         -Andy
> >
> >          > Thank you.
> >          > Brice.
> >          >
> >          > On Tue, May 19, 2020 at 5:53 PM Brice Hamon
> >         <normandviking at gmail.com <mailto:normandviking at gmail.com>
> >          > <mailto:normandviking at gmail.com
> >         <mailto:normandviking at gmail.com>>> wrote:
> >          >
> >          >     Hi Andy,
> >          >
> >          >     Ok got it. Thank you.
> >          >
> >          >     Yes it is withing ws.
> >          >     So I'll chop the big buffers into MTU size packets and
> >         lws_write()
> >          >     them, first n-1 with the FIN flag, and the last one
> without.
> >          >     I'll report.
> >          >     Thank you
> >          >     Brice.
> >          >
> >          >
> >          >
> >          >
> >          >     On Tue, May 19, 2020 at 5:33 PM Andy Green
> >         <andy at warmcat.com <mailto:andy at warmcat.com>
> >          >     <mailto:andy at warmcat.com <mailto:andy at warmcat.com>>>
> wrote:
> >          >
> >          >
> >          >
> >          >         On 5/19/20 9:52 PM, Brice Hamon wrote:
> >          >          > Hi guys,
> >          >          >
> >          >          > I was wondering about payload size in lws.
> >          >          >
> >          >          > Is there a maximum size pre-defined, or I can
> >         allocate a big
> >          >         buffer and
> >          >          > use lws_write() to let lws manage the
> fragmentation?
> >          >          >
> >          >          > So far we are hitting around a 273872 chars limit
> >         somewhere.
> >          >
> >          >         No, should should definitely chop it up yourself into
> >         something
> >          >         reasonable.  lws doesn't have any fixed limit but if
> your
> >          >         connection
> >          >         gets choked, space will appear in the tcp window as
> >         things are
> >          >         acked, if
> >          >         the connection is relatively slow that will manifest
> >         itself as
> >          >         the tcp
> >          >         stack being willing to send "a bit more" measured in
> >         mtu-sized
> >          >         packets.
> >          >         Exactly how much it will accept depends on the
> >         platform, memory
> >          >         pressure
> >          >         and the state of the tcp window, but typically
> >         somewhere between
> >          >         1 and 2
> >          >         mtu is a good bet, so chunks of 1.4 .. 2.8KB.
> >          >
> >          >         If you try to send giant buffers it all goes on the
> >         heap and
> >          >         it's very
> >          >         inefficient in that for every send(), the whole of
> >         the giant
> >          >         buffer is
> >          >         copied into kernel space each time, even if it ends
> >         up the
> >          >         network stack
> >          >         only used 2KB of it.
> >          >
> >          >         If it's ws protocol you can mark the writes after the
> >         first as not
> >          >         having a FIN, so it's a continuation of an existing
> >         ws message
> >          >         in its
> >          >         own frame.
> >          >
> >          >         If it's eg http body you can just keep sending the
> >         next part.
> >          >
> >          >         -Andy
> >          >
> >          >          > Thank you,
> >          >          > Brice.
> >          >          >
> >          >          >
> >          >          >
> >          >          > _______________________________________________
> >          >          > Libwebsockets mailing list
> >          >          > Libwebsockets at ml.libwebsockets.org
> >         <mailto:Libwebsockets at ml.libwebsockets.org>
> >          >         <mailto:Libwebsockets at ml.libwebsockets.org
> >         <mailto:Libwebsockets at ml.libwebsockets.org>>
> >          >          >
> >         https://libwebsockets.org/mailman/listinfo/libwebsockets
> >          >          >
> >          >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://libwebsockets.org/pipermail/libwebsockets/attachments/20200520/4f6949a4/attachment-0001.htm>


More information about the Libwebsockets mailing list