[Libwebsockets] lws_callback_on_writable

andy at warmcat.com andy at warmcat.com
Sat Mar 14 01:41:14 CET 2020

On March 13, 2020 10:58:10 PM UTC, Pranjali Chumbhale <pranjalic at safeai.ai> wrote:
>Hello Andy,
>Do you know what is the data size that a libwebsocket can send in one
>Like if I do a lws_write on a client side, It will not block the
>It makes the websocket client writable as soon as that data is sent.
>Like I can put a limit on the amount of data allowed to be sent at a

Lws is just passing your data through to a POSIX socket, it can restrict the size it sends at one go using the numbers in the struct lws_protocols but typically that's not the limiting factor.

Lws is nonblocking, it will never literally block on sending.  Instead there will just be a longer interval between asking for a callback on writeable, and getting the callback.  In that interval, lws is either idle (sleeping) if nothing to do, or servicing timed events, events on other sockets etc.

So what decides the rate that you can write on the socket?  Ultimately the kernel also has finite memory and for the same reasons you can't just buffer everything in userland heap regardless, he can't just offer to take any amount of outgoing data into kernelside memory either.  The kernel wants to have some packets-worth in its memory prepared to go out immediately it gets a chance to send them for efficiency, so it will spend a little memory on that, but it's motivated to keep a very tight leash on getting ahead of what the far end can consume.

So how does the kernel know the state of consumption on the link, or what is in-flight?  Tcp has an adaptive 'window' that uses acks from the far end to make decisions at the networking stack layer about the performance of the link, and regulate the speed new packets may be sent according to how the peer is reporting its consumption.  The window size is changed to estimate what's a good amount of data to have in-flight on that link dynamically... two datacentres talking to each other is quite different than if one side is on a 56kbps modem, tcp sniffs out the situation and adapts automatically.

How links differ, how tcp works, how your network stack works with that are the underlying, unyielding realities of network programming.  Lws presents all that succinctly as an interval between wanting to write and being able to write.

Underneath, lws does that by using poll() or another network event listener to learn from the network stack when it signals (driven by tcp ack receipt) that the kernel wants to replenish its little stock of prepared tx data.  At that point, the kernel doesn't make any guarantee about how much it will accept, just that it'd like to get 'some' more data to send.  Typically, at least on linux it seems to be prepared to take at least one or two mtu at once, so 1.5 - 3KB, but it can take larger lumps at connection start or on loopback device.  It's legal but unlikelyit would only accept 1 byte; lws will buffer on heap anything the kernel didn't accept for you and send it in the background so you can survive that, but it's not efficient to use routinely.

So the points are, as the sender on a network the rate you can transmit is not yours to decide, tcp and individual link state decides it for you; and a good 'chunk' to send at once is in the region of 1 - 2  mtu.


>On Fri, Mar 13, 2020 at 11:49 AM <andy at warmcat.com> wrote:
>> On March 13, 2020 6:28:14 PM UTC, Pranjali Chumbhale
><pranjalic at safeai.ai>
>> wrote:
>> >Hello Team,
>> >
>> >Is it possible to call lws_callback_on_writable with the data to be
>> >written?
>> No... lws' concept is only produce data "just in time" and don't
>> the system has endless heap.  "Just in time" means when the
>> stack indicates it can take more data for that particular connection,
>> should produce it and send it.
>> When you realized you want to send something and when you actually
>> send something are two different things that may not have much
>> relationship... normally it'll take more in ms, but if your phone
>> cells or you go in a tunnel or drops into the sea it may not take
>more data
>> for seconds or minutes.  So they're treated as two different things.
>> If you need to make it look like one thing you can use a side buffer
>> lws_buflist to hold your data for when the socket does signal its
>> writeble.  But that disconnects that data production from being
>> by the connection... eg if you had 10MB or 100MB or whatevetto send
>it will
>> lead to all 10MB or whatever going on the heap if the connection is
>> This doesn't scale or work on small devices whereas the lws way does.
>> -Andy

More information about the Libwebsockets mailing list