[Libwebsockets] How can I send async data using libwebsocket?

Andy Green andy at warmcat.com
Mon Apr 12 08:15:20 CEST 2021



On 4/12/21 7:01 AM, Trung Do wrote:
> Hi Andy,
> 
> Thank you for your help. I have a few questions:
> 
>   - Looking at the example you linked, it seems the key part is calling
> lws_callback_on_writable() inside the handler code for case
> LWS_CALLBACK_SERVER_WRITEABLE?

If you have more to send, that is the correct pattern.

>   - If I understand correctly, lws_callback_on_writable() will
> eventually trigger LWS_CALLBACK_SERVER_WRITEABLE again. Wouldn't that
> turn into an infinite loop and burn CPU cycles if we have nothing to
> write?

No.  Because that code looks like this

                 /* more to do? */
                 if (lws_ring_get_element(vhd->ring, &pss->tail))
                         /* come back as soon as we can write more */
                         lws_callback_on_writable(pss->wsi);

https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-server/minimal-ws-server-threads/protocol_lws_minimal.c#n255-258

when it has exhausted what it wants to send, it stops asking for more 
writeable callbacks.

Also the point of the WRITEABLE callback is, you don't get one until you 
can write more.  If the peer is not accepting more packets, or the 
network cable fell out or whatever, it will be some time before you get 
another, time the event loop can be servicing other connections.  So it 
will adapt to the connection's experience of network conditions and peer 
state.

It's true if the connection can accept more, and you keep asking for 
another writeable callback when you had no use for it, it will keep 
giving them to you quickly, that's an undesirable bug in the user code 
(and it is not the logic of the example, as explained above).  But even 
this is not spinning in the sense we are in a tight loop, stuck there, 
it is going around the event loop each time and can service other 
connections inbetween, see that this connection closed and stop doing 
that, etc.

>   - Do I need to call lws_callback_on_writable() inside case
> LWS_CALLBACK_ESTABLISHED? I see that is done is the stackoverflow
> answer but not in the example code you linked.

If you want to send something as soon as you got a new connection, yes. 
  If you don't, because your protocol is to wait for the client to say 
something first when it connects, no.

>   - Like I mentioned in my first message, I'm writing an implementation
> for the client side, so I assume instead of
> LWS_CALLBACK_SERVER_WRITEABLE, I need to use
> LWS_CALLBACK_CLIENT_WRITEABLE instead?

Yes I only point to that ws server example because I wondered if you are 
actually using threads.  If not, you can just follow a ws client example 
that matches better.

-Andy

> Thank you again for your help, I really appreciate it.
> 
> Trung
> 
> On Mon, 12 Apr 2021 at 01:31, Andy Green <andy at warmcat.com> wrote:
>>
>>
>>
>> On 4/12/21 2:39 AM, Trung Do wrote:
>>> Hello,
>>>
>>> I am having the exact question as this stackoverflow one:
>>>
>>> https://stackoverflow.com/questions/14261076/how-do-i-send-async-data-via-libwebsocket
>>
>> Well, at least you're asking it in the right place.
>>
>>> In short, after having connected a websocket client to a server, how
>>> can I send data to the server at any time I want, instead of just in
>>
>> Neither you nor anybody else can "send data to the server any time
>> [they] want".  You might be able to send a packet if your connectivity,
>> network conditions, and the remote peer permit it.  But nobody has the
>> spare memory to deal with huge mountains of data you wanted to naively
>> issue that could not be forwarded or at least lined up for forwarding
>> more or less immediately.
>>
>> This network programming reality is the same with or without blocking
>> semantics.  With blocking your thread is just blocked until you're
>> actually able to send something, it's not visible in the code but
>> write() or send() suffers unlimited narcolepsy when it is disallowed
>> from sending anything.  In fact a way to make timeouts becomes a problem
>> with blocking semantics since your sending thread has no upper limit on
>> how long it can be forcibly blocked waiting to actually be able to send
>> the next piece.  And it's just inert and unable to do anything else
>> while blocked.
>>
>> Because the threads block for most of their life, blocking doesn't scale
>> either since you must have an OS thread per connection, just so it can
>> block for when it can send something next to maintain this delusion that
>> the user code can loop "sending as much as it wants".
>>
>>> the callback that gets called when the server sends me something?
>>
>> In nonblocking event loops, you are informed when you can send again on
>> that connection with an event.  The rest of the time while waiting for
>> that, your thread can be busy with other connections or other work.
>>
>> Blocking is like "please hold", and nonblocking is "I'll call you back".
>>
>>> There was one answer in that stackoverflow link but it's from 2013 and
>>> I'm not sure if it's the correct/recommended approach, plus a lot of
>>> the method names have changed. I have digged through the examples but
>>> haven't found anything for this use case.
>>
>> Unfortunately there's a lot of cruft there that I neither wrote nor
>> control.  But there are the minimal examples in-tree the last few years
>> that show the recent best practice.
>>
>> You won't find anything for this "use case" because that is not how
>> nonblocking event loops work.  You should accept your sending action is
>> like a per-wsi state machine inside the WRITEABLE callback.  If you
>> can't get your head around it, well, ok, there are probably other libs
>> out there that offer blocking apis.  But AFAIK the others are also
>> nonblocking event loop based, because it is significantly superior.
>>
>> If you actually wanted to ask "how can I ask for the writeable callback
>> on connections from another thread", look at ws-server-threads
>>
>> https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-server/minimal-ws-server-threads/protocol_lws_minimal.c#n124
>>
>> https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-server/minimal-ws-server-threads/protocol_lws_minimal.c#n124-266
>>
>> -Andy


More information about the Libwebsockets mailing list