[Libwebsockets] Execute callback on event loop thread

[Andy Green]

On 7/24/21 12:34 AM, Ivan Kolesnikov wrote:
> Hi,
> 
> I have two threads in my application: the first is doing my own stuff 
> and the second is running a loop with a lws_service call. And sometimes 
> I want to send some data via WebSocket from the first thread. But 
> WebSockets API is not thread-safe as I understand, so I want to execute 

Yes.

> some callback which will do actual sending on the event loop thread. 
> Like as uv_async_t in libuv can do that.
> 
> Is there any API in libwebsockets which allows it to execute some 
> callback on event loop thread? Or is there a possibility to obtain a 
> uv_loop_t instance to use uv_async_t? (I am using libwebsockets with 
> libuv support enabled).

The recommended way atm is prepare an object in memory shared by the 
threads and protected with your own locking with a list of what you want 
to happen from lws thread context, and then call lws_cancel_service().

This will wake the lws event loop thread by putting a byte in a pipe2() 
(unix) or sending a byte on a UDP socket created for that purpose 
(Windows and RTOS), and since those are monitored by the event loop wait 
whether it is the default one or libuv or any event library the same, 
this works the same and is safe across all the platforms and event lib 
combinations.  It also means lws itself doesn't have to know your 
locking arrangements, since that only relates to the shared object you 
control only from your code.

When the event loop wait wakes, it sees it woke from 
lws_cancel_service() and broadcasts LWS_CALLBACK_EVENT_WAIT_CANCELLED 
callback on all vhost-protocol combinations.  You can pick that up on 
your protocol, lock your shared object and then perform whatever the 
other threads wanted to happen from there.


There's also a scheme built on top of that available, that has semantics 
closer to what you want but requires more set up.  At init you provide 
an lws_system callback that has your locking around a call back into 
lws, you can then use that to manage a list of specified callbacks and 
opaque args that you can add to from other threads safely then without 
more locking, from your thread you call like

	lws_system_get_ops(context)->attach(context, 0, mycallback,
					    LWS_SYSTATE_OPERATIONAL,
					    my_void_ptr_arg, NULL);

and later, when lws reaches OPERATIONAL state, you will get a callback 
to mycallback(my_void_ptr_arg).  There is a minimal example showing the 
whole thing here

https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/http-client/minimal-http-client-attach

> The another solution came to my mind also - use lws_sul_schedule to 
> schedule my callback within one microsecond, but it seems to be not 
> thread-safe too if libwebsockets compiled without SMP support (and SMP 
> cannot be enabled without pthread library which is not available on 
> Win32 MSVC target which is one of these I am compiling - I am using 
> threading support from libuv in my program, so I have no dependency on 
> pthread).

Right SMP doesn't make things generally threadsafe somehow, it just does 
enough using pthreads so lws is safe against itself when running n event 
loops in different threads, and the internal lws code has to cooperate 
on shared data, eg, context or vhost -owned data.

-Andy