[Libwebsockets] multithreaded client and ring buffer question

Andy Green andy at warmcat.com
Sat Jan 12 04:19:02 CET 2019



On 12/01/2019 09:50, Dave Horton wrote:
> Hi - I’m a bit of a newbie, looking forward to using this library to build a high-performance multithreaded websocket client.

It seems you understood the main point, which is lws runs in a single 
thread with an event loop.

   My program will need to establish multiple connections to many 
different far-end web servers, and send a large amount of near real-time 
data over those connections.

Multiple clients is no problem (in one event loop / thread).

>  From reading the README docs and examples (thanks!) I think I have a basic idea of what I need to do (but interested in key things I am missing):
> - While my program will have many threads that want to send, I will have one wsi service thread running my callback, and just call ‘lws_callback_on_writable’ from the other threads
> - I will use ringbuffers to stash the data waiting to go out (e.g waiting to get a LWS_CALLBACK_CLIENT_WRITEABLE event).

Sounds good.

> My first question is whether there is any documentation on the ringbuffer API that I could study?  It looks like there are a bunch of useful functions, but if there is a good overview doc on them it would be helpful.

Not really... there's some docs in the header, and "documentation" in 
the form of the related examples, eg

https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-server/minimal-ws-server-ring/protocol_lws_minimal.c
https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/client-server/minimal-ws-proxy/protocol_lws_minimal.c
https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/dbus-server/minimal-dbus-ws-proxy/protocol_lws_minimal_dbus_ws_proxy.c
https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/http-server/minimal-http-server-sse-ring/minimal-http-server-sse-ring.c
https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-client/minimal-ws-client-echo/protocol_lws_minimal_client_echo.c
https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-client/minimal-ws-client-tx/minimal-ws-client.c
https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-server/minimal-ws-broker/minimal-ws-broker.c
https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-server/minimal-ws-server-echo
https://libwebsockets.org/git/libwebsockets/tree/plugins/raw-proxy/protocol_lws_raw_proxy.c

 > Most of the ws client examples illustrate a client that initializes a 
context and then creates a client connection right away.
 >
 > My case is slightly different — at startup I need to create the 
context and then poll in the service thread, and then some time later a 
foreign thread needs to connect to a remote endpoint.
 >
 > I know how a foreign thread can call call lws_callback_on_writable 
when it has data to write on an existing connection, but how can it 
signal the service thread so as to cause a new client connection 
entirely to be made?

The best tool for thread synchronization is lws_cancel_service().  He's 
very robust (the foreign thread simply adds a byte into a pipe) and lws 
has automatically both created the pipe and set it up that any incoming 
data on the lws end of it "causes an event" in the event loop.  If the 
"event loop" is poll(), it means the poll wait is immediately stopped 
and lws will broadcast LWS_CALLBACK_EVENT_WAIT_CANCELLED to every 
protocol on every vhost.  In the case multiple threads called it before 
we can respond, lws reads and discards all the pipe content, so it only 
creates one cancel event instead of spamming n uselessly.

This is the recommended method to do any thread sync, including the 
"there's something to write in a ringbuffer".  In your code handling 
LWS_CALLBACK_EVENT_WAIT_CANCELLED, you can take your mutex protecting 
your shared structs / ringbuffers and find out what needs attention, 
calling on_writable() from the lws context on affected wsi.

> Any other pointers or guidance welcome..

You don't have to use lws_ring, you can use some other ringbuffer 
abstraction.  But I actually use lws_ring in difficult cases like the 
mirror protocol where it handles important but nonbvious situations like 
rx flow control management.

https://libwebsockets.org/git/libwebsockets/tree/plugins/protocol_lws_mirror.c

... so it may be worth the effort to get it working based on the examples.

-Andy

> Dave
> 
> _______________________________________________
> Libwebsockets mailing list
> Libwebsockets at ml.libwebsockets.org
> https://libwebsockets.org/mailman/listinfo/libwebsockets
> 


More information about the Libwebsockets mailing list