[Libwebsockets] multithreaded client and ring buffer question

Dave Horton daveh at beachdognet.com
Mon Jan 14 03:04:42 CET 2019


Thanks so much for the help on this - its much appreciated.

I'm starting to build my client application now (slowly), and I want to
make sure I understand a few things about what a "wsi" represents, and what
a vhost is.

So far, I have a fairly simply client application that creates a context
with a protocol and starts the event loop.

My protocols structure looks like this:

  static const struct lws_protocols protocols[] = {
    {
      "audiostream.drachtio.org",
      lws_callback,
      0,
      0,
    },
    { NULL, NULL, 0, 0 }
  };

then I am simply calling 'lws-create_context' and then looping calling
'lws_service'.

I am simply logging out the callbacks that are then invoked (not yet doing
anything with them), and I see that the callback is invoked with either of
two wsi pointers:
one has no vhost associated, and the other has a vhost of 'default'.

I'm unclear why there are two vhosts, which each represents, and whether I
need to be doing anything in the callback with the null vhost.

When the service loops starts (and before I attempt any connections) I get
these 3 callbacks:

lws_callback wsi: 0x7fffd79758b0, vhost (null), reason:
LWS_CALLBACK_GET_THREAD_ID
lws_callback wsi: 0x7fffd7975570, vhost: default, reason:
LWS_CALLBACK_EVENT_WAIT_CANCELLED
lws_callback wsi: 0x7fffd7975440, vhost: default, reason:
LWS_CALLBACK_PROTOCOL_INIT

What is the purpose of the callback in the wsi with null vhost?  In
general, am I expected to handle these callbacks when vhost=NULL in any
particular fashion?
****



On Sat, Jan 12, 2019 at 1:33 PM Andy Green <andy at warmcat.com> wrote:

>
>
> On January 12, 2019 11:53:37 PM GMT+08:00, Dave Horton <
> daveh at beachdognet.com> wrote:
> >Trying to understand suggested use of
> >LWS_CALLBACK_EVENT_WAIT_CANCELLED....
> >
> >I notice in minimal-ws-client.c, where you have multiple spam threads
> >creating content, the foreign/spam threads
> >call LWS_CALLBACK_EVENT_WAIT_CANCELLED,
> >and then in the service loop the handler calls
> >'lws_callback_on_writable'.
> >
> >Could not the foreign thread(s) have called lws_callback_on_writable
> >directly?  I want to make sure I follow the best / suggested practice.
>
> lws_cancel_service() is safe against reentrancy, it writes to a pipe and
> job done.  So 2 or more threads can call it without locking.
>
> on_writable() calls through to the event loop library to change POLLOUT
> waits.  Although the lws part can handle a single foreign thread calling
> it, libuv or whatever can't.
>
> On busy systems, cancel_service() is not as expensive as it sounds.  It is
> only checked in the service thread on exit from the event loop wait.
>
> -Andy
>
> >On Fri, Jan 11, 2019 at 10:19 PM Andy Green <andy at warmcat.com> wrote:
> >
> >>
> >>
> >> On 12/01/2019 09:50, Dave Horton wrote:
> >> > Hi - I’m a bit of a newbie, looking forward to using this library
> >to
> >> build a high-performance multithreaded websocket client.
> >>
> >> It seems you understood the main point, which is lws runs in a single
> >> thread with an event loop.
> >>
> >>    My program will need to establish multiple connections to many
> >> different far-end web servers, and send a large amount of near
> >real-time
> >> data over those connections.
> >>
> >> Multiple clients is no problem (in one event loop / thread).
> >>
> >> >  From reading the README docs and examples (thanks!) I think I have
> >a
> >> basic idea of what I need to do (but interested in key things I am
> >missing):
> >> > - While my program will have many threads that want to send, I will
> >have
> >> one wsi service thread running my callback, and just call
> >> ‘lws_callback_on_writable’ from the other threads
> >> > - I will use ringbuffers to stash the data waiting to go out (e.g
> >> waiting to get a LWS_CALLBACK_CLIENT_WRITEABLE event).
> >>
> >> Sounds good.
> >>
> >> > My first question is whether there is any documentation on the
> >> ringbuffer API that I could study?  It looks like there are a bunch
> >of
> >> useful functions, but if there is a good overview doc on them it
> >would be
> >> helpful.
> >>
> >> Not really... there's some docs in the header, and "documentation" in
> >> the form of the related examples, eg
> >>
> >>
> >>
> >
> https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-server/minimal-ws-server-ring/protocol_lws_minimal.c
> >>
> >>
> >
> https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/client-server/minimal-ws-proxy/protocol_lws_minimal.c
> >>
> >>
> >
> https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/dbus-server/minimal-dbus-ws-proxy/protocol_lws_minimal_dbus_ws_proxy.c
> >>
> >>
> >
> https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/http-server/minimal-http-server-sse-ring/minimal-http-server-sse-ring.c
> >>
> >>
> >
> https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-client/minimal-ws-client-echo/protocol_lws_minimal_client_echo.c
> >>
> >>
> >
> https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-client/minimal-ws-client-tx/minimal-ws-client.c
> >>
> >>
> >
> https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-server/minimal-ws-broker/minimal-ws-broker.c
> >>
> >>
> >
> https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-server/minimal-ws-server-echo
> >>
> >>
> >
> https://libwebsockets.org/git/libwebsockets/tree/plugins/raw-proxy/protocol_lws_raw_proxy.c
> >>
> >>  > Most of the ws client examples illustrate a client that
> >initializes a
> >> context and then creates a client connection right away.
> >>  >
> >>  > My case is slightly different — at startup I need to create the
> >> context and then poll in the service thread, and then some time later
> >a
> >> foreign thread needs to connect to a remote endpoint.
> >>  >
> >>  > I know how a foreign thread can call call lws_callback_on_writable
> >> when it has data to write on an existing connection, but how can it
> >> signal the service thread so as to cause a new client connection
> >> entirely to be made?
> >>
> >> The best tool for thread synchronization is lws_cancel_service().
> >He's
> >> very robust (the foreign thread simply adds a byte into a pipe) and
> >lws
> >> has automatically both created the pipe and set it up that any
> >incoming
> >> data on the lws end of it "causes an event" in the event loop.  If
> >the
> >> "event loop" is poll(), it means the poll wait is immediately stopped
> >> and lws will broadcast LWS_CALLBACK_EVENT_WAIT_CANCELLED to every
> >> protocol on every vhost.  In the case multiple threads called it
> >before
> >> we can respond, lws reads and discards all the pipe content, so it
> >only
> >> creates one cancel event instead of spamming n uselessly.
> >>
> >> This is the recommended method to do any thread sync, including the
> >> "there's something to write in a ringbuffer".  In your code handling
> >> LWS_CALLBACK_EVENT_WAIT_CANCELLED, you can take your mutex protecting
> >> your shared structs / ringbuffers and find out what needs attention,
> >> calling on_writable() from the lws context on affected wsi.
> >>
> >> > Any other pointers or guidance welcome..
> >>
> >> You don't have to use lws_ring, you can use some other ringbuffer
> >> abstraction.  But I actually use lws_ring in difficult cases like the
> >> mirror protocol where it handles important but nonbvious situations
> >like
> >> rx flow control management.
> >>
> >>
> >>
> >
> https://libwebsockets.org/git/libwebsockets/tree/plugins/protocol_lws_mirror.c
> >>
> >> ... so it may be worth the effort to get it working based on the
> >examples.
> >>
> >> -Andy
> >>
> >> > Dave
> >> >
> >> > _______________________________________________
> >> > Libwebsockets mailing list
> >> > Libwebsockets at ml.libwebsockets.org
> >> > https://libwebsockets.org/mailman/listinfo/libwebsockets
> >> >
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://libwebsockets.org/pipermail/libwebsockets/attachments/20190113/3df37bd8/attachment.html>


More information about the Libwebsockets mailing list