[Libwebsockets] Interaction of external threads with libwebsockets server

Andy Green andy at warmcat.com
Wed Jan 1 15:38:08 CET 2014



Thomas Spitz <thomas.spitz at hestia-france.com> wrote:
>Hello Andy,
>
>Have you had some time trying to replace poll by ppoll in order to have
>poll triggered on signal from an external thread?

Not yet... in Taiwan the big holiday is Chinese New Year in a few weeks.  I'm still interested in doing it, the weekend is the most likely time.

-Andy

>At the present your lib works very well with intensive data submit from
>external thread.
>
>Happy new year to everyone.
>
>BR,
>
>Thomas
>
>
>On 25 Dec 2013 15:24, Andy Green (林安廸) <andy at warmcat.com> wrote:
>
>> On 25/12/13 20:14, the mail apparently from Thomas Spitz included:
>>
>>>     The choices seem to boil down to this kind of "add a fake
>>>     descriptor" thing (although everything, including the "interrupt
>the
>>>     poll" descriptor and the use of it should be defined inside the
>>>     library), or maybe change to use ppoll() and fire signals at it.
>>>
>>> ppoll() could be an interesting solution but it only interrupt the
>poll.
>>>
>>
>> I think that's all we need to do.
>>
>> Latency is only coming this way on an idle system where we are
>sleeping in
>> the poll(), but another thread asked to change what a pollfd was
>waiting on.
>>
>> As you pointed out originally, under those circumstances the changed
>> pollfd rules won't be seen and handled -- if every fd is idle for the
>> events it started out with -- until the poll() timeout expires.
>>
>> Although in other use-cases this isn't that realistic as a problem,
>since
>> some deal with tens of thousands of simultaneous connections and
>usually
>> someone is breaking the poll after a short time for service, in other
>use
>> cases it is realistic.  You can attack it by reducing the poll sleep
>period
>> but then you're looking at maybe hundreds of wakes a second on what
>should
>> be an idle system, needlessly bad for power.
>>
>> If we provided a way for those use-cases to have very long poll()
>timeouts
>> and minimal latency it's good I think, so long as it doesn't burden
>or make
>> problems when it's not wanted or needed.
>>
>>  I was thinking of interrupting poll() using a named pipe in which I
>>> would have told lws which wsi it needs to write to. The complete
>process
>>> would have been the following:
>>>
>>
>> No it's not a good way... lws already has a good semantic in poll()
>for
>> understanding who needed service.  This would be a lot of new stuff
>doing
>> the same job that only works in the multithreaded case.
>>
>>  1) Before libwebsocket_create_context(), I create the named pipe.
>>> 2) For every client connection, I book for a shm
>>> in LWS_CALLBACK_ESTABLISHED through which I will share incoming data
>>> with my main thread
>>> 3) My main thread process the incoming data and store the answer
>into
>>> the shm. It then indicates lws that an answer is ready for a given
>wsi
>>> indicating the ID of the shm in the named pipe
>>> 4) lws poll() is interupted and it knows immediatly which wsi it
>needs
>>> to write to thanks to the ID of the shm. If the wsi is closed in the
>>> meanwhile, lws indicate it to the shm in LWS_CALLBACK_CLOSED
>>>
>>> If I use ppoll(), I could keep almost the same principle but I would
>>> then need to add a SIGUSR1 and a handler OR loop through my client
>shm
>>> array each time ppoll() got interupted with EINTR flag set...
>Finally I
>>> am still wondering whether my solution is not simpler?
>>>
>>
>> That solution is basically a threaded rewrite of lws not using
>poll(). If
>> you're interested to do that I don't want to discourage you, but it's
>> something different from lws then.  Of course lws is liberally
>licensed so
>> you're welcome to build on it if you have a compatible license.
>>
>> However, if you think about larger scale servers, which do exist
>using
>> lws, "knowing the exact (single) wsi" that woke it is not useful when
>there
>> may be hundreds of fds needing service each poll().
>>
>>      Either way lws_change_pollfd() is central to the solution.
>>>
>>> With my solution or even ppoll one, I don't see when I need to
>>> calllws_change_pollfd() especially as lws_change_pollfd needs a
>pointer
>>> to wsi which I cannot give as my interrupt concerns the complete
>context
>>> and not a special wsi...?I must miss a point.
>>>
>>
>> lws_change_pollfd() is the point that any code which wants to change
>the
>> events on a pollfd ends up at now.  And changing the event on a
>pollfd is
>> the definition of the cause of latency (when poll() is idle and with
>> relatively long timeout).
>>
>> So whether it is doing rx flow control or wait on being able to send,
>that
>> function is the place to signal to break the poll() one way or the
>other.
>>
>>      If you pick a signal like SIGUSR1 and install a do-nothing
>handler
>>>     for it, firing SIGUSR1 at the process from itself in
>>>     lws_change_pollfd() and using ppoll() could be a really small
>and
>>>     robust solution.
>>>     Since the signal is handled it doesn't do anything except
>interrupt
>>>     the ppoll causing a pollfd reload.
>>>     You only need to fire the signal the first time anything wants
>to
>>>     interrupt the wait *from another thread* (because if the lws
>thread
>>>     is in poll(), it isn't doing anything else).  If a pollfd raced
>it
>>>     and changed first, there's no problem with an additional signal
>>>     interrupting the next ppoll loop.
>>>
>>> Ideally, if it is not too much to ask, a simple example of code
>would be
>>> ideal.
>>>
>>
>> I may have some time tomorrow to give this a try.
>>
>> -Andy
>>
>>  BR, Thomas
>>>
>>>
>>>
>>> 2013/12/25 "Andy Green (林安廸)" <andy at warmcat.com
>>> <mailto:andy at warmcat.com>>
>>>
>>>     On 25/12/13 02:06, the mail apparently from Thomas Spitz
>included:
>>>
>>>         Dear Andy,
>>>
>>>         In accordance to your advice, I have created a patch to
>signal
>>>         lws to
>>>         break out and reload the poll() when an external thread
>needs to
>>>         interact with lws. This is done by sending a filedescriptor
>to lws
>>>         on libwebsocket_create_context().
>>>
>>>         I am novice with lws so please let me know whether it is
>correct.
>>>
>>>
>>>     Thanks... it's not incorrect, but I'm wondering if this is the
>best
>>> way.
>>>
>>>     The choices seem to boil down to this kind of "add a fake
>>>     descriptor" thing (although everything, including the "interrupt
>the
>>>     poll" descriptor and the use of it should be defined inside the
>>>     library), or maybe change to use ppoll() and fire signals at it.
>>>
>>>     Either way lws_change_pollfd() is central to the solution.
>>>
>>>     ppoll() is the same as poll() but it lets you basically choose
>>>     signals to wait on inside the poll loop as well as the fds.
>>>
>>>     If you pick a signal like SIGUSR1 and install a do-nothing
>handler
>>>     for it, firing SIGUSR1 at the process from itself in
>>>     lws_change_pollfd() and using ppoll() could be a really small
>and
>>>     robust solution.
>>>
>>>     Since the signal is handled it doesn't do anything except
>interrupt
>>>     the ppoll causing a pollfd reload.
>>>
>>>     You only need to fire the signal the first time anything wants
>to
>>>     interrupt the wait *from another thread* (because if the lws
>thread
>>>     is in poll(), it isn't doing anything else).  If a pollfd raced
>it
>>>     and changed first, there's no problem with an additional signal
>>>     interrupting the next ppoll loop.
>>>
>>>     -Andy
>>>
>>>         Have a nice christmas,
>>>
>>>         Best regards,
>>>         Thomas
>>>
>>>
>>>         2013/12/21 "Andy Green (林安廸)" <andy at warmcat.com
>>>         <mailto:andy at warmcat.com>
>>>         <mailto:andy at warmcat.com <mailto:andy at warmcat.com>>>
>>>
>>>
>>>              On 21/12/13 10:49, the mail apparently from "Andy Green
>(林
>>>         安廸)"
>>>              included:
>>>
>>>                  On 20/12/13 19:38, the mail apparently from Thomas
>>>         Spitz included:
>>>
>>>                      Hello Andy,
>>>
>>>                      In fact, in my application, I mostly often only
>>>         have 0 to 5
>>>                      active
>>>                      connections at the same time with a maximum
>number of
>>>                      simultaneous
>>>                      connections restricted to 32 (Besides, why is
>there
>>> no
>>>                      simple solution
>>>                      to reduce the maximum simultaneous connection?
>I
>>>         have seen
>>>                      that we need
>>>                      to reduce the max system fds...).
>>>
>>>
>>>                  You should be able to do
>>>
>>>                  ulimit -n 64
>>>
>>>                  before starting the server.  It only affects that
>shell
>>> and
>>>                  children. If
>>>                  you're bothered by the memory footprint of the
>parent
>>>         shell, it
>>>                  should
>>>                  be workable to do an exec instead of running as a
>child.
>>>
>>>                  The default is 1024 on most systems... the way we
>deal
>>>         with fd
>>>                  lookups
>>>                  it's just not that expensive to allow 1024 (a few
>KB).
>>>
>>>                      Therefore, I think I should implement your
>following
>>>                      proposal using
>>>                      External_Poll I guess:
>>>
>>>                           Poll can only wait on file or socket
>>>         descriptors... you
>>>                      could
>>>                           reserve a socket to signal to break out
>and
>>>         reload the
>>>                      poll()
>>>                           because somebody's attribute changed.
>>>
>>>
>>>                  Actually I was imagining adding it in lws.
>>>
>>>
>>>              I just pushed a patch
>>>
>>>         http://git.libwebsockets.org/____cgi-bin/cgit/libwebsockets/
>>> ____commit/?id=____91f19d8d799fdce0459c80357b3993____0b3ac55a9c
>>>         <http://git.libwebsockets.org/__cgi-bin/cgit/libwebsockets/_
>>> _commit/?id=__91f19d8d799fdce0459c80357b3993__0b3ac55a9c>
>>>
>>>
>>>         <http://git.libwebsockets.org/__cgi-bin/cgit/libwebsockets/_
>>> _commit/?id=__91f19d8d799fdce0459c80357b3993__0b3ac55a9c
>>>         <http://git.libwebsockets.org/cgi-bin/cgit/libwebsockets/
>>> commit/?id=91f19d8d799fdce0459c80357b39930b3ac55a9c>>
>>>
>>>              which should give you a single place to signal all
>pollfd
>>>         changes
>>>              from if you want to have a go at a patch to implement
>this
>>>         on lws side.
>>>
>>>              If you make a new socket or other descriptor-based
>thing in
>>> the
>>>              context create code that is open for the lifetime of
>the
>>>         client or
>>>              server, and add it to the pollfs there with POLLIN
>event set,
>>>              writing a byte to it from lws_change_pollfd() the first
>time
>>>              something changed before the outer loop reloads should
>do
>>>         what you want.
>>>
>>>              -Andy
>>>
>>>                      I tried to do it but my thread is still not
>able to
>>>         break
>>>                      out the
>>>                      poll(). In my test below (using test-server.c
>as
>>>         skeletton),
>>>                        Although I
>>>
>>>
>>>                  I think it must mean your test is broken. 
>Otherwise
>>> you're
>>>                  saying that
>>>                  poll() doesn't work.
>>>
>>>                  -Andy
>>>
>>>                      write in the tube that is monitored by poll I
>see
>>>         that my
>>>                      asyncSending
>>>                      is never able to break up poll(). poll() always
>>>         break up on
>>>                      10s timeout
>>>                      whereas my thread write into the tube every
>second.
>>>
>>>                      int tube[2];
>>>
>>>                      (...)
>>>
>>>                      void *asyncSending(void *arg) {
>>>
>>>                      unsigned char c=0;
>>>                      printf("START asynchronous sending\n");
>>>
>>>                      while(1){
>>>                      c++;
>>>                      printf("write in tube\n");
>>>                      if(write(tube[1],&c,1)!=1){
>>>                      printf("Erreur write %s\n", strerror(errno));
>>>                      }
>>>
>>>         //libwebsocket_callback_on_____writable_all_protocol(&____
>>> protocols[PROTOCOL_DUMB_____INCREMENT]);
>>>
>>>
>>>                      }
>>>
>>>                      printf("END asynchronous sending\n");
>>>                      return NULL ;
>>>
>>>                      }
>>>
>>>
>>>                         int main(int argc, char **argv) {
>>>
>>>                                (...)
>>>
>>>                      n = 0;
>>>
>>>                      printf("Tube creation\n");
>>>                      if(pipe(tube)!=0){
>>>                      printf("Pipe creation error %s\n",
>strerror(errno));
>>>                      }
>>>
>>>                      if (count_pollfds >= max_poll_elements) {
>>>                      lwsl_err("LWS_CALLBACK_ADD_____POLL_FD: too
>many
>>>         sockets to
>>>
>>>                      track\n");
>>>                      return 1;
>>>                      }
>>>
>>>                      fd_lookup[tube[0]] = count_pollfds;
>>>                      pollfds[count_pollfds].fd = fd_lookup[tube[0]];
>>>                      pollfds[count_pollfds].events = POLLIN |
>POLLPRI
>>>         |POLL_OUT;
>>>                      pollfds[count_pollfds++].____revents = 0;
>>>
>>>
>>>                      int statut_Thread;
>>>                      pthread_t Thread;
>>>
>>>                      statut_Thread = pthread_create(&Thread, NULL,
>>>         asyncSending,
>>>                      context);
>>>                      if (statut_Thread != 0) {
>>>                      printf("Thread creation errror \n");
>>>                      }
>>>
>>>                      while (n >= 0 && !force_exit) {
>>>
>>>                      /*
>>>                      * this represents an existing server's single
>poll
>>>         action
>>>                      * which also includes libwebsocket sockets
>>>                      */
>>>
>>>                      n = poll(pollfds, count_pollfds, 10000);//10s
>>>                      if (n < 0)
>>>                      continue;
>>>
>>>                      if (n)
>>>                      for (n = 0; n < count_pollfds; n++)
>>>                      if (pollfds[n].revents)
>>>                      /*
>>>                      * returns immediately if the fd does not
>>>                      * match anything under libwebsockets
>>>                      * control
>>>                      */
>>>                      if (libwebsocket_service_fd(____context,
>>>
>>>                      &pollfds[n]) < 0)
>>>                      goto done;
>>>
>>>                                       (...)
>>>                               }
>>>                      }
>>>
>>>                      Thanks again in advance for your help.
>>>
>>>                      BR, Thomas
>>>
>>>
>>>                      2013/12/19 "Andy Green (林安廸)" <andy at warmcat.com
>>>         <mailto:andy at warmcat.com>
>>>                      <mailto:andy at warmcat.com
><mailto:andy at warmcat.com>>
>>>                      <mailto:andy at warmcat.com
><mailto:andy at warmcat.com>
>>>         <mailto:andy at warmcat.com <mailto:andy at warmcat.com>>>>
>>>
>>>                           On 19/12/13 02:12, the mail apparently
>from
>>> Thomas
>>>                      Spitz included:
>>>
>>>                               Hello Andy, hello Eugene,
>>>
>>>                               If I understood it correctly, thanks
>to
>>>         your patch,
>>>                      we can now
>>>                               prevent
>>>                               our external thread(s) from calling
>>>                              
>libwebsocket_callback_on_______writable
>>>
>>>                               while lws is actively dealing with
>POLLIN
>>> and
>>>                      POLLOUT of fds
>>>                               using our
>>>                               own mutex? Thus we are sure that the
>>>         request(s)
>>>                      from our external
>>>                               thread(s) will not be lost, or at
>least
>>>         trigger
>>>                               a LWS_CALLBACK_SERVER_WRITEABLE?
>>>
>>>
>>>                           I'm not sure it won't get lost, but it
>won't
>>> leave
>>>                      things in an
>>>                           indeterminate state.  It can still happen
>the
>>>                      non-service thread
>>>                           asks to set POLLOUT just as we're about to
>>>         take the
>>>                      mutex and
>>>                      clear it.
>>>
>>>                           At the moment that'll still break at low
>>>         probability
>>>                      but it should
>>>                           be solveable.
>>>
>>>
>>>                               My second question is related to
>>> performance:
>>>
>>>                               In the documentation it is said
>>>
>>>                                    If you want to send something, do
>not
>>>         just
>>>                      send it but
>>>                               request a
>>>                                    callback |when the socket is
>writeable
>>> |
>>>
>>>                               That is why our external thread call
>>>                              
>libwebsocket_callback_on_______writable
>>>
>>>                               (). Then it is written
>>>
>>>                                    |Usually you will get called back
>>>         immediately
>>>                      next time
>>>                               around the
>>>                                    service loop|
>>>
>>>                               which seems to mean that  (according
>to my
>>>         tests)
>>>                      we have to
>>>                               wait libwebsocket_service() to end up
>>>         before the
>>>                      socket is
>>>                      writeable
>>>                               again. That implies to set a very
>short
>>>         timeout
>>>
>>>
>>>                           Well... you are right if we have one
>>>         connection.  But
>>>                      if you have 50
>>>                           or 50,000 connections, somebody almost
>always
>>> has
>>>                      something going on
>>>                           before the timeout.
>>>
>>>                           Also in your user callback, you can take
>the
>>>         approach
>>>                      like in the
>>>                           test apps to loop using
>lws_send_pipe_choked()
>>>         test to
>>>                      absolutely
>>>                           stuff your pipe each time you get service.
> That
>>>                      doesn't delay
>>>                           anything looping there because it only
>loops
>>>         while the
>>>                      send won't
>>>                           block.  And it means the kernel is passing
>out
>>>         what you
>>>                      stuffed into
>>>                           the pipe while you wait through any
>latency in
>>>         the next
>>>                      "writeable"
>>>                           service.
>>>
>>>
>>>                               for libwebsocket_service(context,
>>>         VERYSHORTTIMEOUT)
>>>                      otherwise it
>>>                               will
>>>                               take a long time before the external
>>>         thread request
>>>                      could be
>>>                               handled...
>>>                               This brings me to wonder why it is not
>>>         possible to
>>>                      loop on
>>>                               libwebsocket_service(context,
>>>         VERYLONGTIMEOUT) that
>>>                      would be
>>>                               interrupted
>>>                               by our external thread instead? Thus
>lws
>>>         thread is
>>>                      asleeped most
>>>                               of the
>>>                               time when there is no incoming IP
>traffic
>>>         but can
>>>                      be awaken as
>>>                               soon as
>>>                               some outgoing traffic needs to be sent
>>>         (generated
>>>                      from the
>>>                               external thread).
>>>
>>>
>>>                           Well lws is a singlethreaded nonblocking
>>>                      implementation.  It is
>>>                           designed to run on small
>resource-costrained
>>>         platforms
>>>                      at the same
>>>                           time a being able to service huge numbers
>of
>>>                      connections in a
>>>                           lightweight way.  It does not require
>threads.
>>>
>>>                           Poll can only wait on file or socket
>>>         descriptors... you
>>>                      could
>>>                           reserve a socket to signal to break out
>and
>>>         reload the
>>>                      poll()
>>>                           because somebody's attribute changed.  But
>in
>>>         most real
>>>                      usage
>>>                           scenarios, some other connection is always
>>> wanting
>>>                      service before
>>>                           the timeout anyway and as more connections
>are
>>>         seen the
>>>                      less it
>>>                           matters about latency from waiting for
>poll()
>>>         reload.
>>>
>>>                           -Andy
>>>
>>>                               I hope my two questions are clear..
>>>
>>>                               Thanks in advance for your rely.
>>>
>>>                               Best regards, Thomas
>>>
>>>                               2013/12/18 "Andy Green (林安廸)"
>>>         <andy at warmcat.com <mailto:andy at warmcat.com>
>>>                      <mailto:andy at warmcat.com
><mailto:andy at warmcat.com>>
>>>                               <mailto:andy at warmcat.com
>>>         <mailto:andy at warmcat.com> <mailto:andy at warmcat.com
>>>         <mailto:andy at warmcat.com>>>
>>>                               <mailto:andy at warmcat.com
>>>         <mailto:andy at warmcat.com> <mailto:andy at warmcat.com
>>>         <mailto:andy at warmcat.com>>
>>>                      <mailto:andy at warmcat.com
><mailto:andy at warmcat.com>
>>>         <mailto:andy at warmcat.com <mailto:andy at warmcat.com>>>>>
>>>
>>>
>>>                                    On 18/12/13 06:40, the mail
>>>         apparently from
>>>                      Andy Green
>>>                               included:
>>>
>>>
>>>
>>>                                        Eugene Agafonov
>>>         <e.a.agafonov at gmail.com <mailto:e.a.agafonov at gmail.com>
>>>                      <mailto:e.a.agafonov at gmail.com
>>>         <mailto:e.a.agafonov at gmail.com>__>
>>>                               <mailto:e.a.agafonov at gmail.com
>>>         <mailto:e.a.agafonov at gmail.com>
>>>                      <mailto:e.a.agafonov at gmail.com
>>>         <mailto:e.a.agafonov at gmail.com>__>__>
>>>                                       
><mailto:e.a.agafonov at gmail.com
>>>         <mailto:e.a.agafonov at gmail.com>
>>>                      <mailto:e.a.agafonov at gmail.com
>>>         <mailto:e.a.agafonov at gmail.com>__>
>>>                               <mailto:e.a.agafonov at gmail.com
>>>         <mailto:e.a.agafonov at gmail.com>
>>>                      <mailto:e.a.agafonov at gmail.com
>>>         <mailto:e.a.agafonov at gmail.com>__>__>__>> wrote:
>>>
>>>                                            Hi!
>>>
>>>                                            I've just implemented the
>>> similar
>>>                      scenario just
>>>                               like it is
>>>                                            stated
>>>                                            in document
>>>
>>>         http://git.libwebsockets.org/________cgi-bin/cgit/__
>>> libwebsockets/______tree/__README.coding
>>>         <http://git.libwebsockets.org/______cgi-bin/cgit/
>>> libwebsockets/______tree/README.coding>
>>>
>>>         <http://git.libwebsockets.org/______cgi-bin/cgit/__
>>> libwebsockets/____tree/README.__coding
>>>         <http://git.libwebsockets.org/____cgi-bin/cgit/
>>> libwebsockets/____tree/README.coding>>
>>>
>>>
>>>
>>>
>>>         <http://git.libwebsockets.org/______cgi-bin/cgit/__
>>> libwebsockets/____tree/README.__coding
>>>         <http://git.libwebsockets.org/____cgi-bin/cgit/
>>> libwebsockets/____tree/README.coding>
>>>
>>>         <http://git.libwebsockets.org/____cgi-bin/cgit/
>>> libwebsockets/____tree/README.coding
>>>         <http://git.libwebsockets.org/__cgi-bin/cgit/libwebsockets/_
>>> _tree/README.coding>>>
>>>
>>>
>>>
>>>
>>>
>>>         <http://git.libwebsockets.org/______cgi-bin/cgit/__
>>> libwebsockets/____tree/README.__coding
>>>         <http://git.libwebsockets.org/____cgi-bin/cgit/
>>> libwebsockets/____tree/README.coding>
>>>
>>>         <http://git.libwebsockets.org/____cgi-bin/cgit/
>>> libwebsockets/____tree/README.coding
>>>         <http://git.libwebsockets.org/__cgi-bin/cgit/libwebsockets/_
>>> _tree/README.coding>>
>>>
>>>
>>>
>>>         <http://git.libwebsockets.org/____cgi-bin/cgit/
>>> libwebsockets/____tree/README.coding
>>>         <http://git.libwebsockets.org/__cgi-bin/cgit/libwebsockets/_
>>> _tree/README.coding>
>>>
>>>         <http://git.libwebsockets.org/__cgi-bin/cgit/libwebsockets/_
>>> _tree/README.coding
>>>         <http://git.libwebsockets.org/cgi-bin/cgit/libwebsockets/
>>> tree/README.coding>>>>
>>>
>>>
>>>
>>>
>>>                                    I have a mutex-protected queue of
>>>         outgoing
>>>                      events (as a
>>>                               std::list of
>>>
>>>                                            strings). Each string is
>>>         intended to
>>>                      be send in
>>>                               separated
>>>                                            message.
>>>
>>>                                            The writer thread posts
>data
>>> onto
>>>                      queue and calls
>>>
>>>
>>>         libwebsocket_callback_on_________writable(context, wsi)
>>>
>>>
>>>
>>>                                            Protocol callback gets
>>>                               LWS_CALLBACK_SERVER_WRITEABLE as soon
>as
>>>                                            websocket thread is ready
>to
>>> send
>>>                      data. Once it
>>>                               comes, the
>>>                                            callback
>>>                                            extracts strings from the
>>>         queue one by
>>>                      one and
>>>                               writes them to
>>>                                            socket using
>>> libwebsocket_write.
>>>
>>>                                            I'm quite newby in
>writing WS
>>>         server with
>>>                               libwebsockets. Any
>>>                                            comments from WS gurus
>are
>>>         welcome
>>>
>>>
>>>                                        What you're doing is a good
>way to
>>>                      interact with what
>>>                               lws needs and
>>>                                        get what you want from
>threading.
>>>
>>>                                        There's one danger with it,
>but
>>>         that is a
>>>                      very tightly
>>>                               defined race
>>>                                        rather than corruption or
>whatever.
>>>
>>>
>>>                                    Well... there's another thing to
>take
>>>         care
>>>                      about... you
>>>                      need to
>>>                                    manage your private list of live
>wsis
>>>                      carefully.  It means
>>>                               managing
>>>                                    that list (with any of your
>locking
>>>         needed) at the
>>>                               ESTABLISHED /
>>>                                    CLOSED callback for the WSI. 
>That
>>>         should be
>>>                      enough I think.
>>>
>>>
>>>                                        The issue is that the 'wait
>for
>>>         writeable'
>>>                      is oneshot,
>>>                               so after it
>>>                                        becomes true in the poll lws
>will
>>>         clear
>>>                      the POLLOUT
>>>                               wait flag.  But
>>>                                        because the threads are
>>> asynchronous,
>>>                      eventually the
>>>                               other thread
>>>                                        will come at the wrong time
>and
>>>         set it
>>>                      just as we're
>>>                               clearing it
>>>                                        with
>>>                                        indeterminate results. 
>Because
>>> it's
>>>                      read-modify-write
>>>                               and flow
>>>                                        control code also modifies
>this,
>>>         there's a low
>>>                               probability deadlock
>>>                                        case where POLLIN or POLLOUT
>from
>>> one
>>>                      thread or another
>>>                               did not get
>>>                                        set for wait as the thread
>thinks
>>>         it is.
>>>
>>>                                        If the extpoll stuff is used
>the
>>>         issue can
>>>                      be worse
>>>                               depending on
>>>                                        what
>>>                                        it is doing.
>>>
>>>                                        However the approach you're
>taking
>>> is
>>>                      otherwise really
>>>                               workable I
>>>                                        think.  To make it rock solid
>>>         you'll need
>>>                      to patch
>>>                      lws to
>>>                                        mutex-protect the few places
>it
>>>         changes
>>>                      poll wait flags.
>>>
>>>
>>>                                    I added a patch
>>>
>>>
>>>         http://git.libwebsockets.org/________cgi-bin/cgit/__
>>>
>libwebsockets/______commit/?__id=________7a1327977ac10bdbace0012274b8ae
>>> ________889219880e
>>>         <http://git.libwebsockets.org/______cgi-bin/cgit/
>>> libwebsockets/______commit/?id=______7a1327977ac10bdbace0012274b8ae
>>> ______889219880e>
>>>
>>>         <http://git.libwebsockets.org/______cgi-bin/cgit/__
>>> libwebsockets/____commit/?id=________7a1327977ac10bdbace0012274b8ae
>>> ______889219880e
>>>         <http://git.libwebsockets.org/____cgi-bin/cgit/
>>> libwebsockets/____commit/?id=____7a1327977ac10bdbace0012274b8ae
>>> ____889219880e>>
>>>
>>>
>>>
>>>
>>>         <http://git.libwebsockets.org/______cgi-bin/cgit/__
>>> libwebsockets/____commit/?id=________7a1327977ac10bdbace0012274b8ae
>>> ______889219880e
>>>         <http://git.libwebsockets.org/____cgi-bin/cgit/
>>> libwebsockets/____commit/?id=____7a1327977ac10bdbace0012274b8ae
>>> ____889219880e>
>>>
>>>         <http://git.libwebsockets.org/____cgi-bin/cgit/
>>> libwebsockets/____commit/?id=____7a1327977ac10bdbace0012274b8ae
>>> ____889219880e
>>>         <http://git.libwebsockets.org/__cgi-bin/cgit/libwebsockets/_
>>> _commit/?id=__7a1327977ac10bdbace0012274b8ae__889219880e>>>
>>>
>>>
>>>
>>>
>>>
>>>         <http://git.libwebsockets.org/______cgi-bin/cgit/__
>>> libwebsockets/____commit/?id=________7a1327977ac10bdbace0012274b8ae
>>> ______889219880e
>>>         <http://git.libwebsockets.org/____cgi-bin/cgit/
>>> libwebsockets/____commit/?id=____7a1327977ac10bdbace0012274b8ae
>>> ____889219880e>
>>>
>>>         <http://git.libwebsockets.org/____cgi-bin/cgit/
>>> libwebsockets/____commit/?id=____7a1327977ac10bdbace0012274b8ae
>>> ____889219880e
>>>         <http://git.libwebsockets.org/__cgi-bin/cgit/libwebsockets/_
>>> _commit/?id=__7a1327977ac10bdbace0012274b8ae__889219880e>>
>>>
>>>
>>>
>>>         <http://git.libwebsockets.org/____cgi-bin/cgit/
>>> libwebsockets/____commit/?id=____7a1327977ac10bdbace0012274b8ae
>>> ____889219880e
>>>         <http://git.libwebsockets.org/__cgi-bin/cgit/libwebsockets/_
>>> _commit/?id=__7a1327977ac10bdbace0012274b8ae__889219880e>
>>>
>>>         <http://git.libwebsockets.org/__cgi-bin/cgit/libwebsockets/_
>>> _commit/?id=__7a1327977ac10bdbace0012274b8ae__889219880e
>>>         <http://git.libwebsockets.org/cgi-bin/cgit/libwebsockets/
>>> commit/?id=7a1327977ac10bdbace0012274b8ae889219880e>>>>
>>>
>>>
>>>                                    that lets you do this in a lock
>and
>>>         unlock
>>>                      callback in
>>>                      the user
>>>                                    code, in one place.
>>>
>>>                                    If you're not using threading you
>>>         don't need
>>>                      to do anything
>>>                               new.
>>>
>>>                                    -Andy
>>>
>>>
>>>                                        -Andy
>>>
>>>                                            BR, Eugene Agafonov.
>>>
>>>                                            On Tuesday 17 December
>2013
>>>         12:03:43
>>>                      Thomas Spitz
>>>                               wrote:
>>>
>>>                                                Hello everyone,
>>>
>>>                                                I am trying to find a
>way
>>> for
>>>                      external threads
>>>                               to ask
>>>                                                libwebsocket
>>>
>>>                                            server
>>>
>>>                                                to send websocket
>message
>>>         on their
>>>                      behalf (as
>>>                      they
>>>                                                cannot do it
>>>
>>>                                            directly).
>>>
>>>                                                The only way I see at
>the
>>>         present
>>>                      is to make a
>>>                               loop with
>>>                                                libwebsocket_service
>>>         (context,
>>>                      VERYSHORTTIME);
>>>                               and test
>>>                                                potential
>>>
>>>                                            incoming
>>>
>>>                                                messages from other
>>>         threads. This
>>>                      is not very
>>>                               elegant as CPU
>>>                                                computes
>>>
>>>                                            all
>>>
>>>                                                the time and not very
>>>         reliable as
>>>                      I should put
>>>                                                VERYSHORTTIME at 0
>>>                                                if
>>>
>>>                                            I
>>>
>>>                                                don't want to miss
>any
>>>         message.
>>>
>>>                                                I tried the following
>>>         modification in
>>>                               test-server.c :
>>>
>>>                                                void
>>>         *connexionServeur(void *arg) {
>>>
>>>                                                    struct
>>>         libwebsocket_context
>>>                      *context = arg;
>>>                               while (1) {
>>>
>>>                        if(libwebsocket_service(________context,
>>>
>>>                      60000)<0){
>>>
>>>
>>>                                                   
>printf("Problem!\n");
>>>
>>>                                                    break; } }
>>>
>>>                                                    return NULL ;
>>>
>>>                                                    } int
>statut_Thread;
>>>         pthread_t
>>>                      Thread;
>>>                               statut_Thread =
>>>
>>>         pthread_create(&Thread, NULL,
>>>                      connexionServeur,
>>>
>>>                                            context
>>>
>>>                                                    );
>>>
>>>                                                    if (statut_Thread
>!=
>>> 0) {
>>>
>>>                                                    printf("Problem
>with
>>>         thread
>>>                      creation \n");
>>>                               } while
>>>                                                    (n >= 0 &&
>>>                                                    !force_exit) {
>struct
>>>         timeval tv;
>>>                               gettimeofday(&tv,
>>>                                                    NULL); if
>>>                                                    (((unsigned
>>>         int)tv.tv_usec -
>>>                      oldus) >
>>>                      50000) {
>>>
>>>
>>>
>>>
>>>
>>>         libwebsocket_callback_on_________writable_all_protocol(&____
>>> ____protocols[PROTOCOL_DUMB_I
>>>
>>>
>>>
>>>
>>>
>>>
>>>                                    NCREMENT]); oldus = tv.tv_usec;
>>>
>>>
>>>         libwebsocket_service(context, 0);
>>>
>>>                                                    }
>>>
>>>                                                    usleep(10000); }
>>>
>>>
>>>                                                but it crashes
>quickly as
>>>         soon as
>>>                      I try
>>>                      drawing . I
>>>                                                suppose that
>>>                                                libwebsocket_service
>is
>>>         called
>>>                      twice at the
>>>                               same time and it
>>>                                                creates conflicts
>that
>>>         lead to
>>>                      stop crash the
>>>                               websocket.
>>>
>>>                                                I have seen in the
>>>         mailing list
>>>                      that Thomas
>>>                               Koeper has
>>>                                                already
>>>                                                had a similar
>question
>>>
>>>
>>>         http://ml.libwebsockets.org/________pipermail/libwebsockets/
>>> ____2013-____April/000403.html
>>>         <http://ml.libwebsockets.org/______pipermail/libwebsockets/_
>>> _2013-____April/000403.html>
>>>
>>>         <http://ml.libwebsockets.org/______pipermail/libwebsockets/_
>>> _2013-____April/000403.html
>>>         <http://ml.libwebsockets.org/____pipermail/libwebsockets/
>>> 2013-____April/000403.html>>
>>>
>>>
>>>
>>>
>>>         <http://ml.libwebsockets.org/______pipermail/libwebsockets/_
>>> _2013-____April/000403.html
>>>         <http://ml.libwebsockets.org/____pipermail/libwebsockets/
>>> 2013-____April/000403.html>
>>>
>>>         <http://ml.libwebsockets.org/____pipermail/libwebsockets/
>>> 2013-____April/000403.html
>>>         <http://ml.libwebsockets.org/__pipermail/libwebsockets/2013-
>>> __April/000403.html>>>
>>>
>>>
>>>
>>>
>>>
>>>         <http://ml.libwebsockets.org/______pipermail/libwebsockets/_
>>> _2013-____April/000403.html
>>>         <http://ml.libwebsockets.org/____pipermail/libwebsockets/
>>> 2013-____April/000403.html>
>>>
>>>         <http://ml.libwebsockets.org/____pipermail/libwebsockets/
>>> 2013-____April/000403.html
>>>         <http://ml.libwebsockets.org/__pipermail/libwebsockets/2013-
>>> __April/000403.html>>
>>>
>>>
>>>
>>>         <http://ml.libwebsockets.org/____pipermail/libwebsockets/
>>> 2013-____April/000403.html
>>>         <http://ml.libwebsockets.org/__pipermail/libwebsockets/2013-
>>> __April/000403.html>
>>>
>>>         <http://ml.libwebsockets.org/__pipermail/libwebsockets/2013-
>>> __April/000403.html
>>>         <http://ml.libwebsockets.org/pipermail/libwebsockets/2013-
>>> April/000403.html>>>>
>>>
>>>
>>>
>>>
>>>                                    . At that time, he was thinking
>of
>>>         creating a
>>>                      mutex on
>>>
>>>                                                libwebsocket_service.
>Do
>>>         you think
>>>                      this is the
>>>                               right
>>>                                                solution?
>>>
>>>                                                Thanks in advance for
>>>         your help
>>>                      Best regards,
>>>                               Thomas
>>>
>>>
>>>
>>>         _______________________________________________________
>>>                               Libwebsockets
>>>                                            mailing list
>>>                      Libwebsockets at ml.__libwebsocke______ts.org
>>>         <http://libwebsocke____ts.org>
>>>                      <http://libwebsocke__ts.org>
>>>                               <http://libwebsockets.org>
>>>                                            <mailto:Libwebsockets at ml
>>>         <mailto:Libwebsockets at ml>.
>>>                      <mailto:Libwebsockets at ml
>>>         <mailto:Libwebsockets at ml>.>__li__b__websockets.org
>>>         <http://lib__websockets.org>
>>>
>>>                      <http://libwebsockets.org>
>>>                               <mailto:Libwebsockets at ml.
>>>         <mailto:Libwebsockets at ml.>__lib__websockets.org
>>>         <http://libwebsockets.org>
>>>                      <mailto:Libwebsockets at ml.__libwebsockets.org
>>>         <mailto:Libwebsockets at ml.libwebsockets.org>>>>
>>>         http://ml.libwebsockets.org/________mailman/listinfo/____
>>> libwebsockets
>>>         <http://ml.libwebsockets.org/______mailman/listinfo/__
>>> libwebsockets>
>>>
>>>         <http://ml.libwebsockets.org/______mailman/listinfo/__
>>> libwebsockets
>>>        
><http://ml.libwebsockets.org/____mailman/listinfo/libwebsockets>>
>>>
>>>
>>>         <http://ml.libwebsockets.org/______mailman/listinfo/____
>>> libwebsockets
>>>        
><http://ml.libwebsockets.org/____mailman/listinfo/__libwebsockets
>>> >
>>>
>>>        
><http://ml.libwebsockets.org/____mailman/listinfo/__libwebsockets<
>>> http://ml.libwebsockets.org/__mailman/listinfo/libwebsockets>>>
>>>
>>>
>>>
>>>         <http://ml.libwebsockets.org/______mailman/listinfo/____
>>> libwebsockets
>>>        
><http://ml.libwebsockets.org/____mailman/listinfo/__libwebsockets
>>> >
>>>
>>>
>>>        
><http://ml.libwebsockets.org/____mailman/listinfo/__libwebsockets<
>>> http://ml.libwebsockets.org/__mailman/listinfo/libwebsockets>>
>>>
>>>
>>>        
><http://ml.libwebsockets.org/____mailman/listinfo/__libwebsockets<
>>> http://ml.libwebsockets.org/__mailman/listinfo/libwebsockets>
>>>        
><http://ml.libwebsockets.org/__mailman/listinfo/libwebsockets
>>>         <http://ml.libwebsockets.org/mailman/listinfo/libwebsockets
>>> >__>__>__>
>>>
>>>
>>>
>>>
>>>         _______________________________________________________
>>>                               Libwebsockets
>>>                                        mailing
>>>                                        list
>>>                      Libwebsockets at ml.__libwebsocke______ts.org
>>>         <http://libwebsocke____ts.org>
>>>                      <http://libwebsocke__ts.org>
>>>                               <http://libwebsockets.org>
>>>                                        <mailto:Libwebsockets at ml
>>>         <mailto:Libwebsockets at ml>.
>>>                      <mailto:Libwebsockets at ml
>>>         <mailto:Libwebsockets at ml>.>__li__b__websockets.org
>>>         <http://lib__websockets.org>
>>>
>>>                      <http://libwebsockets.org>
>>>                               <mailto:Libwebsockets at ml.
>>>         <mailto:Libwebsockets at ml.>__lib__websockets.org
>>>         <http://libwebsockets.org>
>>>                      <mailto:Libwebsockets at ml.__libwebsockets.org
>>>         <mailto:Libwebsockets at ml.libwebsockets.org>>>>
>>>         http://ml.libwebsockets.org/________mailman/listinfo/____
>>> libwebsockets
>>>         <http://ml.libwebsockets.org/______mailman/listinfo/__
>>> libwebsockets>
>>>
>>>         <http://ml.libwebsockets.org/______mailman/listinfo/__
>>> libwebsockets
>>>        
><http://ml.libwebsockets.org/____mailman/listinfo/libwebsockets>>
>>>
>>>
>>>         <http://ml.libwebsockets.org/______mailman/listinfo/____
>>> libwebsockets
>>>        
><http://ml.libwebsockets.org/____mailman/listinfo/__libwebsockets
>>> >
>>>
>>>        
><http://ml.libwebsockets.org/____mailman/listinfo/__libwebsockets<
>>> http://ml.libwebsockets.org/__mailman/listinfo/libwebsockets>>>
>>>
>>>
>>>
>>>         <http://ml.libwebsockets.org/______mailman/listinfo/____
>>> libwebsockets
>>>        
><http://ml.libwebsockets.org/____mailman/listinfo/__libwebsockets
>>> >
>>>
>>>
>>>        
><http://ml.libwebsockets.org/____mailman/listinfo/__libwebsockets<
>>> http://ml.libwebsockets.org/__mailman/listinfo/libwebsockets>>
>>>
>>>
>>>        
><http://ml.libwebsockets.org/____mailman/listinfo/__libwebsockets<
>>> http://ml.libwebsockets.org/__mailman/listinfo/libwebsockets>
>>>        
><http://ml.libwebsockets.org/__mailman/listinfo/libwebsockets
>>>         <http://ml.libwebsockets.org/mailman/listinfo/libwebsockets
>>> >__>__>__>
>>>
>>>
>>>
>>>
>>>         _______________________________________________________
>>>                                    Libwebsockets mailing list
>>>
>>>         Libwebsockets at ml.__libwebsocke______ts.org
>>>         <http://libwebsocke____ts.org>
>>>                      <http://libwebsocke__ts.org>
>>>                               <http://libwebsockets.org>
>>>                                    <mailto:Libwebsockets at ml
>>>         <mailto:Libwebsockets at ml>.
>>>                      <mailto:Libwebsockets at ml
>>>         <mailto:Libwebsockets at ml>.>__li__b__websockets.org
>>>         <http://lib__websockets.org>
>>>
>>>                      <http://libwebsockets.org>
>>>                               <mailto:Libwebsockets at ml.
>>>         <mailto:Libwebsockets at ml.>__lib__websockets.org
>>>         <http://libwebsockets.org>
>>>                      <mailto:Libwebsockets at ml.__libwebsockets.org
>>>
>>
>> ...




More information about the Libwebsockets mailing list