[Libwebsockets] lws v3.0 LWS_CALLBACK_EVENT_WAIT_CANCELLED

Andy Green andy at warmcat.com
Fri Nov 23 03:07:26 CET 2018



On 22/11/2018 21:15, thomas.spitz at hestia-france.com wrote:
> Hello everyone,
> 
> I have just updated my app from lws v2.2 to lws 3.0. I'm not using lwsws 
> but I use several vhosts to implement different API on my app.
> 
> When a client is connected to my app it is connected to the main vhost 
> permanently until it disconnects through callback_mainVhost.
> 
> When It disconnects It gets the following event from the lws library : 
> LWS_CALLBACK_CLOSED
> 
> My concerns is that now, with lws 3.0, as soon as a client is connected, 
> It triggers the LWS_CALLBACK_EVENT_WAIT_CANCELLED event almost every 1 
> to 8s.
> 
> I read that LWS_CALLBACK_EVENT_WAIT_CANCELLED is sent to every protocol 
> of every vhost in response to lws_cancel_service() or 
> lws_cancel_service_pt().

That is true... the underlying story about it is when lws creates a 
service pt ("per-thread") struct it also creates a pipe and registers 
the read side of it with the event loop / pollfds associated with the pt.

This is to deal with situations where the event loop / poll() thinks it 
is idle due to no pending network events, and sleeps the service thread, 
but for whatever reason we know something actually does need attention 
and the event loop / poll() sleep is just unwanted latency.

Lws also calls lws_cancel_service() internally in a few places...

  - at context creation because the vhost protocol init is done after 
going around the event loop... we know we will want to do that service 
action immediately, even when the network is idle.  So we short-circuit 
that initial wait, for any event loop, by calling lws_cancel_service() 
after context creation.

  - Threadpool calls it, when a thread has something to sync with the 
wsi... there's no related network activity to break the poll wait.  So 
the threadpool thread forces lws to wake and notice and service its sync 
request by lws_cancel_service().

  - If you call lws_callback_on_writable() from a different thread than 
the service thread, the changed pollfd will not be understood by the 
service thread already sleeping in poll() until it exits the current 
poll wait and loops around to start a new poll() wait.  So if lws 
detects it's a different thread asking for the writable callback (and so 
setting POLLOUT on the related pollfd) it forces the poll wait to end 
and start again with the latest pollfd info, using lws_cancel_service().

  - On BSD, it turned out its poll() does not work like linux poll(). 
When the poll() ends, and it updates the results into the pollfd array, 
it overwrites not only .revents in the pollfd as expected, but also 
.events, with the original .events that it saw when it entered the 
poll().  Any changes by other threads to any pollfds .events are 
effectively reverted when the service thread poll() ends.  (Linux only 
writes .revents).  To deal with this, lws detects the request is coming 
from a non-service thread, puts it on a queue, and signals 
lws_cancel_service().  If more requests come from other threads in the 
meanwhile in the window between asking for the service thread to exit 
its wait and it actually exiting, then they are added to the queue. 
When the service thread's wait ends, it turns the queued operations into 
pollfd changes strictly in the order they arrived before going back to 
the event loop / poll().

> As far as I know, when a WS client is connecting, it should not trigger 
> a lws_cancel_service() or lws_cancel_service_pt(). The only event it 
> triggers is LWS_CALLBACK_CLOSED when it closes the socket...
> 
> By the way, I didn't change anything between the move from lws 2.2 to 
> 3.0...
> 
> Is LWS_CALLBACK_EVENT_WAIT_CANCELLED a new event? What can trigger this 
> event (is it a misbehavior on the client side or a problem in the app code?

"Cancel" is a scary word but the only thing that was "canceled" is the 
sleep of the event loop.  There's no change to the state of anything 
else implied by it.  If it happens every few seconds, it's not costing 
you anything... you have to wake once a second for timeout / timer 
processing anyway.

-Andy

> Thanks in advance for your feedbacks
> 
> Best regards,
> 
> Thomas
> 
> -- 
> Hestia domotique & sécurité
> 	Nouveautés <https://blog.hestia-france.com> Application iOS pour Varuna 
> <https://itunes.apple.com/us/developer/hestia-france/id930809780> 
> Application Android pour Varuna 
> <https://play.google.com/store/apps/developer?id=Hestia%20France%20S.A.S&hl=fr> 
> 	thomas.spitz at hestia-france.com - Tel: +33(0)6 26 87 13 93
> Technique: +33(0)3 20 04 43 68
> Commandes: +33(0)3 20 96 47 32
> Annuaire de nos commerciaux 
> <https://hestia-france.com/fr/contactez-hestia.php#nos-commerciaux>
> 
> 1011, rue des Saules - Parc d'activités du Mélantois
> CS8458 - 59814 Lesquin Cedex - France
> blog Hestia <http://blog.hestia-france.com/> 
> facebook:hestia-france-domotique 
> <https://www.facebook.com/hestiadomotique> facebook:hestia.domotique 
> <https://plus.google.com/116828455794840525669/posts> 
> twitter:hestiaAutomatio <https://twitter.com/hestiadomotique> 
> skype:hestia-france <Skype:%20hestia-france>
> Hestia france S.A.S
> /Fabricant de systèmes domotiques et d'alarme pour les bâtiments 
> résidentiels et tertiaires
> Manufacturer of Building Management System including alarm security/
> 
> 
> 
> _______________________________________________
> Libwebsockets mailing list
> Libwebsockets at ml.libwebsockets.org
> https://libwebsockets.org/mailman/listinfo/libwebsockets
> 


More information about the Libwebsockets mailing list