[Libwebsockets] Sai administrivia

Andy Green andy at warmcat.com
Wed Nov 25 21:11:37 CET 2020



On 11/6/20 8:49 AM, Andy Green wrote:
> 
> 
> On 11/2/20 4:50 PM, Andy Green wrote:
>> Hi -
>>
>> If you don't care about the issues and solutions maintaining 21 OSes 
>> for libwebsockets CI, you can just ignore this kind of email.

The Sai mac mini builder's updated to Big Sur, an lws user reported that 
aftertheir upgrade, ulimit -n was set to a huge number indicating 
"unlimited" for them, but here it's set to 2560.  There's a patch on lws 
to interpret any unreasonably large ulimit -n as 2560 just in case.


Sai had a big update that has been in progress for a few weeks splitting 
the server part into two, previously all traffic went through lwsws in 
proxy mode, both web traffic from browsers and control messages and 
build logs from builders.

Since there are many builds in parallel and the logs have a lot of 
traffic sent over wss in realtime, so they can be seen by the browsers 
as the build, this was handled fine by lwsws in terms of proxying it 
all, but in the sai server (formerly "sai-master") part its own event 
loop was struggling with the load and having the blocking sqlite 
commands for each log, at peak there are several hundred wss messages a 
second being proxied from the builders and corresponding sqlite 
transactions.  These delays don't affect lwsws, just the getting of the 
sai web rendering over the proxy link for /sai urls.

After the changes there are two separate sai processes that coordinate 
by linking up over a unix domain socket, "sai-server" on :4444 that the 
builders talk to and send their logs to, and "sai-web" which is on its 
own unix domain socket and proxied by lwsws as before, so the web render 
event loop is never blocked by log storage sqlite calls.

This required a lot of work separating out notifications from the 
builders and events from the browsers to go overthe uds link between the 
two.  But now the proxied web stuff to /sai is responsive as normal 
during builds, and various other improvements around build updates being 
done in the DOM layer rather than by rewriting chunks of generated html 
mean it's much smoother.

https://libwebsockets.org/sai/

The gitohashi integration has also been improved so the results are in a 
hidden (but live, over a wss link) div that fades in if you mouse over 
the small sai summary icon, which is moved so it doesn't take up any 
additional real-estate in gitohashi, now there are over 300 builds per 
push for libwebsockets sai it's too big to just show all the time.

https://libwebsockets.org/git/libwebsockets/tree

The Sai project docs are updated to reflect the changes, the overview 
diagram should be easier to see how it fits together now

https://warmcat.com/git/sai

-Andy


More information about the Libwebsockets mailing list