[Libwebsockets] Sai administrivia
andy at warmcat.com
Wed Nov 25 21:11:37 CET 2020
On 11/6/20 8:49 AM, Andy Green wrote:
> On 11/2/20 4:50 PM, Andy Green wrote:
>> Hi -
>> If you don't care about the issues and solutions maintaining 21 OSes
>> for libwebsockets CI, you can just ignore this kind of email.
The Sai mac mini builder's updated to Big Sur, an lws user reported that
aftertheir upgrade, ulimit -n was set to a huge number indicating
"unlimited" for them, but here it's set to 2560. There's a patch on lws
to interpret any unreasonably large ulimit -n as 2560 just in case.
Sai had a big update that has been in progress for a few weeks splitting
the server part into two, previously all traffic went through lwsws in
proxy mode, both web traffic from browsers and control messages and
build logs from builders.
Since there are many builds in parallel and the logs have a lot of
traffic sent over wss in realtime, so they can be seen by the browsers
as the build, this was handled fine by lwsws in terms of proxying it
all, but in the sai server (formerly "sai-master") part its own event
loop was struggling with the load and having the blocking sqlite
commands for each log, at peak there are several hundred wss messages a
second being proxied from the builders and corresponding sqlite
transactions. These delays don't affect lwsws, just the getting of the
sai web rendering over the proxy link for /sai urls.
After the changes there are two separate sai processes that coordinate
by linking up over a unix domain socket, "sai-server" on :4444 that the
builders talk to and send their logs to, and "sai-web" which is on its
own unix domain socket and proxied by lwsws as before, so the web render
event loop is never blocked by log storage sqlite calls.
This required a lot of work separating out notifications from the
builders and events from the browsers to go overthe uds link between the
two. But now the proxied web stuff to /sai is responsive as normal
during builds, and various other improvements around build updates being
done in the DOM layer rather than by rewriting chunks of generated html
mean it's much smoother.
The gitohashi integration has also been improved so the results are in a
hidden (but live, over a wss link) div that fades in if you mouse over
the small sai summary icon, which is moved so it doesn't take up any
additional real-estate in gitohashi, now there are over 300 builds per
push for libwebsockets sai it's too big to just show all the time.
The Sai project docs are updated to reflect the changes, the overview
diagram should be easier to see how it fits together now
More information about the Libwebsockets