[Libwebsockets] Sai administrivia

[Andy Green]

On 11/2/20 4:50 PM, Andy Green wrote:
> Hi -
> 
> If you don't care about the issues and solutions maintaining 21 OSes for 
> libwebsockets CI, you can just ignore this kind of email.

I went around updating the Sai guest images as well, moving the f32 
guest to f33 and the others just updated inside their release, and 
rebuild latest lws and sai inside the guests too, this mostly went 
smoothly, except:

1) ESP32 esp-idf runs under a Fedora guest, this was OK after the 
upgrade but python user-specific deps needed updating as the 'sai' user 
that performs the builds, by running the esp-idf install.sh script as 
that user.  After that it could build OK.

I put off updating esp-idf itself since I have an ESP32S2 dev board I 
will get to at some point and do it then.

2) Gentoo sai-builder could no longer fetch any remote git ref

[2020/11/06 05:50:57:5517] N: sai_mirror_local_checkout: Attempting to 
fetch 
ref-0865b9692e649103979944774d128eb78a7905e6:ref-0865b9692e649103979944774d128eb78a7905e6 
from mirror to /home/sai/jobs/0-0.2/libwebsockets/
[2020/11/06 05:50:57:5517] N: sai_mirror_local_checkout: 
git_remote_fetch() says -1
[2020/11/06 05:50:57:5517] E: sai_mirror_local_checkout: 
git_remote_fetch libgit err: -1 invalid version 0 on git_proxy_options

...this turned out to be caused by the libgit2 version in current gentoo 
(1.0.1) deviating from the comment in the api header that the "default 
is no proxy" to require the proxy member of the remote fetch options to 
be initialized now, it's enough to add this in Sai builder before the 
remote fetch

    git_proxy_options_init(&opts.proxy_opts, GIT_PROXY_OPTIONS_VERSION);

f33 has 1.0.1 but built OK without changes, it looks like the version 
gentoo emerged is actually a post-1.0.1 development version and this 
change will anyway be needed in the others later, so I added a cmake 
test for the api and compile it conditionally.


There's an interesting FOSS / CI related article today via hackernews...

https://www.jeffgeerling.com/blog/2020/travis-cis-new-pricing-plan-threw-wrench-my-open-source-works

...the author recognizes he got useful free help from Travis for FOSS 
projects CI (as did we), is upset it's going away and he has to 
un-lockin himself from their build system, but somehow thinks github 
actions is the answer.  I didn't know that travis was bought by a 
private equity group, but I saw the fallout from the problems around New 
Year's time, with 24h build delays.

Obviously putting all your CI eggs in microsoft's basket is not solving 
the real issue that made the problem with relying on travis' largesse. 
The article just doesn't consider that with github the lock-in has gone 
up a level and the core problem just moved sideways.  IIRC Travis was 
building on AWS, I don't know where github actions run but it is also 
likely to be someone's commercial cloud with a bill to pay, even if the 
money is passing from microsoft's left hand to its right hand.  The only 
real solution is selfhost the CI, and not in the mindbending definition 
"self-hosted" in, eg, "github actions self-hosted runners on google cloud"

https://github.blog/2020-08-04-github-actions-self-hosted-runners-on-google-cloud/

This has costs since someone has to have physical boxes and be the 
meta-admin and maintain the guests for a lot of different OSes, but it 
has advantages, eg, the ESP32 builds are auto-flashed and checked on 
WROVER physical boards here via CTest and sai-expect over serial to 
confirm it works, and I can support a much wider range of platforms 
including ones in qemu like freebsd and risc-v.

-Andy