[Libwebsockets] new plugin: session-cgi

Andy Green andy at warmcat.com
Sun Oct 14 04:54:20 CEST 2018

On 14/10/18 08:36, Void Your Warranty wrote:
> Hi,
> thank you to Andy and the community for this fantastic library!
> I needed the following functionality: For each session that upgrades to web sockets, the server spawns a new process in which it runs a certain executable ("application"). All messages from the client go to the standard input of the application, and the standard output of the application is sent back to the client.

Right, basically ws CGI.

> This is because I wanted to play around with web sockets, but I didn't want to use node.js because I don't want to program the application logic in javascript. I also didn't want to program the application logic in C/C++, but, hey, I can still use libwebsockets and lwsws in order to "feed" all incoming messages to my own executable and vice versa. I am then free to use any language I like for my application. The only downside is that different sessions can no longer "see each other" at web server level. Luckily, this is no problem im my case and I suppose in many other situations, too. Most applications probably use some persistent data base, and so a common data base is a way that different sessions can interact.
> As I wasn't able to find such a functionality (neither anywhere else nor here), I tried to cook up a new protocol for libwebsockets. I call it "session-cgi" because it is like good old CGI applied to the individual web socket sessions. Here is the plugin I came up with:
> https://gitlab.com/voidyourwarranty/session-cgi
> Please feel free to use it and perhaps tell me what you (dis)like.

It looks like you did a nice job on it.

Lws does support HTTP / CGI already, in fact mailman for the mailing 
list on the site runs through it.  It also translates headers and 
payload for h2 (CGI is only specified for h1 between the server and the 
CGI; chunked payload is explicitly forbidden on h2 and has to be 
dechunked before it's passed back to the h2 client).

But in lws it's just wired up to http; so for what you want, your 
protocol solution is pretty reasonable.

Although CGI generally is flexible and easy to integrate (and still 
practically useful, cf mailman), it's generally dying out very slowly as 
a technology model.

In addition to the (v)fork + exec burden, they are typically stateless 
and have to reeducate themselves about their config for each connection 
on top of the interpreter startup and init, before they can do anything 
useful.  For example python takes the best part of 1s to start up on my 
RPi3, if you wanted that to do the CGI, it's already a millstone around 
its neck.

There's a slightly different model which avoids this, which is make a 
separate, longrunning daemon for the functionality (wholly in whatever 
language you like) that communicates over (typically) a unix domain 
socket with the main server.

Integration with the front-end general server is by a so-called reverse 
proxy.  When the daemon gets the proxied connection, it either adds the 
fd to its event loop or spawns a thread to service it.  Either way, that 
avoids the statelessness of cgi, and almost all the per-connection init 
costs of cgi.  For the python case, you paid the python startup cost 
once when the daemon started.

Again though, lws only supports the reverse proxying for http (h1 in 
fact, but again it will translate it to h2 for h2 clients).  Gitohashi


uses this scheme with lwsws (on master) doing the reverse proxying... 
gitohashi uses lws to do the serving on a unix domain socket its side 
too, but the scheme doesn't depend on it being so.

Comparing this daemon scheme with CGI, the task is proxied over a socket 
(using h1 semantics), not stdin/out/err pipes.  That has a much cleaner 
story for scalability (the front end server could (but lws doesn't 
support it yet :-) ) round-robin between n tcp sockets on different 
physical machines running the server to get its job done, not just the 
local unix domain one).

Anyway CGI is definitely workable and it's great you have sent your work 
out as a FOSS project.  And extra marks for not on github...


> Kind regards,
> Void Your Warranty
> _______________________________________________
> Libwebsockets mailing list
> Libwebsockets at ml.libwebsockets.org
> https://libwebsockets.org/mailman/listinfo/libwebsockets

More information about the Libwebsockets mailing list