[Libwebsockets] new plugin: session-cgi
andy at warmcat.com
Sun Oct 14 04:54:20 CEST 2018
On 14/10/18 08:36, Void Your Warranty wrote:
> thank you to Andy and the community for this fantastic library!
> I needed the following functionality: For each session that upgrades to web sockets, the server spawns a new process in which it runs a certain executable ("application"). All messages from the client go to the standard input of the application, and the standard output of the application is sent back to the client.
Right, basically ws CGI.
> As I wasn't able to find such a functionality (neither anywhere else nor here), I tried to cook up a new protocol for libwebsockets. I call it "session-cgi" because it is like good old CGI applied to the individual web socket sessions. Here is the plugin I came up with:
> Please feel free to use it and perhaps tell me what you (dis)like.
It looks like you did a nice job on it.
Lws does support HTTP / CGI already, in fact mailman for the mailing
list on the site runs through it. It also translates headers and
payload for h2 (CGI is only specified for h1 between the server and the
CGI; chunked payload is explicitly forbidden on h2 and has to be
dechunked before it's passed back to the h2 client).
But in lws it's just wired up to http; so for what you want, your
protocol solution is pretty reasonable.
Although CGI generally is flexible and easy to integrate (and still
practically useful, cf mailman), it's generally dying out very slowly as
a technology model.
In addition to the (v)fork + exec burden, they are typically stateless
and have to reeducate themselves about their config for each connection
on top of the interpreter startup and init, before they can do anything
useful. For example python takes the best part of 1s to start up on my
RPi3, if you wanted that to do the CGI, it's already a millstone around
There's a slightly different model which avoids this, which is make a
separate, longrunning daemon for the functionality (wholly in whatever
language you like) that communicates over (typically) a unix domain
socket with the main server.
Integration with the front-end general server is by a so-called reverse
proxy. When the daemon gets the proxied connection, it either adds the
fd to its event loop or spawns a thread to service it. Either way, that
avoids the statelessness of cgi, and almost all the per-connection init
costs of cgi. For the python case, you paid the python startup cost
once when the daemon started.
Again though, lws only supports the reverse proxying for http (h1 in
fact, but again it will translate it to h2 for h2 clients). Gitohashi
uses this scheme with lwsws (on master) doing the reverse proxying...
gitohashi uses lws to do the serving on a unix domain socket its side
too, but the scheme doesn't depend on it being so.
Comparing this daemon scheme with CGI, the task is proxied over a socket
(using h1 semantics), not stdin/out/err pipes. That has a much cleaner
story for scalability (the front end server could (but lws doesn't
support it yet :-) ) round-robin between n tcp sockets on different
physical machines running the server to get its job done, not just the
local unix domain one).
Anyway CGI is definitely workable and it's great you have sent your work
out as a FOSS project. And extra marks for not on github...
> Kind regards,
> Void Your Warranty
> Libwebsockets mailing list
> Libwebsockets at ml.libwebsockets.org
More information about the Libwebsockets