[Libwebsockets] CGI + proxying update and lwsws

Andy Green andy at warmcat.com
Mon Mar 28 05:27:59 CEST 2016



On 03/21/2016 04:29 PM, Andy Green wrote:
> Hi -
>
> Some updates on stuff of master first
>
> 1) The CGI support on master is now good enough to run cgit
> (https://git.zx2c4.com/cgit/) cgi, although out of the box it spawns a
> test script that sends back an html table of /proc/meminfo.  The bit in
> the test-server-http.c that instantiates it is like this, so you can see
> if you enabled cmake .. -DLWS_WITH_CGI=1  then you can got to /cgitest
> in the test server to see the script output.
>
> #ifdef LWS_WITH_CGI
>          if (!strncmp(in, "/cgitest", 8)) {
>              static char *cmd[] = {
>                  "/bin/sh",
>                  "-c",
>
> INSTALL_DATADIR"/libwebsockets-test-server/lws-cgi-test.sh",
>                  NULL
>              };
>
>              n = lws_cgi(wsi, cmd, 8 /* len of /cgitest */,
>                      5 /* timeout secs */);
> ...
>
> vfork + execvpe is only available on Linux + GNU stack.  Otherwise it
> falls back to fork + execvp.
>
> At the moment the work is split between the user callback and the
> library for maxiumum flexibility, but it will likely turn out some of
> the user callback code can go in the library as the default callback
> action.
>
> 2) lws now has basic support for http rewriting proxying.  If you enable
> it with cmake .. -DLWS_WITH_HTTP_PROXY=1  then you will also need to
> have libhubbub on your box
> (http://www.netsurf-browser.org/projects/hubbub/) but this is already in
> the major distros.
>
> It actually parses and regenerates the HTML using libhubbub operating in
> stream parser mode, replacing one selected string with another if it
> understands that the string is a legit URL.
>
> This is enough for me to proxy http://git.libwebsockets.org through
> https in the test server, with the URIs rewritten to continue to work.
>
>
>
> The end goal of these changes for me is to introduce a new "test server"
> lwsws, "libwebsockets webserver" which just does enough, as simply as
> possible, so I can run all my own web services on it instead of Apache.
>
> Above all it should be sanely secure by default.
>
> I don't think it should try to be like Apache, for that you can actually
> use Apache: instead it should just do all of what 90% of people need in
> as little space, with as few options and as little configuration as
> possible.
>
> I'm particularly interested in simplifying out apache permissions and
> htaccess and such so they just work for normal cases using the main
> process permissions, cgi should just work and proxying should just work.
>   And because the backend is already in lws, lwsws should be very little
> code itself in addition.  If you have to add a vhost it should be so
> trivial you can just edit the JSON without docs.
>
> Currently I'm imagining it takes JSON config file(s) from a ./conf.d/
> something like this:

This is now pushed on master.  The changes should have no impact on user 
applications.

There are two main patches, one

https://github.com/warmcat/libwebsockets/commit/d526c50c2227ab3daf43d8651c9ea82c26f84aec

adds the vhost layer throughout lws in a backwards-compatible way, and 
the other

https://github.com/warmcat/libwebsockets/commit/cd0c696a0d879f66c8142ace0619b944efd08915

adds a new toplevel application "lwsws".

I confirmed the original test apps work OK, and that lwsws works via 
both the default poll() loop and libuv, if you ask to build libuv 
support it will work via libuv automatically.

Windows made trouble again because they set their face against dirent.h 
the last ten years, so on windows only lwsws does not go through 
/etc/lwsws/conf.d/*, it only looks in /etc/lwsws/conf (patches 
gratefully accepted).  Otherwise I can see from Appveyor he builds OK.

Some basic docs on lwsws are here:

https://github.com/warmcat/libwebsockets/blob/master/README.lwsws.md

As it says lwsws is in an incomplete state but it can run and serve from 
multiple vhosts with and without ssl already, using the JSON config. 
And because lws is the backend for everything, it should be fairly solid 
already.

The vhost changes are transparent unless you use the 
LWS_SERVER_OPTION_EXPLICIT_VHOSTS option at context creation time.

If you do, you are expected to subsequently call 
lws_create_vhost(context, info, mounts); with the vhost-related members 
of the context creation info struct set for what you want, and 
optionally a linked-list of struct lws_http_mount * you created in 
memory with lws_write_http_mount(...);  See ./lwsws/conf.c for an 
example of how to do it.

As many listen sockets as needed will be created and vhosts that match 
each socket will be considered for Host: or SNI matching on connection.

-Andy



More information about the Libwebsockets mailing list