[Libwebsockets] v2.0 coming very soon

Andy Green andy at warmcat.com
Tue May 17 12:44:14 CEST 2016



On May 17, 2016 6:20:30 PM GMT+08:00, Thomas Spitz <thomas.spitz at hestia-france.com> wrote:
>Hello Andy,
>
>Thanks for the new simplified etc-lwsws-conf.d-localhost-EXAMPLE. For
>your
>info, as SSL is commented (I suppose),
>http://localhost:7681/server-status
>redirects to the same page but in SSL.. Thus it doesn't work except if
>we
>call directly http://localhost:7681/server-status.html (I had to copy
>server-status.html
>into share/libwebsockets-test-server/)

I'm not sure what that's about, there's no such redirect in the example conf

https://github.com/warmcat/libwebsockets/blob/master/lwsws/etc-lwsws-conf.d-localhost-EXAMPLE

I wonder if the problem is previously you had STS enabled, that is sticky and causes your browser the subsequently force the connection the the same host to be https.

Either that or you have other conf files defining it in /etc/lwsws/conf.d, not just the example one.

I think you should first make sure if you just use the example in conf.d as they are, everything is correct for the test page, because that is what it is supposed to do, and I have the conf so I can also check it.  If you can confirm that, you can add your additional stuff.

So let's check just the example stuff alone first.

>I'm also trying to use your cgitest script adding the following line in
>bold in etc-lwsws-conf.d-localhost-EXAMPLE:
>
>> {
>>
>>  "vhosts": [ {
>>      "name": "localhost",
>>      "port": "7681",
>>      "interface": "lo",
>> #     "host-ssl-key":  "/etc/pki/tls/private/libwebsockets.org.key",
>> #     "host-ssl-cert": "/etc/pki/tls/certs/libwebsockets.org.crt",
>> #     "host-ssl-ca":   "/etc/pki/tls/certs/libwebsockets.org.cer",
>>      "access-log": "/var/log/lwsws/test-access-log",
>> #     "sts": "on",
>>      "mounts": [{
>>        "mountpoint": "/",
>>        "origin":
>>
>"file:///home/thomas/workspace/libwebsockets/buildPC-lwsws/bin/share/libwebsockets-test-server",

If you are running lwsws as root and telling it to drop privs to 'apache' as in the example, you will face a problem 'apache' does not have rights to go in /home/thomas.

Apache can go in /usr/share typically, so you might want to install things there.

>>        "default": "test.html",
>>        "cache-max-age": "60",
>>        "cache-reuse": "1",
>>        "cache-revalidate": "1",
>>        "cache-intermediaries": "0"
>>        }, {
>> #semble ne pas fonctionner car nécessite SSL
>>         "mountpoint": "/server-status",
>>         "origin":
>>
>"file:///home/thomas/workspace/libwebsockets/buildPC-lwsws/bin/share/libwebsockets-test-server/server-status",
>>         "default": "server-status.html"
>>        }, {
>>         "mountpoint": "/formtest",
>>         "origin": "callback://protocol-post-demo"
>>        }
>>
>>
>> *,{        "mountpoint": "/cgitest",        "origin":
>>
>"cgi:///home/thomas/workspace/libwebsockets/buildPC-lwsws/bin/share/libwebsockets-test-server/lws-cgi-test.sh"
>>      }*],
>>      # which protocols are enabled for this vhost, and optional
>>      # vhost-specific config options for the protocol
>>      #
>>      "ws-protocols": [{
>>        "dumb-increment-protocol": {
>>          "status": "ok"
>>        },
>>        "lws-mirror-protocol": {
>>          "status": "ok"
>>        },
>>        "lws-status": {
>>          "status": "ok"
>>        },
>>        "protocol-post-demo": {
>>          "status": "ok"
>>        },
>>        "lws-server-status": {
>>          "status": "ok",
>> "update-ms": "5000"
>>        }
>>      }]
>>     },
>> # redirect any guys coming in on http:8080 to http:7681
>>     {
>>      "name": "localhost",
>>      "port": "8080",
>> #     "sts": "on",
>>      "mounts": [{
>>        "mountpoint": "/",
>>        "origin": ">http://localhost:7681"
>>      }]
>> }
>>   ]
>> }
>>
>> But I only get 404 error pae.
>Here below in bold two stange lines in valgrind lwsws debug (file names
>seem truncated):
>
>> sudo valgrind ./lwsws
>> ==1678== Memcheck, a memory error detector
>> ==1678== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et
>al.
>> ==1678== Using Valgrind-3.10.1 and LibVEX; rerun with -h for
>copyright info
>> ==1678== Command: ./lwsws
>> ==1678==
>> lwsws[1678]: lwsws libwebsockets web server - license GPL2.1
>> lwsws[1678]: (C) Copyright 2010-2016 Andy Green <andy at warmcat.com>
>> lwsws[1678]: Using config dir: "/etc/lwsws/conf.d"
>> lwsws[1678]: Initial logging level 7
>> lwsws[1678]: Libwebsockets version: *2.0.0*
>> thomas at thomas-laptop-v2.0.0-44-g81a5a12

The -44 here means "44 patches ahead of 2.0.0"

>> lwsws[1678]: IPV6 not compiled in
>> lwsws[1678]: libev support not compiled in
>> lwsws[1678]: libuv support compiled in and enabled
>> lwsws[1678]:  Threads: 1 each 1024 fds
>> lwsws[1678]:  mem: platform fd map:  8192 bytes
>> lwsws[1678]:   Plugins:
>> lwsws[1678]:   Scanning ./share/libwebsockets-test-server/plugins/
>> lwsws[1678]:    libprotocol_dumb_increment.so
>> lwsws[1678]:    libprotocol_lws_mirror.so
>> lwsws[1678]:    libprotocol_lws_server_status.so
>> lwsws[1678]:    libprotocol_lws_status.so
>> lwsws[1678]:    libprotocol_post_demo.so
>> lwsws[1678]:  Compiled with OpenSSL support
>> lwsws[1678]:  mem: per-conn:          680 bytes + protocol rx buf
>> lwsws[1678]:  canonical_hostname = thomas-laptop
>> ==1678== Conditional jump or move depends on uninitialised value(s)
>> ==1678==    at 0x54E5A03: vfprintf (vfprintf.c:1661)
>> ==1678==    by 0x550C578: vsnprintf (vsnprintf.c:119)
>> ==1678==    by 0x54EE531: snprintf (snprintf.c:33)
>> ==1678==    by 0x401EC2: lejp_vhosts_cb (in
>> /home/thomas/workspace/libwebsockets/buildPC-lwsws/bin/lwsws)
>> ==1678==    by 0x4E4FD13: lejp_parse (in
>>
>/home/thomas/workspace/libwebsockets/buildPC-lwsws/lib/libwebsockets.so.8)
>> ==1678==    by 0x402D4C: lwsws_get_config_vhosts (in
>> /home/thomas/workspace/libwebsockets/buildPC-lwsws/bin/lwsws)
>> ==1678==    by 0x401677: main (in
>> /home/thomas/workspace/libwebsockets/buildPC-lwsws/bin/lwsws)

Can you build lws + lwsws in debug mode so I can see the line numbers?

cmake .. -DCMAKE_BUILD_TYPE=DEBUG

I guess what happened is lejp string limit is too small for your unexpectedly big filenames.

>> ==1678==
>> lwsws[1678]: unsupported protocol:// er-status
>> lwsws[1678]: unsupported protocol:// gi-test.sh
>> lwsws[1678]:   adding protocol dumb-increment-protocol
>> lwsws[1678]:   adding protocol lws-mirror-protocol
>> lwsws[1678]:   adding protocol lws-status
>> lwsws[1678]:   adding protocol protocol-post-demo
>> lwsws[1678]:   adding protocol lws-server-status
>> lwsws[1678]: Creating Vhost 'localhost' port 7681, 6 protocols
>> lwsws[1678]:    mounting
>>
>file:///home/thomas/workspace/libwebsockets/buildPC-lwsws/bin/share/libwebsockets-test-server
>> to /
>> lwsws[1678]:    mounting callback://protocol-post-demo to /formtest
>> lwsws[1678]:    mounting server:dumb-increment-protocol to
>> lwsws[1678]:  Using non-SSL mode
>> lwsws[1678]: Creating Vhost 'localhost' port 8080, 1 protocols
>> lwsws[1678]:    mounting >http://localhost:7681 to /
>> lwsws[1678]:  Using non-SSL mode
>>
>> *lwsws[1678]: unsupported protocol:// er-statuslwsws[1678]:
>unsupported
>> protocol:// gi-test.sh*
>> lwsws[1678]:   adding protocol dumb-increment-protocol
>> lwsws[1678]:   adding protocol lws-mirror-protocol
>> lwsws[1678]:   adding protocol lws-status
>> lwsws[1678]:   adding protocol protocol-post-demo
>> lwsws[1678]:   adding protocol lws-server-status
>> lwsws[1678]: Creating Vhost 'localhost' port 7681, 6 protocols
>> lwsws[1678]:    mounting
>>
>file:///home/thomas/workspace/libwebsockets/buildPC-lwsws/bin/share/libwebsockets-test-server
>> to /
>> lwsws[1678]:    mounting callback://protocol-post-demo to /formtest
>> lwsws[1678]:    mounting http://dumb-increment-protocol to
>> lwsws[1678]:  Using non-SSL mode
>> lwsws[1678]:  using listen skt from vhost localhost
>> lwsws[1678]: Creating Vhost 'localhost' port 8080, 1 protocols
>> lwsws[1678]:    mounting >http://localhost:7681 to /
>> lwsws[1678]:  Using non-SSL mode
>> lwsws[1678]:  using listen skt from vhost localhost
>> lwsws[1678]: getpwuid: unable to find uid 48
>> lwsws[1678]: lws_protocol_init
>> lwsws[1678]:     vh localhost prot protocol-post-demo opt status
>> lwsws[1678]:     vh localhost prot lws-status opt status
>> lwsws[1678]:     vh localhost prot lws-server-status opt update-ms
>> lwsws[1678]:     vh localhost prot lws-server-status opt status
>> lwsws[1678]:     vh localhost prot lws-mirror-protocol opt status
>> lwsws[1678]:     vh localhost prot dumb-increment-protocol opt status
>> lwsws[1678]:     vh localhost prot protocol-post-demo opt status
>> lwsws[1678]:     vh localhost prot lws-status opt status
>> lwsws[1678]:     vh localhost prot lws-server-status opt update-ms
>> lwsws[1678]:     vh localhost prot lws-server-status opt status
>> lwsws[1678]:     vh localhost prot lws-mirror-protocol opt status
>> lwsws[1678]:     vh localhost prot dumb-increment-protocol opt status
>> lwsws[1678]:  permessage-deflate requires the protocol
>> (dumb-increment-protocol) to have an RX buffer >= 128
>> lwsws[1678]: ext permessage-deflate failed construction
>> lwsws[1678]:  Capping pmd rx to 128
>> lwsws[1678]: cache_len 318
>> lwsws[1678]:  Capping pmd rx to 128
>> lwsws[1678]: cache_len 112
>
>
>By the way I am using last git version (2.0.0 should be changed in
>Cmake ?)

No master branch continues from the last major release.  The release won't change until 2.1, because it's in development, not a release.

You can use the other info to track it, -44 if how many patches on top, and gxxxxxx is the git hash last time cmake was run.

-Andy

>Best regards,
>Thomas
>
>
>2016-05-17 3:02 GMT+02:00 Andy Green <andy at warmcat.com>:
>
>>
>>
>> On 05/16/2016 07:34 PM, Thomas Spitz wrote:
>>
>>> Hello Andy,
>>>
>>> Even if I think option 1 will better suit my final need (Everything
>>> compiled in one project), I'm trying to test the second option
>>> (Replacing my whole server app with lwsws using JSON config files
>and
>>> compile my app as plugins).
>>>
>>> Therefore I created /etc/lwsws/conf and /etc/lwsws/cond/d/localhost
>>> files. Do you have a localhost file corresponding to test-server
>example
>>> (etc-lwsws-conf.d-localhost-EXAMPLE seems unrelated to test-server)?
>>>
>>
>> Sorry this is a very reasonable question I missed last night.
>>
>> I have added a patch on master that changes the example config to
>match
>> the old test server, and adds how to set it up in README.lwsws.md
>>
>>
>>
>https://github.com/warmcat/libwebsockets/commit/34b8161e04ffd1f6506bfc9014b649625e8c5212
>>
>> That should just work then as before (port 7681, no SSL).
>>
>> You can both meddle with that file, and also add cut-and-pastes of it
>with
>> different "port" to a different file in
>/etc/lwsws/conf.d/something-else
>> that will also exist in lwsws alongside the sample config.
>>
>> -Andy
>>
>>
>> I also copied all libprotocol_* files from /build/lib to
>>> build/bin/share/libwebsockets-test-server/plugins/ but I got the
>>> following error when launching lwsws
>>>
>>>     lwsws[22220]: lwsws libwebsockets web server - license GPL2.1
>>>     lwsws[22220]: (C) Copyright 2010-2016 Andy Green
><andy at warmcat.com
>>>     <mailto:andy at warmcat.com>>
>>>     lwsws[22220]: Using config dir: "/etc/lwsws/conf.d"
>>>     lwsws[22220]: Initial logging level 7
>>>     lwsws[22220]: Libwebsockets version: 2.0.0 thomas at thomas-laptop-
>>>     lwsws[22220]: IPV6 not compiled in
>>>     lwsws[22220]: libev support not compiled in
>>>     lwsws[22220]: libuv support compiled in and enabled
>>>     lwsws[22220]:  Threads: 1 each 1024 fds
>>>     lwsws[22220]:  mem: platform fd map:  8192 bytes# mkdir -p
>>> /etc/lwsws/conf.d /var/log/lwsws
>>>
>> # cp ./lwsws/etc-lwsws-conf-EXAMPLE /etc/lwsws/conf
>> # cp ./lwsws/etc-lwsws-conf.d-localhost-EXAMPLE
>> /etc/lwsws/conf.d/test-server
>> # sudo lwsws
>>
>>>     lwsws[22220]:   Plugins:
>>>     lwsws[22220]:    libprotocol_client_loopback_test.so
>>>     lwsws[22220]:    libprotocol_dumb_increment.so
>>>     lwsws[22220]:    libprotocol_lws_mirror.so
>>>     lwsws[22220]:    libprotocol_lws_server_status.so
>>>     lwsws[22220]:    libprotocol_lws_status.so
>>>     lwsws[22220]:    libprotocol_post_demo.so
>>>     *** Error in `./lwsws': double free or corruption (fasttop):
>>>     0x0000000001e9a380 ***
>>>     Aborted (core dumped)
>>>
>>> I understand the global philosophy of lwsws 2.0 and plugins but I
>>> haven't yest understood how to implement it simply.
>>>
>>> Best regards,
>>> Thomas
>>>
>>> 2016-04-24 20:20 GMT+02:00 Andy Green <andy at warmcat.com
>>> <mailto:andy at warmcat.com>>:
>>>
>>>
>>>
>>>     On April 24, 2016 9:28:32 PM GMT+08:00, Thomas Spitz
>>>     <thomas.spitz at hestia-france.com
>>>     <mailto:thomas.spitz at hestia-france.com>> wrote:
>>>      >Hello Andy, hello everyone,
>>>
>>>     (adding back the ml)
>>>
>>>     >I have looked at the new features you have introduced and I
>wonder
>>>     >whether
>>>     >this could solve one of the old issue we discussed in the
>following
>>>     >posts :
>>>     >
>>>
>https://libwebsockets.org/pipermail/libwebsockets/2016-January/002164.html
>>>     >
>>>
>https://libwebsockets.org/pipermail/libwebsockets/2014-September/001355.html
>>>     >
>>>     >To make it clearer, here is my application :
>>>     >At the present, I use libwebsocket on an embedded linux (1core
>450MHz
>>>     >ARM)
>>>     >mainly to communicate (two-way communication) in real time
>through
>>> one
>>>     >port
>>>     >(SSL port 443) with desktop softwares, mobile apps and also
>browsers
>>>     >(less
>>>     >important) using my own protocol over websocket.
>>>     >
>>>     >I now also need to retrieve files that are stored on my
>embedded
>>> linux
>>>     >(data log files, pictures, sounds, ....) in a simple way (eg: a
>bit
>>>     >like a
>>>     >ftp) but using the same port 443? Thus, we only need to create
>one
>>> NAT
>>>     >rule
>>>     >in routers to access our embedded linux server from the WAN.
>>>
>>>     Right... you could always do that on one port by choosing what
>files
>>>     to serve in the http callback.  For example the test-server
>serves
>>>     both his ws connections and the html and png all on one port. 
>But
>>>     for that you had to write the low-level code.
>>>
>>>     Master branch (to be lws v2.0) gives you two additional options
>>>
>>>       1) Keep your existing code and attach your own mounts at vhost
>>>     creation time
>>>
>>>
>>>
>https://github.com/warmcat/libwebsockets/blob/master/README.coding.md#using-lws-v2-vhosts
>>>
>>>     Lws will serve the mounted contents automatically from where
>it's
>>>     mounted in the server namespace
>>>
>>>       2) Replace your whole server app with lwsws, which is how
>>>     libwebsockets.org <http://libwebsockets.org> is working now.
>>>
>>>    
>https://github.com/warmcat/libwebsockets/blob/master/README.lwsws.md
>>>
>>>     Then you can configure the vhosts and mounts (and everything
>else)
>>>     in JSON instead of writing code.
>>>
>>>
>>>
>https://github.com/warmcat/libwebsockets/blob/master/lwsws/etc-lwsws-conf-EXAMPLE
>>>
>>>
>https://github.com/warmcat/libwebsockets/blob/master/lwsws/etc-lwsws-conf.d-localhost-EXAMPLE
>>>
>>>     To implement your custom ws protocol part, do it as an lws
>protocol
>>>     plugin.  These are much simpler and more selfcontained than 'cut
>and
>>>     paste the test server'... eg
>>>
>>>
>>>
>https://github.com/warmcat/libwebsockets/blob/master/plugins/protocol_dumb_increment.c
>>>
>>>     You can build your plugin against lws headers and libs, and copy
>to
>>>     /usr/[local/]share/libwebsockets-test-server/plugins/, lws scans
>the
>>>     dir and initializes any plugins found.
>>>
>>>     You need to enable the ws protocols you want to offer in each
>vhost,
>>>     see how the libwebsockets.org <http://libwebsockets.org> conf
>above
>>>     does it.  You can also set per vhost protocol options there to
>>>     modify or control what yhe protocol plugin does on a per-vhost
>basis.
>>>
>>>     This way you can shed almost all your custom server code and get
>>>     more featureful and maintainable results.
>>>
>>>     -Andy
>>>
>>>      >If I am correct, it seems that vhost can help us to have the
>>> standard
>>>      >LWS
>>>      >webserver, the WS server and a kind of "FTP" all running on
>the same
>>>      >443
>>>      >port? If so, do you have an idea of a service that could work
>with
>>> LWS
>>>      >in
>>>      >order to serve files from an embedded linux file system?
>Ideally, I
>>>      >wouldn't like to have to create my own protocol to retrieve
>files
>>> from
>>>      >my
>>>      >filesystem.
>>>      >
>>>      >Thanks in advance for your very nice support and sorry for
>such
>>> novice
>>>      >questions.
>>>      >
>>>      >Best regards,
>>>      >Thomas
>>>      >
>>>      >
>>>      >2016-04-23 14:12 GMT+02:00 Andy Green <andy at warmcat.com
>>>     <mailto:andy at warmcat.com>>:
>>>
>>>      >
>>>      >> Hi -
>>>      >>
>>>      >> In the spirit of eating my own dogfood the last couple of
>weeks my
>>>      >own web
>>>      >> services for https://libwebsockets.org and
>https://warmcat.com
>>> have
>>>      >been
>>>      >> running using lwsws on master, with real traffic and with
>real
>>> CGIs
>>>      >(cgit +
>>>      >> mailman).
>>>      >>
>>>      >> After tracking down the last CGI related bugs this week, it
>seems
>>> to
>>>      >be
>>>      >> stable.
>>>      >>
>>>      >> It's running 11 vhosts, although a lot of these are
>redirects for
>>>      >older
>>>      >> services.  You can see its current status (via websockets,
>>>     naturally)
>>>      >here:
>>>      >>
>>>      >> https://libwebsockets.org/server-status/
>>>      >>
>>>      >> (This is a ws protocol plugin on lwsws and is included in
>>> ./plugins)
>>>      >>
>>>      >> I've prepared a page listing the new features here
>>>      >>
>>>      >> https://libwebsockets.org/lws-2.0-new-features.html
>>>      >>
>>>      >> Basically a big advantage is if you are making a server type
>>>      >application,
>>>      >> you can just use lwsws and only need to write the ws
>protocol
>>> plugin
>>>      >and
>>>      >> set up some JSON to deploy it, instead of cut and pasting
>custom
>>>      >code.
>>>      >>
>>>      >> If you want to deploy more protocols later, they are
>selfcontained
>>>      >plugins
>>>      >> and will play nice together.  The server stack is already
>much
>>> more
>>>      >mature
>>>      >> than 1.7 (the automated mountpoint serving and http cache
>control,
>>>      >> apache-compatible logging amongst other things).  And you
>can
>>>     benefit
>>>      >from
>>>      >> centralized maintenance of the whole server stack that way.
>>>      >>
>>>      >> For example the lws test server on
>>>      >https://libwebsockets.org/testserver
>>>      >> has its ws protocols served by lwsws plugins; there's no
>code at
>>> all
>>>      >about
>>>      >> them in lwsws itself.
>>>      >>
>>>      >> If you don't care right now, well it's disabled in cmake by
>>> default
>>>      >> currently and everything works as before by default.
>>>      >>
>>>      >> There's some docs on lwsws here:
>>>      >>
>>>      >>
>>> https://github.com/warmcat/libwebsockets/blob/master/README.lwsws.md
>>>      >>
>>>      >> and you can see the actual lwsws JSON config for
>>>     libwebsockets.org <http://libwebsockets.org> in
>>>      >the
>>>      >> tree as an example
>>>      >>
>>>      >>
>>>      >>
>>>      >
>>>
>https://github.com/warmcat/libwebsockets/blob/master/lwsws/etc-lwsws-conf.d-localhost-EXAMPLE
>>>      >>
>>>      >> There are two features that will have to wait for a
>subsequent
>>>      >version I
>>>      >> think, one is http proxy, although it works it's not
>integrated
>>> into
>>>      >lwsws
>>>      >> yet (because I don't have a use for it right now and nobody
>has
>>>     asked
>>>      >for
>>>      >> it), the other is mbedtls (people asking for it but nobody
>wants
>>> it
>>>      >enough
>>>      >> to help work on it, even though I have done all the
>plumbing).
>>>      >>
>>>      >> Otherwise it's in good shape AFAICT, it was in Coverity
>today and
>>>      >it's
>>>      >> back at zero defects.
>>>      >>
>>>      >> If there's any comments, questions or suggestions, it'd be
>>>     helpful to
>>>      >hear
>>>      >> them before it gets released ^^
>>>      >>
>>>      >> -Andy
>>>      >> _______________________________________________
>>>      >> Libwebsockets mailing list
>>>      >> Libwebsockets at ml.libwebsockets.org
>>>     <mailto:Libwebsockets at ml.libwebsockets.org>
>>>      >> http://libwebsockets.org/mailman/listinfo/libwebsockets
>>>      >>
>>>
>>>
>>>




More information about the Libwebsockets mailing list