[Libwebsockets] pointers for implementing websocket tcp gateway and a static http server

Andy Green andy at warmcat.com
Tue Oct 18 15:10:34 CEST 2016

On October 18, 2016 7:24:05 PM GMT+08:00, jsaak <jsaak at napalm.hu> wrote:
>I hacked and copy-pasted something together.
>It does sort of work. Until it crashes.


>(man, your library is confusing :)

Maybe you are just easily confused, and / or did not really look at the test + example code and docs provided with the library.

>you can see the code here:

Thanks but it sounds kind of broken.

>I do not want to use lwsws, since i can not run new processes, i have

Lwsws exists wholly in the library, except for its 'main.c'


which is CC0 to encourage integrating and adapting that by cut-and-paste.  It doesn't require any new process just an event loop which you need anyway.  If you can't use libuv in your case, fair enough, but absent your explaining that...

>embed this thing to an existing process. And dynamic .so loading is not
>very cross platform friendly i guess.

Since we use libuv for that, and it works on the platforms libuv supports, including windows.

Also there are two related but separate issues there, logical protocol encapsulation at source level, and dynamic plugin loading.  Even ESP8266 which just has a single thread in non-RTOS mode uses the plugins via static #includes; this keeps the benefits of the clean source code partitioning and reuse without needing dynamic plugin support.


>What I need is some strategy on rx/tx buffering and partial reads.

Depends what you're trying to do.

>Also I want to avoid malloc if possible.

Well stay away from node.js.

>How does your lib work in this regard?

Have you read

 - The 'coding with lws' README


 - info about 'callback when writable'


Generally posix leaves nonblocking code with a problem.  You can get a POLLOUT indication that you may write something to the socket.  But there is no indication or guarantee of how much you may write.

You actually have to do the write and learn if it failed to take everything.

Lws will pick up the pieces automagically if that happens (malloc a temp buffer and conceal it is sending the remainder piecemeal before letting you see it is writable again).  This is expensive, so the way atm is tune the lump size you send things in that your box rarely fails to accept everything.

>Shall I implement a dynamic buffer?
>The OS TCP stack has a buffer, maybe i can use that and disable, and 
>reenable filehandles, as needed?

In fact that is not enough: lws maintains a per-connection rx buffer for you internally so you can turn rx on and off in user code implicitly or explicitly with immediate effect up to the point rx has been parsed and return safely to the event loop.

Consider the case multiple transactions of http/1.1 keepalive headers were pipelined by the client, lws read them all since it cannot know where the partition between the transactions is, but your user code must return to the event loop as part of handling the first http transaction.  Lws deals with it transparently and feeds the buffered rx cleanly when your code can deal with it.

At least I appreciate you understand the generic need for rx flow control.

>(1. read some bytes
>2. disable filehandle, so i do not get any more data until i wrote them

Did you look at how the mirror sample does rx flow control?


The api is documented


 * lws_rx_flow_control() - Enable and disable socket servicing for
 *				received packets.
 * If the output side of a server process becomes choked, this allows flow
 * control for the input side.
 * \param wsi:	Websocket connection instance to get callback for
 * \param enable:	0 = disable read servicing for this connection, 1 = enable
lws_rx_flow_control(struct lws *wsi, int enable);

>  3. wait for write
>  4. write those bytes
>  5. reenable read filehandle)
>OR leave the filehandles, and set an internal flag in my app so I do
>read more data until it is successfully written?

Wouldn't help since lws decides when to call you back with rx.

>I guess that they are level triggered.
>OR do i have to buffer everything in my app?

If you want message reassembly, yes that is your job.  If you don't, just eat rx when you are called back with some, and use the rx flow control to modulate getting any more rx callbacks. ----->

>Shall I copy the data to my app when reason == LWS_CALLBACK_RECEIVE?
>Or can i use it later?

Why don't you properly study the mirror test protocol.


There isn't much code, but it is both very demanding as the clients increase and the FIFO "shared world updated at the different client rates" represents a model commonly needed by user code.  Often many 'confusions' disappear if you look at what the mirror protocol succeeds to reliably do in your browser, and then look at how it does it in the code and think how it can be adapted for your purpose (the test stuff is all CC0 to facilitate that).

>Also i am confused about partial reads.

You are confused about that because you assumed (nothing in lws suggested it) it assembles messages.  Lws always streams whatever bits of frames it gets as it comes into the user code.  It doesn't assume it can burn the memory, or latency to defer notification of reception, to assemble the messages in one place at one time.  If you want that, it's your job in the user code, which will receive the pieces in order.

>From a library pov you may be streaming audio or video of single message size up to 2^63, or even endless message content allowed by ws.  And lws is designed to be modest about memory.

>according to the log i have some kind of protocol rx buf.
>(how much? the 4th parameter in struct lws_protocols protocols[] ? )
>" NOTICE: mem: per-conn: 712 bytes + protocol rx buf"

You can set the size of the per-connection buffer used for rx chunks when you declare the protocol struct.  Look at the test code / libwebsockets.h definition of lws_protocols or


>but you mention elsewhere that i need to check for final fragment?

If you care to follow message boundaries, yes you need to know if the rx buffer you just got called back for is the end of a message.

>"To support fragmented messages you need to check for the final frame
>a message with lws_is_final_fragment."
>So what is that buffer doing if not assembling fragmented packets?

It's not pretending to be javascript.  Lws allows you to reassemble it if you want, or to stream endless non-message content like audio if you want without exhausting memory needlessly and delaying logically receiving anything until you have everything, for unconstrained sizes of 'everything'.


>Big thanks if you find the time to answer my questions!
>On 10/12/2016 10:21 AM, Andy Green wrote:
>> On Wed, 2016-10-12 at 09:38 +0200, jsaak wrote:
>>> On 10/12/2016 09:25 AM, Andy Green wrote:
>>>> On Wed, 2016-10-12 at 09:16 +0200, jsaak wrote:
>>>>> I have an app which works under linux/android/ios/osx/windows,
>>>>> and
>>>>> uses
>>>>> TCP/IP to communicate. I want to bundle a websocket to tcp
>>>>> gateway
>>>>> to
>>>>> the app so i can access it through a browser. Also i want to
>>>>> include
>>>>> a
>>>>> static http server.
>>>>> I found libwebsockets, and i think it might be a solution.
>>>> Sounds like the right kind of task for lws.
>>>>> All I ask of you is some advice. How to start with libwebsockets?
>>>>> It
>>>>> is
>>>>> a bit confusing for me.
>>>> Run the test server is the best way to get started.
>>>> The key question is "How does your existing app wait on network
>>>> events
>>>> at the moment"?  It's like poll() or like an event loop?
>>>> If it already has its own poll() management, you should look at
>>>> using
>>>> External Poll support to integrate lws with it.
>>>> If it uses an event loop, lws has libuv (preferred) and libev event
>>>> loop support, you can integrate using that.
>>>> How nice a ride you get is mainly depending on the existing app's
>>>> wait
>>>> for network events method.
>>> I am using SDL_Net which uses select() on all platforms i think.
>>> But it may be irrelevant, since i want to put libwebsocket to a new
>>> thread (so i can use any event loop there), and only communicate
>>> it
>>> through TCP/IP.
>> Fair enough...
>>> I think libuv is the way to go for me. So shall i start with
>>> -server-v2.0.c
>>> ?
>> Yes... either that or lwsws.
>> Lwsws will take care of all of the boilerplate and give you an lws
>> webserver configurable with JSON in one step.  Basically you can
>> writing any code that is unrelated to your protocol.
>>> Do I have to make a default websocket protocol? And in that protocol
>>> i
>>> have to open/read/write/close the real TCP connection?
>> If you use test-server-v2.0.c style, you would copy the "main.c" type
>> stuff and write a plugin (initially, just copy dumb-increment plugin
>> enough) that will hold your protocol actions.
>> You'll also need to set up mounts etc to serve the http stuff
>> programatically, it shows you how in the example code.
>> lwsws lets you configure the mounts and so on in JSON.  You only need
>> to provide the protocol plugin then which is only concerned with your
>> actions.
>> Any way you do it, lws takes care of socket-level stuff including
>> accept and close, without you needing to do anything.
>> It's best to take advantage of lwsws if your server is essentially
>> standalone.  https://libwebsockets.org (and https://warmcat.com) are
>> served from an Rpi 3 using lwsws.
>> -Andy
>>> _______________________________________________
>>> Libwebsockets mailing list
>>> Libwebsockets at ml.libwebsockets.org
>>> http://libwebsockets.org/mailman/listinfo/libwebsockets
>Libwebsockets mailing list
>Libwebsockets at ml.libwebsockets.org

More information about the Libwebsockets mailing list