[Libwebsockets] Setting up a high-throughput libwebsockets server.

Andy Green andy at warmcat.com
Tue Oct 4 06:56:53 CEST 2016


On Mon, 2016-10-03 at 19:33 -0700, Chris Evans wrote:
> Thanks for the quick reply,
> 
> Re allocations: Heap allocations during the initial setup/handshake
> are fine so long as there are no allocations per query. (The
> connection will be long-running but serve 10,000K queries/sec)

Whether there are allocations handling what the connection is doing is
mainly up to the user code.

lws uses poll() type semantics by default, and waits to hear if it is
able to send something more on the connection by waiting for POLLOUT.

When that arrives, it's a sign we can send more on the connection, but
actually it gives no clue about how much the kernel will accept, just
that it will accept something.

If what you send per POLLOUT is modest and is always accepted (it seems
on Linux when I have studied it, < 1 MTU is always acceptable if
POLLOUT has come, but it's not guaranteed anywhere AFAIK) then there
won't be any malloc()s once the ws connection is established.

However in theory it's open to the OS to signal POLLOUT and then accept
one byte.  To handle that kind of fatal situation, lws will
automatically malloc up a buffer for the remainder and drain it
autonomously in an emergency.  That lets you always use transient stack
buffers to create packets safely.

However if you just throw megabytes at lws_write() in one go, which
will certainly not be accepted by send(), it will be forced to always
use this "emergency mechanism" and it'll be expensive mallocs and
memcpys every time.

So you must send in chunks sizes that the kernel will almost always
accept in send() after a POLLOUT if you want to more or less eliminate
mallocs.  If you expect to send packets < 1MTU, I guess nothing to do.
 But if you will send larger things, you will have to break them up
into frames and use ws FIN to manage the larger message.  Lws supports
everything needed to do that.

> Re echo-server: `libwebsockets-test-echo` seems to behave how I would
> expect an "echo server" to. i.e. if I connect to and query it with
> iocat (https://www.npmjs.com/package/iocat -- tool like netcat but
> speaking the websockets protocol), it echos back my query. (Though I
> understand if it wasn't built for performance testing)

OK.

> I've attached the pcap output from `tcpdump -i lo0 -c 200 -w
> output.pcap src port 6666 or dst port 6666` (running the same
> client/server commands as before). I'm not too familiar with
> networking/tcpdump/wireshark but it looks like the handshake goes
> down, a few "Test Message"s are echoed, and then the server resets
> the connection. 
> 
> I also added print statements to print the `lws_callback_reason` in
> the callback (First 250 lines of output attached as stdout.txt) which
> seem to confirm the reset/reconnects from the server's point of view.

I can see the log, but I will have to study it tonight or tomorrow.

-Andy

> Thanks again for the help,
>   Chris
> 
> On Mon, Oct 3, 2016 at 6:11 PM, Andy Green <andy at warmcat.com> wrote:
> > On Mon, 2016-10-03 at 14:22 -0700, Chris Evans wrote:
> > > Howdy,
> > >
> > > I'm looking to set up a simple query/response websockets-protocol
> > > server in a high-throughput environment: Serving a machine
> > learning
> > > model behind Amazon's new Application Load Balancer (which
> > > necessitates speaking the WebSockets protocol) . `libwebsockets`
> > > seems a good choice as 1) I'm not looking to build a full web
> > app,
> > > and 2) It doesn't allocate per query (is that correct?)
> > 
> > Depends on what you do while processing the query... by default it
> > allocates some space at the connection and ws protocol negotiation
> > and
> > doesn't need the heap otherwise.
> > 
> > > But I've run into issues when stress testing with the `tcpkali`
> > tool
> > > (https://github.com/machinezone/tcpkali), even against the
> > provided
> > > test-echo server; `tcpkali` is unable to make a connection (I've
> > had
> > > success with tcpkali in other contexts):
> > 
> > The echo server is just designed to talk to himself in client mode.
> >  When I wrote it, I didn't realize "echo" has a bigger meaning for
> > people looking to make performance measurements.  So it may not do
> > what
> > you want for that.
> > 
> > I tried to install tcpkali but it's not packaged in Fedora.
> > 
> > > ```
> > > chrisevans at CHRIS-EVANS:bin ((detached from v2.0.0))$
> > ./libwebsockets-
> > > test-echo --port 6666
> > > [2016/09/03 13:44:51:3543] NOTICE: Built to support client
> > operations
> > > [2016/09/03 13:44:51:3543] NOTICE: Built to support server
> > operations
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: libwebsockets
> > test
> > > server echo - license LGPL2.1+SLE
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: (C) Copyright
> > > 2010-2016 Andy Green <andy at warmcat.com>
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: Running in
> > server
> > > mode
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: Initial
> > logging
> > > level 7
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: Libwebsockets
> > > version: 2.0.0 chrisevans at CHRIS-EVANS-v2.0.0
> > 
> > You should use v2.0-stable branch or master, there are many fixes
> > since
> > v2.0, although I don't think that's the problem.
> > 
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: IPV6 not
> > compiled
> > > in
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: libev support
> > not
> > > compiled in
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: libuv support
> > not
> > > compiled in
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:  Threads: 1
> > each
> > > 10240 fds
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:  mem: platform
> > fd
> > > map: 81920 bytes
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:  Compiled with
> > > OpenSSL support
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:  SSL disabled:
> > no
> > > LWS_SERVER_OPTION_DO_SSL_GLOBAL_INIT
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: Creating Vhost
> > > 'default' port 6666, 1 protocols
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:  Listening on
> > port
> > > 6666
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:  mem: per-
> > conn:   
> > >       488 bytes + protocol rx buf
> > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: 
> > > canonical_hostname = CHRIS-EVANS
> > > ```
> > >
> > > ```
> > > chrisevans at CHRIS-EVANS:Clients (test_suite)$ tcpkali -m "Test
> > > Message" -c 1 --ws  127.0.0.1:6666
> > > Destination: [127.0.0.1]:6666
> > > Interface lo0 address [127.0.0.1]:0
> > > Interface lo0 address [169.254.169.254]:0
> > > Using interface lo0 to connect to [127.0.0.1]:6666
> > > Could not create 1 connection in allotted time (10s)
> > > ```
> > > Any help would be greatly appreciated!
> > 
> > This doesn't give me anything to go on... if you can tcpdump what's
> > happening on the port, and / or increase the logging (eg -d 65535)
> > at
> > the echo server, we might get a clue.
> > 
> > -Andy
> > 
> > 
> > >  
> > > -Chris
> > > _______________________________________________
> > > Libwebsockets mailing list
> > > Libwebsockets at ml.libwebsockets.org
> > > http://libwebsockets.org/mailman/listinfo/libwebsockets
> > 
> 



More information about the Libwebsockets mailing list