[Libwebsockets] Setting up a high-throughput libwebsockets server.

Chris Evans chris.evans at adroll.com
Tue Oct 4 18:45:12 CEST 2016


Re allocations: Thanks for the detailed explanation. The writes will be
small (user code stores what to write in a fixed buffer < 1MTU) so
everything looks good on that front.

On Mon, Oct 3, 2016 at 9:56 PM, Andy Green <andy at warmcat.com> wrote:

> On Mon, 2016-10-03 at 19:33 -0700, Chris Evans wrote:
> > Thanks for the quick reply,
> >
> > Re allocations: Heap allocations during the initial setup/handshake
> > are fine so long as there are no allocations per query. (The
> > connection will be long-running but serve 10,000K queries/sec)
>
> Whether there are allocations handling what the connection is doing is
> mainly up to the user code.
>
> lws uses poll() type semantics by default, and waits to hear if it is
> able to send something more on the connection by waiting for POLLOUT.
>
> When that arrives, it's a sign we can send more on the connection, but
> actually it gives no clue about how much the kernel will accept, just
> that it will accept something.
>
> If what you send per POLLOUT is modest and is always accepted (it seems
> on Linux when I have studied it, < 1 MTU is always acceptable if
> POLLOUT has come, but it's not guaranteed anywhere AFAIK) then there
> won't be any malloc()s once the ws connection is established.
>
> However in theory it's open to the OS to signal POLLOUT and then accept
> one byte.  To handle that kind of fatal situation, lws will
> automatically malloc up a buffer for the remainder and drain it
> autonomously in an emergency.  That lets you always use transient stack
> buffers to create packets safely.
>
> However if you just throw megabytes at lws_write() in one go, which
> will certainly not be accepted by send(), it will be forced to always
> use this "emergency mechanism" and it'll be expensive mallocs and
> memcpys every time.
>
> So you must send in chunks sizes that the kernel will almost always
> accept in send() after a POLLOUT if you want to more or less eliminate
> mallocs.  If you expect to send packets < 1MTU, I guess nothing to do.
>  But if you will send larger things, you will have to break them up
> into frames and use ws FIN to manage the larger message.  Lws supports
> everything needed to do that.
>
> > Re echo-server: `libwebsockets-test-echo` seems to behave how I would
> > expect an "echo server" to. i.e. if I connect to and query it with
> > iocat (https://www.npmjs.com/package/iocat -- tool like netcat but
> > speaking the websockets protocol), it echos back my query. (Though I
> > understand if it wasn't built for performance testing)
>
> OK.
>
> > I've attached the pcap output from `tcpdump -i lo0 -c 200 -w
> > output.pcap src port 6666 or dst port 6666` (running the same
> > client/server commands as before). I'm not too familiar with
> > networking/tcpdump/wireshark but it looks like the handshake goes
> > down, a few "Test Message"s are echoed, and then the server resets
> > the connection.
> >
> > I also added print statements to print the `lws_callback_reason` in
> > the callback (First 250 lines of output attached as stdout.txt) which
> > seem to confirm the reset/reconnects from the server's point of view.
>
> I can see the log, but I will have to study it tonight or tomorrow.
>
> -Andy
>
> > Thanks again for the help,
> >   Chris
> >
> > On Mon, Oct 3, 2016 at 6:11 PM, Andy Green <andy at warmcat.com> wrote:
> > > On Mon, 2016-10-03 at 14:22 -0700, Chris Evans wrote:
> > > > Howdy,
> > > >
> > > > I'm looking to set up a simple query/response websockets-protocol
> > > > server in a high-throughput environment: Serving a machine
> > > learning
> > > > model behind Amazon's new Application Load Balancer (which
> > > > necessitates speaking the WebSockets protocol) . `libwebsockets`
> > > > seems a good choice as 1) I'm not looking to build a full web
> > > app,
> > > > and 2) It doesn't allocate per query (is that correct?)
> > >
> > > Depends on what you do while processing the query... by default it
> > > allocates some space at the connection and ws protocol negotiation
> > > and
> > > doesn't need the heap otherwise.
> > >
> > > > But I've run into issues when stress testing with the `tcpkali`
> > > tool
> > > > (https://github.com/machinezone/tcpkali), even against the
> > > provided
> > > > test-echo server; `tcpkali` is unable to make a connection (I've
> > > had
> > > > success with tcpkali in other contexts):
> > >
> > > The echo server is just designed to talk to himself in client mode.
> > >  When I wrote it, I didn't realize "echo" has a bigger meaning for
> > > people looking to make performance measurements.  So it may not do
> > > what
> > > you want for that.
> > >
> > > I tried to install tcpkali but it's not packaged in Fedora.
> > >
> > > > ```
> > > > chrisevans at CHRIS-EVANS:bin ((detached from v2.0.0))$
> > > ./libwebsockets-
> > > > test-echo --port 6666
> > > > [2016/09/03 13:44:51:3543] NOTICE: Built to support client
> > > operations
> > > > [2016/09/03 13:44:51:3543] NOTICE: Built to support server
> > > operations
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: libwebsockets
> > > test
> > > > server echo - license LGPL2.1+SLE
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: (C) Copyright
> > > > 2010-2016 Andy Green <andy at warmcat.com>
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: Running in
> > > server
> > > > mode
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: Initial
> > > logging
> > > > level 7
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: Libwebsockets
> > > > version: 2.0.0 chrisevans at CHRIS-EVANS-v2.0.0
> > >
> > > You should use v2.0-stable branch or master, there are many fixes
> > > since
> > > v2.0, although I don't think that's the problem.
> > >
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: IPV6 not
> > > compiled
> > > > in
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: libev support
> > > not
> > > > compiled in
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: libuv support
> > > not
> > > > compiled in
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:  Threads: 1
> > > each
> > > > 10240 fds
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:  mem: platform
> > > fd
> > > > map: 81920 bytes
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:  Compiled with
> > > > OpenSSL support
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:  SSL disabled:
> > > no
> > > > LWS_SERVER_OPTION_DO_SSL_GLOBAL_INIT
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>: Creating Vhost
> > > > 'default' port 6666, 1 protocols
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:  Listening on
> > > port
> > > > 6666
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:  mem: per-
> > > conn:
> > > >       488 bytes + protocol rx buf
> > > > Oct  3 13:44:51 CHRIS-EVANS lwsts[14575] <Notice>:
> > > > canonical_hostname = CHRIS-EVANS
> > > > ```
> > > >
> > > > ```
> > > > chrisevans at CHRIS-EVANS:Clients (test_suite)$ tcpkali -m "Test
> > > > Message" -c 1 --ws  127.0.0.1:6666
> > > > Destination: [127.0.0.1]:6666
> > > > Interface lo0 address [127.0.0.1]:0
> > > > Interface lo0 address [169.254.169.254]:0
> > > > Using interface lo0 to connect to [127.0.0.1]:6666
> > > > Could not create 1 connection in allotted time (10s)
> > > > ```
> > > > Any help would be greatly appreciated!
> > >
> > > This doesn't give me anything to go on... if you can tcpdump what's
> > > happening on the port, and / or increase the logging (eg -d 65535)
> > > at
> > > the echo server, we might get a clue.
> > >
> > > -Andy
> > >
> > >
> > > >
> > > > -Chris
> > > > _______________________________________________
> > > > Libwebsockets mailing list
> > > > Libwebsockets at ml.libwebsockets.org
> > > > http://libwebsockets.org/mailman/listinfo/libwebsockets
> > >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://libwebsockets.org/pipermail/libwebsockets/attachments/20161004/6a3db572/attachment-0001.html>


More information about the Libwebsockets mailing list