[Libwebsockets] Using websockets as a pure buffer manipulation engine.

Alan Conway aconway at redhat.com
Thu Nov 17 14:45:03 CET 2016

On Wed, 2016-11-16 at 10:23 +0800, Andy Green wrote:
> On Wed, 2016-11-16 at 08:46 +0800, Andy Green wrote:
> > 
> > On Tue, 2016-11-15 at 14:59 -0500, Alan Conway wrote:
> > 
> > > 
> > > I had a look in docs etc. but cound't find an answer for this:
> > > 
> > > I want to add websocket support to a project that already has
> > > it's
> > > own
> > > multi-platform proactor, multi-SSL support etc. I would like to
> > > use
> > > libwebsockets as a pure codec, dealing with just byte buffers
> > > from
> > > my
> > > own IO layer (which might not even use file descriptors in some
> > > cases).
> > > Is that possible?
> > 
> > As it is, no.  Lws can hook into someone else's existing event loop
> > okay, and many people have done that over the years, but the lowest
> > common denominator as it stands is a socketfd.
> > 
> > Inside lws the stuff that actually touches the fds is isolated in a
> > few
> > places and eg, pure external event loops are supported like libuv
> > that
> > expose wrapped platform-specific objects instead of fds.  So it's
> > not
> > out of the question to modify it to allow it.
> > 
> > Still the most important event for lws is that the connection to
> > the
> > peer is "writeable", eg, POLLOUT from poll().  In all the ways to
> > use
> > lws this provides the basic heartbeat for the sequencing.  Is that
> > an
> > event that fits into your picture?
> ... and thinking about it RX flow control management, that lws
> controls
> whether pending RX packets remain unread by POLLIN control or other
> similar scheme, is also indispensable for a practical system.

Thanks! I have a short term project where this will work but a longer
term one that is more interesting. 

I'm working on qpid.apache.org/proton, an AMQP messaging library. It
currently has a single-threaded, event-based "reactor" API using
sockets & poll. It also has a low-level "protocol driver" that deals
strictly with byte buffers in memory - no IO assumptions.

That can be used to build more interesting, multi-threaded apps. For
example we have a concurrent Go binding that manages native Go net.Conn
connections and pumps data thru the driver. Go has no notion of
"polling" and net.Conn connections don't have an FD (some do deep
inside, but e.g. in memory pipes don't at all)

The FD notion also isn't very helpful for fast native windows IOCP - we
ended up having to fake a poll()-like loop which is a terribly
inefficient way to use IOCP.

So I'd love to see a byte-buffer based websocket "driver" that could be
used independently of the libwebsocket loop and has no assumptions
about FDs. Is this a feature you would consider? I would be happy to
contribute (with help!) as I've just gone thru this process with proton
so I have some experience :) It is not as drastic as it might sound -
the "driver" is still event-based, it is simply being pumped bytes from
some IO-aware code instead of being directly coupled to that code.


More information about the Libwebsockets mailing list