[Libwebsockets] polling for file descriptor i/o

Per Bothner per at bothner.com
Sun Mar 5 03:34:45 CET 2017


On 03/04/2017 03:39 PM, Andy Green wrote:

> Is it ok if multiple rx come before the remote side becomes writeable?  Mirror can do that safely only because he has a fifo and ultimately rx flow control to manage when he will get new rx.  So he never gets rx with nowhere to put it.

Bytes from the pty are appended to a buffer, so it is ok if multiple rx come before the remote side becomes writeable.
It would be easy enough to reallocate the buffer if it fills up.

However, we don't want to let the client process run wild - just like with a pipe,
we'd want suspend the client when it has produced too much output.

If a less-like pager is built into ldomterm (as I've been thinking about),
you'd want the user to be able to use a commend to kill the running process and
discard extra output.  That's easier (and cheaper) if the client gets paused
when it has produced huge amounts of output.

So perhaps this makes sense:  On each LWS_CALLBACK_RAW_RX_FILE append to a buffer
(which LWS_CALLBACK_SERVER_WRITEABLE later extracts).  If the buffer is getting
too much, send a SIGTSTP to the child progess.

> Without that since rx can come when it likes you can get a system that is fragile against real-world network links where the remote connection becoming writeable in effect races the next rx coming.
>
> If you will do the read in the rx callback and buffer it, it's ok but the rx flow control stuff is mandatory.

I'm having problems getting rx flow control to work.  I'll experiment some more.

>  Doing the read in the writable callback lets you eliminate the memory for the per-connection bulk data buffer, but otherwise the same.
>
>
> BTW I am still going on the zip integration... I can't use mmap since some platforms we support don't have it.

On non-mmap platforms you could read the zip file into a buffer in memory.  That is always supported
- but of course it might be expensive on low-memory platforms.

>So several iterations of optimization have ended up with libz stuff directly being called from zip_fops, no storage of the zip index in memory, that suggested other improvements to fops like generic file position and length held there, that has other knock-ons.  I also have implemented chunked inflate, so it does not require memory to inflate the whole file any more, but does it piecemeal like permessage-deflate, and seeking (although it is expensive since it restarts the inflate and inflates its way to the seek point).
>
> The last problem yesterday was I changed the code to use really virtual file paths, like /var/www/docs/manual.zip/index.html... that looks like it will require adding fstat to the fops, since lws serving code wants fstat it before opening it.

As long as I have the *option* of serving the zip file from memory (either
mmap or initialized data).
-- 
	--Per Bothner
per at bothner.com   http://per.bothner.com/



More information about the Libwebsockets mailing list