[Libwebsockets] serving compressed http response from a zip archive

Andy Green andy at warmcat.com
Sun Feb 12 02:00:16 CET 2017



On 12 February 2017 05:23:20 GMT+08:00, Per Bothner <per at bothner.com> wrote:
>I'm using libwebsockets for my JavaScript-based DomTerm terminal
>emulator
>(http://domterm.org).  The "ldomterm" application is the combination of
>DomTerm + libwebsockets + application code (derived from ttyd), which
>optionally starts up a window in a specified browser.

Sounds very cool.

>It's working quite well, but for various practical reasons it might
>make sense to
>put the "resources' (html + js + css) in a zip archive.  It seems
>straight-forward
>to mmap the zip file, and then use JUnzip
>(https://github.com/jokkebk/JUnzip) to
>decompress the requested file and send the data with lws_write_http.
>
>What would be really interesting if we could send the compressed data
>directly,

Right.

>and let the browser decompress it.  Most browsers accept gzip
>compressed replies,
>and creating a gzip result from a deflate-compressed zip entry should
>be
>just a matter of sending a gzip header, followed by the raw compressed
>zip entry,
>followed by the gzip tail (using the crc32 checksum present in the zip
>file).
>
>Does this make sense?  Has anyone tried something similar?

I have not done it, but I think it should be workable.

The main thing that will help stitch this in is lws file serving arrangements have a thin VFS between the code doing the serving and the filesystem, via a file_ops type struct of callbacks

https://github.com/warmcat/libwebsockets/blob/master/lib/libwebsockets.h#L4177

You can set your own file ops callback which provide normal posix semantics to open / seek / read / close that hides the work you're doing underneath.  The main things missing are

 - someone has to look at Accept-Encoding and see gzip, deflate to 'enable' this, if missing your code has to inflate it and not send compressed because peer can't handle it

 - someone has to send Content-Encoding: gzip,deflate if the url falls into a part of the namespace that can be served compressed

 - in the case you need to mix compressed and uncompressed (generated, say) content with no clear url namespace delineation, and no dynamic compression support on precompressed support, a way to inform lws the individual file is compressed or not

How does it sound if we add these to lws, so it

 - checks Accept-Encoding and passes a flag to .open() indicating if the peer could handle gzip compressed

 - add a pointer to .open that may be written to with returned flags indicating, eg, content is gzip-compressed

 - if the suitable return flag was set above and Accept-Encoding matched it, lws generic file serving will deal with appending Content-Encoding

-Andy

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.



More information about the Libwebsockets mailing list