[Libwebsockets] serving compressed http response from a zip archive

Denis Osvald denis.osvald at sartura.hr
Wed Feb 15 10:59:22 CET 2017



On 2017-02-13 11:50, Andy Green wrote:
> 
> 
> On 02/13/2017 06:31 PM, Denis Osvald wrote:
>> Hello everybody,
>>
>> I'm using lws to make a web UI from a system with possibly constrained
>> disk space. The resources (html, js, css) are stored on disk gzipped
>> individually and content-encoding: gzip is added for such files when
>> served by lws.
>>
>> Since this has (so far) been a special case not supported by lws,
>> currently I'm handling this case "manually" by bypassing lws
>> file-serving logic.
>>
>> I haven't looked at the recent changes in master, but do you think this
>> use case (where GET /file.js ends up with /file.js.gz +
>> content-encoding: gzip) would be resolved / made easier by new lws APIs?
> 
> Should be so.  Basically lws will signal to the VFS "open" in the flags
> if gzip-coding is possible for the related client.
> 
> And the VFS "open" can signal back if it is providing something already
> gzip-coded or not.
> 
> LWS file serving code will now deal with the headers both ways based on
> what the client and the VFS open told in *flags.
> 
> So there is no bloat overall, since someone had to handle the headers
> each way.  But now lws handles it in a standardized way.

Okay, thanks for the info. I should take a look and try this new feature.

Just another thought - the feature to support compressed serving is
implemented through struct lws_plat_file_ops . Could it have been also
implemented as an additional mount-protocol like LWSMPRO_(G)ZIP(PED)
which would map a zipped resource directory into some path?

Of course, then you couldn't mix-and-match /a/b.js.gz and /a/c.js.
To do this, you'd have to reinvent some kind of overlay and/or bind
mounts inside lws anyway...

>>
>> If not (or if I have more special-case requirements), what is the best
>> way to "bypass" lws's VFS / file-serving logic? Currently I'm doing it
>> as a hack by making a mount of "/dev/null" which makes lws code fall
>> through to my protocol handler, where I can handle special cases like
>> this. It works alright this way.
> 
> The choice is clear in the v1 test server, either let lws do it or do it
> manually like leaf.jpg.  In v2 it became more automatic with mounts or
> do it manually.
> 
> It's great that hack worked but it's completely random that it did, it's
> not intentional.
> 
> If there's something else useful to do in lws to support it we can talk
> about if it makes sense.

Okay. I'll see about that when I get around to porting the stuff to
v2-style file serving.

> 
> -Andy
> 
> 
> 
>>
>>
>> Regards,
>>
>> Denis Osvald
>>
>>
>> On 2017-02-12 11:54, Andy Green wrote:
>>>
>>> On 02/12/2017 10:27 AM, Per Bothner wrote:
>>>> On 02/11/2017 05:00 PM, Andy Green wrote:
>>>>
>>>>> On 12 February 2017 05:23:20 GMT+08:00, Per Bothner <per at bothner.com>
>>>>> wrote:
>>>>>> I'm using libwebsockets for my JavaScript-based DomTerm terminal
>>>>>> emulator
>>>>>> (http://domterm.org).  The "ldomterm" application is the
>>>>>> combination of
>>>>>> DomTerm + libwebsockets + application code (derived from ttyd), which
>>>>>> optionally starts up a window in a specified browser.
>>>>> Sounds very cool.
>>>> Thanks - I'm pretty excited about it.
>>>>
>>>> I previously uses Java-WebSocket (or for qtdomterm I used Qt's internal
>>>> WebSocket server). However, I'm happy to deprecate Java-WebSocket in
>>>> favor of libwebsockets, for multiple reasons: The latter would
>>>> typically
>>>> use less resources; it is nice to have http and ws on the same port;
>>>> and Java-WebSocket is not seeing any development.
>>>>
>>>> I'd like to give some credit to Shuanglei Tao, whose ttyd made it
>>>> a lot easier to switch to libwebsockets.  ldomterm is basically ttyd
>>>> but with xterm.js replaced by DomTerm.  (Plus I added options to
>>>> automatically open a browser window.)
>>>>
>>>>> The main thing that will help stitch this in is lws file serving
>>>>> arrangements have a thin VFS between the code doing the serving and
>>>>> the filesystem, via a file_ops type struct of callbacks
>>>>>
>>>>> https://github.com/warmcat/libwebsockets/blob/master/lib/libwebsockets.h#L4177
>>>>>
>>>>>
>>>>>
>>>>> You can set your own file ops callback which provide normal posix
>>>>> semantics to open / seek / read / close that hides the work you're
>>>>> doing underneath.
>>>> Thanks - I will look into serving from a zip archive - but initially
>>>> without
>>>> attempting to serve compressed data.  The latter will be a nice
>>>> optimization
>>>> that can be added when lws has the necessary support.
>>> The suggested changes are added on master now.
>>>
>>> I wasn't completely sure about the relationship between gzip and deflate
>>> tokens in the headers.  What I did is added both "gzip" and "deflate"
>>> need to be coming in to understand we could send gzip format, and if the
>>> .open() handler chose to do it, we will send "gzip, deflate" as the
>>> Content-Encoding.  If when you get to the point of testing it that's not
>>> right, we can change it.
>>>
>>> -Andy
>>>
>>> -Andy
>>>
>>>> I expect I'll have more questions soon.
>>> _______________________________________________
>>> Libwebsockets mailing list
>>> Libwebsockets at ml.libwebsockets.org
>>> https://libwebsockets.org/mailman/listinfo/libwebsockets
>> _______________________________________________
>> Libwebsockets mailing list
>> Libwebsockets at ml.libwebsockets.org
>> https://libwebsockets.org/mailman/listinfo/libwebsockets
> 



More information about the Libwebsockets mailing list