[Libwebsockets] permessage-deflate

Andy Green andy at warmcat.com
Tue Jan 5 00:56:13 CET 2016


Hi -

The remaining tests in Autobahn rely on permessage-deflate, which was 
standardized a few weeks ago as RFC7692

https://tools.ietf.org/html/rfc7692

It's basically the same as the existing deflate-frame extension we have, 
but clearly the old extensions are dead (deflate-stream has been 
disabled from build for more than a year) or dying.

However the implementation of deflate-frame that we have takes the 
approach to insist to deflate and inflate whole messages at once, using 
malloc / realloc until it will fit.

Autobahn sends large messages in fragments and expects them to be 
streamed back... it's much more useful if the compression extension can 
handle the input and output as it comes without large buffers or limit 
on message size.

I have implemented the changes needed under the hood (only under he 
hood...) so that TX path extensions can chunk their output through the 
event loop.  And I implemented permessage-deflate here with RX chunking too.

It can work with Chrome and the test server + client, but it still fails 
about half the compression-related Autobahn tests.  Much of that is 
because I didn't implement the option parsing in RFC7692 yet, but some 
of them are corner cases in the chunking that still need fixing. 
Unfortunately these remaining problems occur depending on compression 
fragment size after tens of MB of logs from what Autobahn's sending.

So my current plan is not push this stuff at the moment, and make a test 
jig I can control better than Autobahn to be able to provoke all 
possible chunking / framing conditions at will, rather than when the 
data Autobahn sends randomly generates it.  After it hopefully passes 
that I will go back to Autobahn.  This stuff was very difficult 
originally, that is how come it ended up with the full message copout 
implementation, but this new implementation is much cleaner.

Unfortunately I am out of free time to work on this, I'll resume next 
weekend.

-Andy



More information about the Libwebsockets mailing list