[Libwebsockets] how is partial write handled

"Andy Green (林安廸)" andy at warmcat.com
Fri Oct 18 13:08:17 CEST 2013

On 18/10/13 07:38, the mail apparently from "Andy Green (林安廸)" included:
> On 18/10/13 06:59, the mail apparently from LANGLOIS Olivier PIS -EXT
> included:
>> Andy,
>> I think that I understand a little bit better what happens.
>> Remember when I said the way you describe SO_SNDBUF sounded more like
>> I did remember that the library also disable Nagle with TCP_NODELAY.
>> In a LAN, what this combination will do is make a TCP connection
>> approximate a UDP socket in a LAN because RTT is very low. On the
>> Internet with RTT > 200 ms and with an intensive bandwidth app (ie:
>> video streaming on websockets:
>> http://phoboslab.org/log/2013/09/html5-live-video-streaming-via-websockets
>> ), we will get partial writes even with appropriate SO_SNDBUF and
> I don't really like the stability of my explanations for this problem...
> the fact it keeps coming up and I'm having to use words like "belief"
> probably means something is wrong.  However it remains true I can't
> reproduce the issue with the current setup.
> I started on a patch which will either handle truncated send internally
> using a malloc'd buffer for the missing part, and / or let the user code
> know accurately that less of his buffer was consumed than he sent.  I'll
> provide this on a test branch later... I can't reproduce the situation
> to test it.

I added a patched aimed at comprehensively handling partial sends.  It's 
not tested due to problems reproducing but it should be no worse than 
what was there before this patch (and I still believe for packets below 
the SO_SNDBUF reservation this may never occur)



This patch adds code to handle the situation that a prepared user buffer 
could not all be sent on the
socket at once.  There are two kinds of situation to handle

1) User code handles it: The connection only has extensions active that 
do not rewrite the buffer.
In this case, the patch caused libwebsocket_write() to simply return the 
amount of user buffer that
was consumed (this is specifically the amount of user buffer used in 
sending what was accepted,
nothing else).  So user code can just advance its buffer that much and 
resume sending when the socket
is writable again.  This continues the frame rather than starting a new 
one or new fragment.

2) The connections has extensions active which actually send something 
quite different than what the
user buffer contains, for example a compression extension.  In this 
case, libwebsockets will dynamically
malloc a buffer to contain a copy of the remaining unsent data, request 
notifiction when writeable again,
and automatically spill and free this buffer with the highest priority 
before passing on the writable
notification to anything else.  For this situation, the call to write 
will return that it used the
whole user buffer, even though part is still rebuffered.

This patch should enable libwebsockets to detect the two cases and take 
the appropriate action.

There are also two choices for user code to deal with partial sends.

1) Leave the no_buffer_all_partial_tx member in the protocol struct at 
zero.  The library will dyamically
buffer anything you send that did not get completely written to the 
socket, and automatically spill it next
time the socket is writable.  You can use this method if your sent 
frames are relatvely small and unlikely to get
truncated anyway.

2) Set the no_buffer_all_partial_tx member in the protocol struct.  User 
code now needs to take care of the
return value from libwebsocket_write() and deal with resending the 
remainder if not all of the requested amount
got sent.  You should use this method if you are sending large messages 
and want to maximize throughput and efficiency.

Since the new member no_buffer_all_partial_tx will be zero by default, 
this patch will auto-rebuffer any
partial sends by default.  That's good for most cases but if you attempt 
to send large blocks, make sure you
follow option 2) above.



>>>> Can I use your library if I cannot predict ahead of time the
>>>> length of my messages because their length is variable?
>>> Yes... websockets itself defines fragments for that purpose.
>> I have read the readme file describing fragments. My understanding
>> was that this was supported for the receiving side. Does the library
>> support fragments on the sending side as well?
> Yes, have a look at the fraggle test app, it defines a test client and
> server that tests fragmentation reliability on both sides for lws by
> spamming random size messages made from a random number of fragments of
> random size between the client and server.
> The code for that shows how to deal with fragments.
> -Andy
>> ________________________________ CONFIDENTIALITY : This e-mail and
>> any attachments are confidential and may be privileged. If you are
>> not a named recipient, please notify the sender immediately and do
>> not disclose the contents to another person, use it for any purpose
>> or store or copy the information in any medium.

More information about the Libwebsockets mailing list