[Libwebsockets] Truncated send handled by user not working as expected

Andy Green andy at warmcat.com
Mon Aug 18 11:19:50 CEST 2014

On 18 August 2014 17:00:17 GMT+08:00, Roger Light <roger at atchoo.org> wrote:
>>>libwebsockets was handling truncates and consequent calls to fail
>>>assertion at line 112.
>> I don't see why it fails that assertion though, did you understand
>why during
>> your debugging?  Not having sent anything and buffer the whole thing
>> have been okay.
>I believe I could trigger the assert easily on Linux by calling
>libwebsocket_write() without checking lws_send_pipe_choked() first. I
>was checking that the socket was available to write, then attempting
>to write as much as possible until it blocked. So crudely, and iirc:
>    /* Gives assert */
>    while(have_data){
>        count = libwebsockets_write(len);
>        if(count < 0) return 0;
>        if(wlen != len) break;
>    }

Yeah you can't do that.  Say you send a lot each time like 1MByte.

The kernel won't accept that much for buffering, so a partial send is guaranteed.

Once lws took on dealing with your partial send, flushing the buffered remainder has to take priority over everything else, since it has to get sent first before anything else.

When there was a partial send, a writable callback is scheduled automatically, and before the user callback is notified, lws checks to see if there is a pending partial send.
If so, it will send as much of the pending buffer as the kernel will take.  If it was everything, then the partial send is marked as emptied out and the next writeable callback will get passed through to the user code again to start sending something new.

What your asserting code does is ignore all that and just sit looping, blocking and spamming more new stuff into libwebsockets_write() every time without going around the service loop and let it dump what it buffered.

On the other hand -->

>    /* No assert */
>    while(have_data && !lws_send_pipe_choked()){
>        count = libwebsockets_write(len);
>        if(count < 0) return 0;
>        if(wlen != len) break;
>    }

This makes sense because the test for if we can write means that the send() libwebsockets_write() eventually does never returns -EWOULDBLOCK or whatever, because we already checked he can actually send.

And in the event he did a partial send the pipe is choked and he stops spamming new data and goes back into the service loop where he can dump his buffer of remaining data from the partial send, before coming back here when that lws partial send buffer is empty.

>I've been meaning to send an email about it but hadn't got round to
>it, sorry. I've just done some quick testing haven't been able to
>reproduce it, but I do remember that the problem occurred more
>frequently on my slower box that I was connecting to over wifi,
>compared to the powerful box I'm using now and connecting to
>localhost. I can jump back through history to try and isolate the
>problem if necessary.

If you still have problems with the second example, we should try to understand what's going on. But the first example won't fly because he doesn't let lws have the service calls necessary to transparently empty the partial send buffer.



More information about the Libwebsockets mailing list