[Libwebsockets] I may find a bug on lws_write behaviour

Andy Green andy at warmcat.com
Fri Feb 28 13:34:46 CET 2020

On 2/28/20 11:49 AM, 雷 鸣 wrote:
> 在 Fri, 28 Feb 2020 16:40:52 +0800,Andy Green <andy at warmcat.com> 写道:
> Thank you for your explanation. I have maintained the 1:1 relationship
> in my code and it works.
> May I have a suggestion? It may be better if the behavior for additional
> writable callback could be appended to the comment/documentation of
> lws_write.
> Another aspect, is there any plan to offer API that will return sent
> bytes instead of buffering them? It looks like a solution to the
> problem.
> I have another problem that LWS_MAX_SMP defined to 1 on Windows and seems
> to be on purpose according to cmakelists. Is it a temporary limit?

All of the (optional) stuff in lws around forking or threads is done and 
tested using unix vfork / fork() and pthreads.  If windows supports that 
it might just work (I know it doesn't have fork(), but it might have 
pthreads).  I have no use for windows so other than trying to keep it 
building for users who are using it, windows support relies on windows 
users to contribute whatever is needed.

If you're interested in this and want to make it work I'll try to help.


>> On 2/28/20 7:50 AM, 雷 鸣 wrote:
>>> Hello!
>>>   According to the comment of function 'lws_write' I found it says that
>>>      * The OS may not accept everything you asked to write on the
>>> connection.
>>>     *
>>>     * Posix defines POLLOUT indication from poll() to show that the
>>> connection
>>>     * will accept more write data, but it doesn't specifiy how much.  It
>>> may
>>> just
>>>     * accept one byte of whatever you wanted to send.
>>>     *
>>>     * LWS will buffer the remainder automatically, and send it out
>>> autonomously.
>>>     *
>>>     * During that time, WRITABLE callbacks will be suppressed.
>>>   But I found the callback comes when last output buffer block is sent.
>>>   I build libwebsockets with libuv and mbedtls in Windows 7 x64, VS2019
>>> by
>>> the
>>>      arguments of
>>>        -DLWS_WITH_LWSWS=ON
>>>        -DLWS_WITH_LIBUV=ON
>>>   I have been with this library for three days in learning how to build a
>>> server.
>>>      I try limiting the transfer speed to simulate FLV live streaming,
>>> using
>>>      lws_set_timer_usecs to control the send-flow.
>>>   I found it works with lws_write 1024 bytes per call, but runs into
>>> wrong if
>>>      I did it with 65536 bytes per call.
>>>   I traced the program flow and found it  ALWAYS request another
>>> writeable
>>>      notify (core-net/output.c, line 178) regardless buffered_out is just
>>>      consumed up.
>> It's true... its problem is while it is concealing writeable callbacks
>> and sending the oversize data in the background, lws must request by
>> itself writeable callbacks on that connection, and handle them.
>> It does not know then if user code has requsted its own writeable
>> callback that it has trashed and lost by doing the background work.  For
>> that reason, as you say, it always makes a writeable callback request
>> itself at the end of internally handling them.
>>> I doubt that it should check buffered_out again before requesting
>>> writable
>>>      notify here. And user may or may not request a writeable notify
>>> after
>>>      calling lws_write with large block of data.
>>>   Could you staff have a look at the logic whether works as expected?
>> It is as expected... there is no guarantee you will get callback
>> requests that you asked for in a 1:1 relationship.  You can get extra
>> ones and additional requests after the first pending one will be ignored
>> until after the callback happened.
>> For that reason you have to consult your "pss" per-connection state and
>> find out what, if anything, you wanted to do with that callback.  If
>> nothing, you can just return 0.
>> It's better to size what you send to what the kernel will accept in one
>> go when it tries to send it.  TLS has practical a limit of 16KB record
>> buffer anyway, but once you filled the tcp window, the kernel may only
>> accept 1 or 2 x mtu (1.5 .. 3KB... but is platform dependent) each time
>> depending on connection characteristics.  There's no point "sending"
>> something bigger than this then, it's just taking up heap having to hold
>> it for you while lws hands it out in small chunks the background same as
>> you would have done yourself.
>> -Andy

More information about the Libwebsockets mailing list