[Libwebsockets] WebSocket messages: when are they atomic?
"Andy Green (林安廸)"
andy at warmcat.com
Sun May 12 04:35:18 CEST 2013
On 12/05/13 00:45, the mail apparently from Daniel Griscom included:
> Let me make sure I understand. In my application, I don't care much
> about either generating or receiving partial blocks in order to save
> memory, because my messages won't be that large (perhaps a few kB max).
> I just want to be absolutely sure that the messages I send (either way)
> are interpreted atomically.
> So, I have four cases to handle:
> send() method. The whole text will be sent as a single message.
Yeah but I fear from what else you wrote, you didn't get that a "single
message" may be composed of many fragments of any length. In reality if
your data is small, it'll go as a single fragment, but there's no
standard for what "small" means.
> handed to the WebSocket's onmessage() handler as a single string
That is correct, but "atomic" is redundant. Every message, even if it
is made of many fragments, will be marshalled and presented as a single
> 3) Sending from libwebsockets in server: use libwebsocket_write() to
> send the whole message (but what if not all characters are sent? see
> 4) Receiving in libwebsockets in server: use protocol callback routine
> when reason == LWS_CALLBACK_RECEIVE, calling
> libwebsockets_remaining_packet_payload() to make sure there aren't more
> characters coming in the message
> Did I get them all correctly?
> And, in case number 3, when might libwebsocket_write() not write all the
> characters? Only if the application as a whole is running out of memory?
No, it's an OS buffering issue.
If you call write() on a socket for 100MByte, the OS has to find 100MB
to store your data and return back, sending it asynchronously.
Sometimes, it judges it doesn't want to reserve the amount of memory
you're asking, considering other memory consumers in the kernel than the
network stack. You don't need to try crazy sizes like 100MB to trigger
So when the OS is under memory pressure, it may signal that the pipe is
not choked, that you can send another packet, but only accept some of
what you're offering. It's a function of the size of the packet you
want to send and the OS memory situation (and perhaps some dynamic
assessment of how much more data to put in flight in that connection
given its recent performance, I don't know).
The smallest accepted amount I saw was 2.8KBytes on this box.
I've since added code that forces the send buffer to be reserved in the
OS to be at least the size of the max rx buffer set in the protocol,
that should help.
> And, if it does bail early, are the sent characters packaged as a single
> message, or may I append more characters to the message with an
> additional call?
No it'll close the connection, since the library can't cope with that
situation at the moment.
The best solution is don't send fragments larger than 2K, if you have a
larger message than one fragment use the CONTINUATION / FIN control
stuff to send as large a message as you like in 2K fragments.
> At 9:05 AM +0800 5/7/13, Andy Green (ó-à¿úN) wrote:
>> On 07/05/13 08:57, the mail apparently from Daniel Griscom included:
>>> I'm getting up to speed on libwebsockets and WebSockets, to be used for
>>> communication between a browser and the web host (vanilla installation:
>>> no extensions). I'm designing the protocol which will be used across the
>>> WebSocket, and it would be great if the messages sent back and forth
>>> were atomic.
>>> I see that the struct libwebsocket_protocols has a rx_buffer_size; if I
>>> set this to the largest message that will be sent by a client then will
>>> each client message arrive in a single callback?
>>> And, in the other direction, how will libwebsocket_write() handle large
>>> messages? Can I make sure that messages below some size limit will
>> With websockets even massive messages can be logically atomic, and its
>> However it doesn't mean it's one network packet or needs to exist in
>> memory all at one time at the sender (or the receiver, if it's not a
>> Normally for small messages you just send a buffer in memory in one go
>> and it is done. One part of the message goes out with its FIN bit set
>> indicating it's also the last part of the message.
>> If the messages start to get large compared to the buffer for a
>> network packet, you need to cut the message into several parts. Only
>> the last part has its FIN bit set and after the initial message, the
>> other parts have a CONTINUATION opcode.
>> You can deal with multi-part messages all in the libwebsocket_write()
>> call if you're the sender. Have a look at the test-fraggle.c example,
>> for LWS_WRITE_NO_FIN and LWS_WRITE_CONTINUATION.
More information about the Libwebsockets