[Libwebsockets] WebSocket messages: when are they atomic?

"Andy Green (林安廸)" andy at warmcat.com
Sun May 12 04:35:18 CEST 2013

On 12/05/13 00:45, the mail apparently from Daniel Griscom included:
> Let me make sure I understand. In my application, I don't care much
> about either generating or receiving partial blocks in order to save
> memory, because my messages won't be that large (perhaps a few kB max).
> I just want to be absolutely sure that the messages I send (either way)
> are interpreted atomically.
> So, I have four cases to handle:
> 1) Sending from JavaScript in browser: just use the WebSocket instance's
> send() method. The whole text will be sent as a single message.

Yeah but I fear from what else you wrote, you didn't get that a "single 
message" may be composed of many fragments of any length.  In reality if 
your data is small, it'll go as a single fragment, but there's no 
standard for what "small" means.

> 2) Receiving in JavaScript in browser: each atomic message will be
> handed to the WebSocket's onmessage() handler as a single string

That is correct, but "atomic" is redundant.  Every message, even if it 
is made of many fragments, will be marshalled and presented as a single 
message in javascript when the last fragment comes.

> 3) Sending from libwebsockets in server: use libwebsocket_write() to
> send the whole message (but what if not all characters are sent? see
> below...)
> 4) Receiving in libwebsockets in server: use protocol callback routine
> when reason == LWS_CALLBACK_RECEIVE, calling
> libwebsockets_remaining_packet_payload() to make sure there aren't more
> characters coming in the message


> Did I get them all correctly?
> And, in case number 3, when might libwebsocket_write() not write all the
> characters? Only if the application as a whole is running out of memory?

No, it's an OS buffering issue.

If you call write() on a socket for 100MByte, the OS has to find 100MB 
to store your data and return back, sending it asynchronously. 
Sometimes, it judges it doesn't want to reserve the amount of memory 
you're asking, considering other memory consumers in the kernel than the 
network stack.  You don't need to try crazy sizes like 100MB to trigger 
it either.

So when the OS is under memory pressure, it may signal that the pipe is 
not choked, that you can send another packet, but only accept some of 
what you're offering.  It's a function of the size of the packet you 
want to send and the OS memory situation (and perhaps some dynamic 
assessment of how much more data to put in flight in that connection 
given its recent performance, I don't know).

The smallest accepted amount I saw was 2.8KBytes on this box.

I've since added code that forces the send buffer to be reserved in the 
OS to be at least the size of the max rx buffer set in the protocol, 
that should help.

> And, if it does bail early, are the sent characters packaged as a single
> message, or may I append more characters to the message with an
> additional call?

No it'll close the connection, since the library can't cope with that 
situation at the moment.

The best solution is don't send fragments larger than 2K, if you have a 
larger message than one fragment use the CONTINUATION / FIN control 
stuff to send as large a message as you like in 2K fragments.


> At 9:05 AM +0800 5/7/13, Andy Green (ó-à¿úN) wrote:
>> On 07/05/13 08:57, the mail apparently from Daniel Griscom included:
>>> I'm getting up to speed on libwebsockets and WebSockets, to be used for
>>> communication between a browser and the web host (vanilla installation:
>>> no extensions). I'm designing the protocol which will be used across the
>>> WebSocket, and it would be great if the messages sent back and forth
>>> were atomic.
>>> I see that the struct libwebsocket_protocols has a rx_buffer_size; if I
>>> set this to the largest message that will be sent by a client then will
>>> each client message arrive in a single callback?
>>> And, in the other direction, how will libwebsocket_write() handle large
>>> messages? Can I make sure that messages below some size limit will
>>> arrive in a single JavaScript onmessage call?
>> With websockets even massive messages can be logically atomic, and its
>> the websocket message as a unit that gets given to Javascript.
>> However it doesn't mean it's one network packet or needs to exist in
>> memory all at one time at the sender (or the receiver, if it's not a
>> browser).
>> Normally for small messages you just send a buffer in memory in one go
>> and it is done.  One part of the message goes out with its FIN bit set
>> indicating it's also the last part of the message.
>> If the messages start to get large compared to the buffer for a
>> network packet, you need to cut the message into several parts.  Only
>> the last part has its FIN bit set and after the initial message, the
>> other parts have a CONTINUATION opcode.
>> You can deal with multi-part messages all in the libwebsocket_write()
>> call if you're the sender. Have a look at the test-fraggle.c example,
>> -Andy

More information about the Libwebsockets mailing list