[Libwebsockets] trouble with external POLL array handling

"Andy Green (林安廸)" andy at warmcat.com
Wed Jan 16 11:07:54 CET 2013


On 16/01/13 17:51, the mail apparently from Edwin van den Oetelaar included:
> Hello Andy (and others)
>
> I am a bit confused about the meaning of MAX_CLIENTS (which is 100 by default)
> The thing is that although there is space of 3201 items (MAX_CLIENTS *
> FD_HASHTABLE_MODULUS + 1) in the fds struct (inside the struct
> libwebsocket_context)
> I get refused connections in line 918 of libwebsockets.c
>
> 		if (context->fds_count >= MAX_CLIENTS) {
> 			lwsl_warn("too busy to accept new client\n");
> 			break;
> 		}
>
> My mental image of how this works is broken I think, put another way,
> how many concurrent sockets are supported by default and why? I think
> it could be 3200 but it looks like it is limited to 100 (or I am
> missing the point here)

I think it has gotten a bit confused over time, it's not just you.

Originally MAX_CLIENTS was the only limit and there was no hashtable. 
So the header did like struct pollfd fds[MAX_CLIENTS] and it was simple 
to track if we had too many connections.

With the hash arrays, the point that we can't accept another connection 
is when there is no more room in the hash bin array that the fd maps to.

struct libwebsocket_context {
	struct libwebsocket_fd_hashtable fd_hashtable[FD_HASHTABLE_MODULUS];
	struct pollfd fds[MAX_CLIENTS * FD_HASHTABLE_MODULUS + 1];

In theory the hash function should spread the fds evenly over the 
fd_hashtable array, but in fact at any given time it is possible only 
fds in one hash bin are left active.

Imagine there were only two hash bins, for 'odd' and 'even' fds.

There were lots of connections, all the connections with fds that went 
to the 'odd' bins closed, it keeps happening like that until the only 
active fds are all in the 'even' bin.  If that worst case happened, to 
maintain allowing MAX_CLIENTS, we have to allocate space for MAX_CLIENTS 
in each bin.

Then because that crazy worse case is unlikely, in practice we fill up 
the bins so the total number of sockets is much greater than 
MAX_CLIENTS, the true limit is running out of space in fds[] array.  So 
that gets allocated for another worst case, that all the hash bins got 
totally filled up.

So you're right the actual limit of client connections is

MAX_CLIENTS * FD_HASHTABLE_MODULUS

and the code is wrong to stop it at MAX_CLIENTS, or MAX_CLIENTS needs a 
new name.

Going back to extpoll it's wrong there too it needs to allocate the same 
size as fds[], ie, MAX_CLIENTS * FD_HASHTABLE_MODULUS

Does that make sense (not so much asking if you understood my wisdom 
than if what I am saying actually does make sense itself...)

-Andy

> thanks already,
> Edwin van den Oetelaar
>
> some info :
>
> Usages of Macro MAX_CLIENTS: [10 occurrences]
> libwebsockets
> extension-x-google-mux.h
>      63:  struct libwebsocket *wsi_muxconns[MAX_CLIENTS];
>      81:  struct libwebsocket *wsi_children[MAX_CLIENTS];
> libwebsockets.c
>      84:  if (context->fd_hashtable[h].length == MAX_CLIENTS - 1) {
>    918:  if (context->fds_count >= MAX_CLIENTS) {
> 1046:  if (context->fds_count >= MAX_CLIENTS) {
> 1770:  lwsl_info(" MAX_CLIENTS: %u\n", MAX_CLIENTS);
> private-libwebsockets.h
>    129:  #ifndef MAX_CLIENTS
>    130:  #define MAX_CLIENTS 100
>    286:  struct libwebsocket *wsi[MAX_CLIENTS + 1];
>    294:  struct pollfd fds[MAX_CLIENTS * FD_HASHTABLE_MODULUS + 1];
>
>
> On Wed, Jan 16, 2013 at 9:36 AM, Edwin van den Oetelaar
> <oetelaar.automatisering at gmail.com> wrote:
>> Wonderful how fast development is happening here!
>> I am testing your patches right now, but my system (gcc version 4.4.3
>> (Ubuntu 4.4.3-4ubuntu5.1)) does not like your patch.
>>
>> +/* some versions of gcc see a false positive here, workaround */
>> +#ifdef __GNUC__
>> +#pragma GCC diagnostic ignored "-Warray-bounds"
>> +#endif
>>
>> I get fails on compile :
>> test-server.c: In function ‘callback_http’:
>> test-server.c:257: error: #pragma GCC diagnostic not allowed inside functions
>> test-server.c:265: error: #pragma GCC diagnostic not allowed inside functions
>>
>> I can just remove it for now, but maybe this helps someone else.
>>
>> Great work, I like it!!
>> Edwin van den Oetelaar
>>




More information about the Libwebsockets mailing list