[Libwebsockets] trouble with external POLL array handling

Edwin van den Oetelaar oetelaar.automatisering at gmail.com
Thu Jan 17 21:14:59 CET 2013


Isn't this a lot of fun :-)

I got the latest patches and built the library again, some things did
not compile on my system.
Nothing major, just added some checking of return values.

cc1: warnings being treated as errors
daemonize.c: In function ‘child_handler’:
daemonize.c:49: error: ignoring return value of ‘write’, declared with
attribute warn_unused_result
daemonize.c: In function ‘lws_daemonize’:
daemonize.c:140: error: ignoring return value of ‘freopen’, declared
with attribute warn_unused_result
daemonize.c:141: error: ignoring return value of ‘freopen’, declared
with attribute warn_unused_result
daemonize.c:142: error: ignoring return value of ‘freopen’, declared
with attribute warn_unused_result
make[1]: *** [libwebsockets_la-daemonize.lo] Error 1


Here is the patch.




diff --git a/lib/daemonize.c b/lib/daemonize.c
index 1d0c800..10f90c7 100644
--- a/lib/daemonize.c
+++ b/lib/daemonize.c
@@ -24,10 +24,12 @@
 static int pid_daemon;
 static char lock_path[PATH_MAX];

-static void child_handler(int signum)
+static void
+child_handler(int signum)
 {
        int fd;
        char sz[20];
+    int len;

        switch (signum) {

@@ -46,7 +48,14 @@ static void child_handler(int signum)
                        exit(1);
                }
                sprintf(sz, "%u", pid_daemon);
-               write(fd, sz, strlen(sz));
+        len = write(fd, sz, strlen(sz));
+        if (strlen(sz) != len) {
+            fprintf(stderr, "unable write pid to lock"
+                    " file %s, code=%d (%s)",
+                    lock_path, errno, strerror(errno));
+            close(fd);
+            exit(1);
+        }
                close(fd);
                exit(0);

@@ -64,10 +73,11 @@ static void child_handler(int signum)
  * The process context you called from has been terminated then.
  */

-int lws_daemonize(const char *_lock_path)
+int
+lws_daemonize(const char *_lock_path)
 {
        pid_t sid, parent;
-
+    FILE *f;
        /* already a daemon */
        if (getppid() == 1)
                return (1);
@@ -137,9 +147,20 @@ int lws_daemonize(const char *_lock_path)
        }

        /* Redirect standard files to /dev/null */
-       freopen("/dev/null", "r", stdin);
-       freopen("/dev/null", "w", stdout);
-       freopen("/dev/null", "w", stderr);
+    f = freopen("/dev/null", "r", stdin);
+    if (NULL == f)
+        fprintf(stderr, "unable to freopen() stdin, code %d (%s)",
+                errno, strerror(errno));
+
+    f = freopen("/dev/null", "w", stdout);
+    if (NULL == f)
+        fprintf(stderr, "unable to freopen() stdout, code %d (%s)",
+                errno, strerror(errno));
+
+    f = freopen("/dev/null", "w", stderr);
+    if (NULL == f)
+        fprintf(stderr, "unable to freopen() stderr, code %d (%s)",
+                errno, strerror(errno));

        /* Tell the parent process that we are A-okay */
        kill(parent, SIGUSR1);


On Thu, Jan 17, 2013 at 12:51 PM, "Andy Green (林安廸)" <andy at warmcat.com> wrote:
> On 17/01/13 16:01, the mail apparently from Edwin van den Oetelaar included:
>
>>> You are right a lookup table will do fine, and it blows through the
>>> complexity and remaining iterations in the hash scheme.  On a really
>>> low-end
>>> ARM or MIPS box with only 4 or 8MB of memory though it's quite expensive
>>> to
>>> reserve 128KBytes for this if he only expects one client at a time.
>>>
>>> I have been thinking for a while that MAX_CLIENTS needs to become
>>> something
>>> set at runtime by the app if it doesn't like the default (what should we
>>> tell a distro packager to set MAX_CLIENTS to otherwise?), your scheme
>>> looks
>>> like a good way to solve that and can also replace the library's internal
>>> polling hash tables we discussed yesterday are also bloated, statically
>>> allocated and complex.  We should get a nice throughput increase there
>>> too
>>> under heavy load.
>>>
>>> So I will look at adapting (the second version of) this today both in and
>>> out of the library and in the in-library case, runtime control over the
>>> allocation for it.
>>>
>>> Thanks a lot for studying it!
>>>
>>> -Andy
>
>
> First I committed your changes to the test server here, after some minor
> meddling
>
> http://git.libwebsockets.org/cgi-bin/cgit/libwebsockets/commit/?id=6c720c44403c9d09a8a0b28ec95f418456ec15fe
>
>
>> Some thoughts I have.
>> The size of the lookup table must be at least the largest number of FD
>> that the whole application expects.
>> So if program has 1000 files open but just 2 sockets you would still
>> need at least 1002 entries in the lookup table, since fds are recycled
>> by the OS.
>> If a lookup table was to be used for quickly finding the fd -> wsi
>> mapping, I think the memory should be allocated by the user code and
>> passed to the library.
>
>
> Understood... when I implemented it thismorning I realized the scope of the
> "fd space" is not only smaller than your example of 30K typically (it's 1024
> on my Fedora box) but the user can control it through ulimit.  By having the
> server run as a "websocket" user or such he can also control just what the
> server sees as the limit.
>
> So I took the approach to get rid of MAX_CLIENTS and any other
> configure-time decision and always malloc the tables to match
> getdtablesize() seen at init.  Then the user can forget about it and just
> set ulimit as he would have to do anyway to get more file descriptors
> available, and the library will adapt.
>
> http://git.libwebsockets.org/cgi-bin/cgit/libwebsockets/commit/?id=dfb23041684e1c3bf01b869f09e5b88c7ad5931c
>
> Overall that patch removed 100 lines and made things much nicer in that
> area, thanks a lot for the work you did on the scheme.
>
>
>> My possible use case would be to run this library on linux, for now it
>> is just an idea but my goal is to write a little server that connects
>> 200k devices with low traffic, low memory, low CPU,  low latency using
>> websockets, to create the Internet of Things.
>
>
> Yes the reason I made the library was exactly with an embedded server in
> mind.  I'll do more debloating shortly for that case, like --without-client.
>
>
>> If I can get that up and running quick I have a potential client/use-case.
>> I do like python and all those fancy websocket-servers but this 'C'
>> library can beat all of them. I am also mixing the poll loop with
>> zero-mq and that brings all kinds of nice things.
>>
>> Maybe the handling of server/client could take into account that a
>> server might have the memory for lookup tables but the client might
>> not?
>
>
> Even a small client will be OK with lookup tables for file descriptors if he
> won't hold many open files / sockets.  For now I think it's OK to control it
> by ulimit... the main thing to enable smaller clients will be
> --without-server that doesn't exist yet.
>
> Actually most hours today went on finally getting rx flow control for the
> server to work correctly, that's critical for server situations where it's
> passing on to clients who cannot keep up with an incoming burst
>
> http://git.libwebsockets.org/cgi-bin/cgit/libwebsockets/commit/?id=706961dbb5b962d24a3e6e26d8053976b14bb026
>
> -Andy
>



More information about the Libwebsockets mailing list