{"schema":"libjg2-1", "vpath":"/git/", "avatar":"/git/avatar/", "alang":"", "gen_ut":1752652521, "reponame":"libwebsockets", "desc":"libwebsockets lightweight C networking library", "owner": { "name": "Andy Green", "email": "andy@warmcat.com", "md5": "c50933ca2aa61e0fe2c43d46bb6b59cb" },"url":"https://libwebsockets.org/repo/libwebsockets", "f":3, "items": [ {"schema":"libjg2-1", "cid":"058011fe7c6829ffaa5714b20f644f22", "commit": {"type":"commit", "time": 1570964159, "time_ofs": 60, "oid_tree": { "oid": "ab44cd04ff051449a1cb36f95ccff2c13b8d573f", "alias": []}, "oid":{ "oid": "efc35fe1e1af9d668e6d4b487b1d3ae7f938bfd7", "alias": []}, "msg": "async dns: recursion", "sig_commit": { "git_time": { "time": 1570964159, "offset": 60 }, "name": "Andy Green", "email": "andy@warmcat.com", "md5": "c50933ca2aa61e0fe2c43d46bb6b59cb" }, "sig_author": { "git_time": { "time": 1570964159, "offset": 60 }, "name": "Andy Green", "email": "andy@warmcat.com", "md5": "c50933ca2aa61e0fe2c43d46bb6b59cb" }}, "body": "async dns: recursion\n\nHandle the situation that we are told to use a CNAME, but the CNAME is not resolved\nby the remote server... adapt the query to resolve the CNAME and restart it, while\nretaining the original query name for the cache entry generation.\n\n\u0022Recursion\u0022 doesn't mean function-calling-a-function type recursion, it remains\ncompletely asynchronous on the event loop." , "diff": "diff --git a/README.md b/README.md\nindex 94e1c2f..e58e6cc 100644\n--- a/README.md\n+++ b/README.md\n@@ -91,11 +91,11 @@ lws will generate PINGs and take PONGs as the indication of validity.\n \n ## `lws_system`: Async DNS support\n \n-Master now provides optional Asynchronous (ie, nonblocking) DNS resolving. Enable\n-with `-DLWS_WITH_SYS_ASYNC_DNS\u003d1` at cmake. This provides a quite sophisticated\n-ipv4 + ipv6 capable resolver that autodetects the dns server on several platforms\n-and operates a UDP socket to its port 53 to produce and parse DNS packets\n-from the event loop. And of course, it's extremely compact.\n+Master now provides optional Asynchronous (ie, nonblocking) recursive DNS resolving.\n+Enable with `-DLWS_WITH_SYS_ASYNC_DNS\u003d1` at cmake. This provides a quite\n+sophisticated ipv4 + ipv6 capable resolver that autodetects the dns server on\n+several platforms and operates a UDP socket to its port 53 to produce and parse DNS\n+packets from the event loop. And of course, it's extremely compact.\n \n It broadly follows the getaddrinfo style api, but instead of creating the results\n on the heap for each caller, it caches a single result according to the TTL and\ndiff --git a/READMEs/README.async-dns.md b/READMEs/README.async-dns.md\nindex 7defcae..64cbf34 100644\n--- a/READMEs/README.async-dns.md\n+++ b/READMEs/README.async-dns.md\n@@ -2,7 +2,7 @@\n \n ## Introduction\n \n-Lws now features optional asynchronous, ie, nonblocking DNS\n+Lws now features optional asynchronous, ie, nonblocking recursive DNS\n resolution done on the event loop, enable `-DLWS_WITH_SYS_ASYNC_DNS\u003d1`\n at cmake to build it in.\n \n@@ -35,6 +35,8 @@ Other features\n a list on the first request, not spawn multiple requests)\n - observes TTL in cache\n - TTL and timeout use `lws_sul` timers on the event loop\n+ - Uses CNAME resolution inside the same response if present, otherwise\n+ recurses to resolve the CNAME (up to 3 deep)\n - ipv6 pieces only built if cmake `LWS_IPV6` enabled\n \n ## Api\n@@ -74,3 +76,25 @@ first cache entry is extended into the second one. At the time the\n second result arrives, the query is destroyed and the cached results\n provided on the result callback.\n \n+## Recursion\n+\n+Where CNAMEs are returned, DNS servers may take two approaches... if the\n+CNAME is also resolved by the same server and so it knows what it should\n+resolve to, it may provide the CNAME resolution in the same response\n+packet.\n+\n+In the case the CNAME is actually resolved by a different name server,\n+the server with the CNAME does not have the information to hand to also\n+resolve the CNAME in the same response. So it just leaves it for the\n+client to sort out.\n+\n+The lws implementation can deal with both of these, first it \u0022recurses\u0022\n+(it does not recurse on the process stack but uses its own manual stack)\n+to look for results in the same packet that told it about the CNAME. If\n+there are no results, it resets the query to look instead for the CNAME,\n+and restarts it. It allows this to happen for 3 CNAME deep.\n+\n+At the end, either way, the cached result is set using the original\n+query name and the results from the last CNAME in the chain.\n+\n+\ndiff --git a/lib/system/async-dns/async-dns-parse.c b/lib/system/async-dns/async-dns-parse.c\nindex a796690..bbfed34 100644\n--- a/lib/system/async-dns/async-dns-parse.c\n+++ b/lib/system/async-dns/async-dns-parse.c\n@@ -71,12 +71,18 @@ again1:\n \t\tpointer \u003d 1;\n \t}\n \n-again:\n \tif (ls \u003e\u003d e)\n \t\treturn -1;\n+\n \tll \u003d *ls++;\n-\tif (ls + ll + 4 \u003e e || ll \u003e budget) {\n-\t\tlwsl_notice(\u0022%s: label len invalid\u005cn\u0022, __func__);\n+\tif (ls + ll + 1 \u003e e) {\n+\t\tlwsl_notice(\u0022%s: label len invalid, %ld vs %ld\u005cn\u0022, __func__,\n+\t\t\t\t(ls + ll + 1) - pkt, e - pkt);\n+\n+\t\treturn -1;\n+\t}\n+\tif (ll \u003e budget) {\n+\t\tlwsl_notice(\u0022%s: label too long %d vs %d\u005cn\u0022, __func__, ll, budget);\n \n \t\treturn -1;\n \t}\n@@ -97,7 +103,7 @@ again:\n \n \tif (pointer) {\n \t\tif (*ls)\n-\t\t\tgoto again;\n+\t\t\tgoto again1;\n \n \t\t/*\n \t\t * special fun rule... if whole qname was a pointer label,\n@@ -126,7 +132,7 @@ typedef int (*lws_async_dns_find_t)(const char *name, void *opaque,\n /* locally query the response packet */\n \n struct label_stack {\n-\tchar name[64];\n+\tchar name[DNS_MAX];\n \tint enl;\n \tconst uint8_t *p;\n };\n@@ -143,18 +149,18 @@ struct label_stack {\n */\n \n static int\n-lws_adns_iterate(const uint8_t *pkt, int len, const char *expname,\n-\t\t lws_async_dns_find_t cb, void *opaque)\n+lws_adns_iterate(lws_adns_q_t *q, const uint8_t *pkt, int len,\n+\t\t const char *expname, lws_async_dns_find_t cb, void *opaque)\n {\n \tconst uint8_t *e \u003d pkt + len, *p, *pay;\n \tstruct label_stack stack[4];\n+\tint n \u003d 0, stp \u003d 0, ansc, m;\n \tuint16_t rrtype, rrpaylen;\n-\tint n \u003d 0, stp \u003d 0, ansc;\n \tchar *sp, inq;\n \tuint32_t ttl;\n \n-\tlws_strncpy(stack[stp].name, expname, sizeof(stack[stp].name));\n-\tstack[stp].enl \u003d strlen(expname);\n+\tlws_strncpy(stack[0].name, expname, sizeof(stack[0].name));\n+\tstack[0].enl \u003d strlen(expname);\n \n start:\n \tansc \u003d lws_ser_ru16be(pkt + DHO_NANSWERS);\n@@ -168,7 +174,7 @@ start:\n \t * query class\n \t */\n \n-resume:\n+\n \twhile (p + 14 \u003c e \u0026\u0026 (inq || ansc)) {\n \n \t\tif (!inq \u0026\u0026 !stp)\n@@ -215,7 +221,7 @@ resume:\n \t\t */\n \n \t\tif (lws_ser_ru16be(\u0026p[2]) !\u003d 1) {\n-\t\t\tlwsl_debug(\u0022%s: non-IN response 0x%x\u005cn\u0022, __func__,\n+\t\t\tlwsl_err(\u0022%s: non-IN response 0x%x\u005cn\u0022, __func__,\n \t\t\t\t\t\tlws_ser_ru16be(\u0026p[2]));\n \n \t\t\treturn -1;\n@@ -251,9 +257,13 @@ resume:\n \t\tif (stack[0].name[n - 1] \u003d\u003d '.')\n \t\t\tn--;\n \n-\t\tif (n \u003c 1 || n !\u003d stack[stp].enl ||\n-\t\t strcmp(stack[0].name, stack[stp].name)) {\n-\t\t\tlwsl_debug(\u0022%s: skipping %s vs %s\u005cn\u0022, __func__,\n+\t\tm \u003d stack[stp].enl;\n+\t\tif (stack[stp].name[m - 1] \u003d\u003d '.')\n+\t\t\tm--;\n+\n+\t\tif (n \u003c 1 || n !\u003d m ||\n+\t\t strncmp(stack[0].name, stack[stp].name, n)) {\n+\t\t\tlwsl_notice(\u0022%s: skipping %s vs %s\u005cn\u0022, __func__,\n \t\t\t\t\tstack[0].name, stack[stp].name);\n \t\t\tgoto skip;\n \t\t}\n@@ -275,14 +285,18 @@ resume:\n \t\t */\n \n \t\tswitch (rrtype) {\n-#if defined(LWS_WITH_IPV6)\n+\n \t\tcase LWS_ADNS_RECORD_AAAA:\n \t\t\tif (rrpaylen !\u003d 16) {\n \t\t\t\tlwsl_err(\u0022%s: unexpected rrpaylen\u005cn\u0022, __func__);\n \t\t\t\treturn -1;\n \t\t\t}\n+#if defined(LWS_WITH_IPV6)\n \t\t\tgoto do_cb;\n+#else\n+\t\t\tbreak;\n #endif\n+\n \t\tcase LWS_ADNS_RECORD_A:\n \t\t\tif (rrpaylen !\u003d 4) {\n \t\t\t\tlwsl_err(\u0022%s: unexpected rrpaylen4\u005cn\u0022, __func__);\n@@ -297,7 +311,7 @@ do_cb:\n \n \t\tcase LWS_ADNS_RECORD_CNAME:\n \t\t\t/*\n-\t\t\t * The name the CNAME refers to should itself be\n+\t\t\t * The name the CNAME refers to MAY itself be\n \t\t\t * included elsewhere in the response packet.\n \t\t\t *\n \t\t\t * So switch tack, stack where to resume from and\n@@ -314,6 +328,7 @@ do_cb:\n \t\t\t\treturn -1;\n \t\t\t}\n \t\t\tsp \u003d stack[stp].name;\n+\t\t\t/* get the cname alias */\n \t\t\tn \u003d lws_adns_parse_label(pkt, len, p, rrpaylen, \u0026sp,\n \t\t\t\t\t\t sizeof(stack[stp].name) -\n \t\t\t\t\t\t lws_ptr_diff(sp, stack[stp].name));\n@@ -329,11 +344,14 @@ do_cb:\n \t\t\t/* it should have exactly reached rrpaylen */\n \n \t\t\tif (p !\u003d pay + rrpaylen) {\n-\t\t\t\tlwsl_err(\u0022%s: cname name bad len\u005cn\u0022, __func__);\n+\t\t\t\tlwsl_err(\u0022%s: cname name bad len %d\u005cn\u0022, __func__, rrpaylen);\n \n \t\t\t\treturn -1;\n \t\t\t}\n \n+\t\t\tlwsl_info(\u0022%s: recursing looking for %s\u005cn\u0022, __func__,\n+\t\t\t\t\tstack[stp].name);\n+\n \t\t\tstack[stp].enl \u003d lws_ptr_diff(sp, stack[stp].name);\n \t\t\t/* when we unstack, resume from here */\n \t\t\tstack[stp].p \u003d pay + rrpaylen;\n@@ -350,16 +368,61 @@ skip:\n \tif (!stp)\n \t\treturn 1; /* we didn't find anything, but we didn't error */\n \n+\tlwsl_info(\u0022%s: '%s' -\u003e CNAME '%s' resolution not provided, recursing\u005cn\u0022,\n+\t\t\t__func__, ((const char *)\u0026q[1]) + DNS_MAX,\n+\t\t\tstack[stp].name);\n+\n \t/*\n \t * This implies there wasn't any usable definition for the\n-\t * CNAME in the end, eg, only AAAA when we needed and A.\n+\t * CNAME in the end, eg, only AAAA when we needed an A.\n \t *\n-\t * Short-circuit the whole stack and resume from after the\n-\t * original CNAME reference.\n+\t * It's also legit if the DNS just returns the CNAME, and that server\n+\t * did not directly know the next step in resolution of the CNAME, so\n+\t * instead of putting the resolution elsewhere in the response, has\n+\t * told us just the CNAME and left it to us to find out its resolution\n+\t * separately.\n+\t *\n+\t * Reset this request to be for the CNAME, and restart the request\n+\t * action with a new tid.\n \t */\n-\tp \u003d stack[1].p;\n-\tstp \u003d 0;\n-\tgoto resume;\n+\n+\tif (lws_async_dns_get_new_tid(q-\u003econtext, q))\n+\t\treturn -1;\n+\n+\tq-\u003etid \u0026\u003d 0xfffe;\n+\tq-\u003easked \u003d q-\u003eresponded \u003d 0;\n+#if defined(LWS_WITH_IPV6)\n+\tq-\u003esent[1] \u003d 0;\n+#endif\n+\tq-\u003esent[0] \u003d 0;\n+\tq-\u003erecursion++;\n+\tif (q-\u003erecursion \u003d\u003d DNS_RECURSION_LIMIT) {\n+\t\tlwsl_err(\u0022%s: recursion overflow\u005cn\u0022, __func__);\n+\n+\t\treturn -1;\n+\t}\n+\n+\tif (q-\u003efirstcache)\n+\t\tlws_adns_cache_destroy(q-\u003efirstcache);\n+\tq-\u003efirstcache \u003d NULL;\n+\n+\t/* overwrite the query name with the CNAME */\n+\n+\tn \u003d 0;\n+\t{\n+\t\tchar *cp \u003d (char *)\u0026q[1];\n+\n+\t\twhile (stack[stp].name[n])\n+\t\t\t*cp++ \u003d tolower(stack[stp].name[n++]);\n+\t\t/* trim the following . if any */\n+\t\tif (n \u0026\u0026 cp[-1] \u003d\u003d '.')\n+\t\t\tcp--;\n+\t\t*cp \u003d '\u005c0';\n+\t}\n+\n+\tlws_callback_on_writable(q-\u003edns-\u003ewsi);\n+\n+\treturn 2;\n }\n \n int\n@@ -464,12 +527,12 @@ lws_async_dns_store(const char *name, void *opaque, uint32_t ttl,\n void\n lws_adns_parse_udp(lws_async_dns_t *dns, const uint8_t *pkt, size_t len)\n {\n+\tconst char *nm, *nmcname;\n \tlws_adns_cache_t *c;\n \tstruct adstore adst;\n \tlws_adns_q_t *q;\n-\tconst char *nm;\n+\tint n, ncname;\n \tsize_t est;\n-\tint n;\n \n \t// lwsl_hexdump_notice(pkt, len);\n \n@@ -503,7 +566,10 @@ lws_adns_parse_udp(lws_async_dns_t *dns, const uint8_t *pkt, size_t len)\n \t}\n \n \tq-\u003eresponded |\u003d n;\n-\tnm \u003d (const char *)\u0026q[1];\n+\n+\t/* we want to confirm the results against what we last requested... */\n+\n+\tnmcname \u003d ((const char *)\u0026q[1]);\n \n \t/*\n \t * First walk the packet figuring out the allocation needed for all\n@@ -514,13 +580,26 @@ lws_adns_parse_udp(lws_async_dns_t *dns, const uint8_t *pkt, size_t len)\n \t * char []: copy of resolved name\n \t */\n \n-\tn \u003d strlen(nm) + 1;\n+\tncname \u003d strlen(nmcname) + 1;\n \n-\test \u003d sizeof(lws_adns_cache_t) + n;\n-\tif (lws_ser_ru16be(pkt + DHO_NANSWERS) \u0026\u0026\n-\t lws_adns_iterate(pkt, len, nm, lws_async_dns_estimate, \u0026est) \u003c 0)\n+\test \u003d sizeof(lws_adns_cache_t) + ncname;\n+\tif (lws_ser_ru16be(pkt + DHO_NANSWERS)) {\n+\t\tint ir \u003d lws_adns_iterate(q, pkt, len, nmcname,\n+\t\t\t\t\t lws_async_dns_estimate, \u0026est);\n+\t\tif (ir \u003c 0)\n \t\t\tgoto fail_out;\n \n+\t\tif (ir \u003d\u003d 2) /* CNAME recursive resolution */\n+\t\t\treturn;\n+\t}\n+\n+\t/* but we want to create the cache entry against the original request */\n+\n+\tnm \u003d ((const char *)\u0026q[1]) + DNS_MAX;\n+\tn \u003d strlen(nm) + 1;\n+\n+\tlwsl_info(\u0022%s: create cache entry for %s, %zu\u005cn\u0022, __func__, nm,\n+\t\t\test - sizeof(lws_adns_cache_t));\n \tc \u003d lws_malloc(est, \u0022async-dns-entry\u0022);\n \tif (!c) {\n \t\tlwsl_err(\u0022%s: OOM %zu\u005cn\u0022, __func__, est);\n@@ -550,7 +629,7 @@ lws_adns_parse_udp(lws_async_dns_t *dns, const uint8_t *pkt, size_t len)\n \t */\n \n \tif (lws_ser_ru16be(pkt + DHO_NANSWERS) \u0026\u0026\n-\t lws_adns_iterate(pkt, len, nm, lws_async_dns_store, \u0026adst) \u003c 0) {\n+\t lws_adns_iterate(q, pkt, len, nmcname, lws_async_dns_store, \u0026adst) \u003c 0) {\n \t\tlws_free(c);\n \t\tgoto fail_out;\n \t}\ndiff --git a/lib/system/async-dns/async-dns.c b/lib/system/async-dns/async-dns.c\nindex 30c995d..33c8f39 100644\n--- a/lib/system/async-dns/async-dns.c\n+++ b/lib/system/async-dns/async-dns.c\n@@ -482,7 +482,7 @@ check_tid(struct lws_dll2 *d, void *user)\n \treturn q-\u003etid \u003d\u003d (uint16_t)(long)user;\n }\n \n-static int\n+int\n lws_async_dns_get_new_tid(struct lws_context *context, lws_adns_q_t *q)\n {\n \tlws_async_dns_t *dns \u003d \u0026context-\u003easync_dns;\n@@ -492,13 +492,17 @@ lws_async_dns_get_new_tid(struct lws_context *context, lws_adns_q_t *q)\n \t * Make the TID unpredictable, but must be unique amongst ongoing ones\n \t */\n \tdo {\n-\t\tif (lws_get_random(context, \u0026q-\u003etid, 2) !\u003d 2)\n+\t\tuint16_t tid;\n+\n+\t\tif (lws_get_random(context, \u0026tid, 2) !\u003d 2)\n \t\t\treturn -1;\n \n \t\tif (lws_dll2_foreach_safe(\u0026dns-\u003ewaiting,\n-\t\t\t\t\t (void *)(long)q-\u003etid, check_tid))\n+\t\t\t\t\t (void *)(long)tid, check_tid))\n \t\t\tcontinue;\n \n+\t\tq-\u003etid \u003d tid;\n+\n \t\treturn 0;\n \n \t} while (budget--);\n@@ -536,6 +540,9 @@ lws_async_dns_query(struct lws_context *context, int tsi, const char *name,\n \t}\n #endif\n \n+\tif (nlen \u003e\u003d DNS_MAX - 1)\n+\t\tgoto failed;\n+\n \t/*\n \t * we magically know 'localhost' and 'localhost6' if IPv6, this is a\n \t * sort of canned /etc/hosts\n@@ -674,12 +681,18 @@ lws_async_dns_query(struct lws_context *context, int tsi, const char *name,\n \t * For ipv6, A / ipv4 is routable over ipv6. So we always ask for A\n \t * first and then if ipv6, AAAA separately.\n \t *\n-\t * The results need binding into\n+\t * Allocate for DNS_MAX, because we may recurse and alter what we're\n+\t * looking for.\n+\t *\n+\t * 0 sizeof(*q) sizeof(*q) + DNS_MAX\n+\t * [lws_adns_q_t][ name (DNS_MAX reserved) ] [ name \u005c0 ]\n \t */\n \n-\tq \u003d (lws_adns_q_t *)lws_zalloc(sizeof(*q) + nlen + 1, __func__);\n+\tq \u003d (lws_adns_q_t *)lws_malloc(sizeof(*q) + DNS_MAX + nlen + 1,\n+\t\t\t\t\t__func__);\n \tif (!q)\n \t\tgoto failed;\n+\tmemset(q, 0, sizeof(*q));\n \n \tif (wsi)\n \t\tlws_dll2_add_head(\u0026wsi-\u003eadns, \u0026q-\u003ewsi_adns);\n@@ -702,10 +715,19 @@ lws_async_dns_query(struct lws_context *context, int tsi, const char *name,\n \tlws_retry_sul_schedule_retry_wsi(dns-\u003ewsi, \u0026q-\u003esul,\n \t\t\t\t\t lws_async_dns_sul_cb_retry, \u0026q-\u003eretry);\n \n+\t/*\n+\t * We may rewrite the copy at +sizeof(*q) for CNAME recursion. Keep\n+\t * a second copy at + sizeof(*q) + DNS_MAX so we can create the cache\n+\t * entry for the original name, not the last CNAME we met.\n+\t */\n+\n \tp \u003d (char *)\u0026q[1];\n-\twhile (nlen--)\n+\twhile (nlen--) {\n \t\t*p++ \u003d tolower(*name++);\n+\t\tp[DNS_MAX - 1] \u003d p[-1];\n+\t}\n \t*p \u003d '\u005c0';\n+\tp[DNS_MAX] \u003d '\u005c0';\n \n \tlws_callback_on_writable(dns-\u003ewsi);\n \ndiff --git a/lib/system/async-dns/private-lib-async-dns.h b/lib/system/async-dns/private-lib-async-dns.h\nindex 473c90e..eafd4d0 100644\n--- a/lib/system/async-dns/private-lib-async-dns.h\n+++ b/lib/system/async-dns/private-lib-async-dns.h\n@@ -23,10 +23,11 @@\n */\n \n \n-#define\tDNS_MAX\t\t\t128\t/* Maximum host name\t\t*/\n-#define\tDNS_PACKET_LEN\t\t1400\t/* Buffer size for DNS packet\t*/\n-#define\tMAX_CACHE_ENTRIES\t10\t/* Dont cache more than that\t*/\n-#define\tDNS_QUERY_TIMEOUT\t30\t/* Query timeout, seconds\t*/\n+#define DNS_MAX\t\t\t96\t/* Maximum host name\t\t*/\n+#define DNS_RECURSION_LIMIT\t3\n+#define DNS_PACKET_LEN\t\t1400\t/* Buffer size for DNS packet\t*/\n+#define MAX_CACHE_ENTRIES\t10\t/* Dont cache more than that\t*/\n+#define DNS_QUERY_TIMEOUT\t30\t/* Query timeout, seconds\t*/\n \n /*\n * ... when we completed a query then the query object is destroyed and a\n@@ -78,6 +79,8 @@ typedef struct {\n \tuint8_t\t\t\tasked;\n \tuint8_t\t\t\tresponded;\n \n+\tuint8_t\t\t\trecursion;\n+\n \t/* name overallocated here */\n } lws_adns_q_t;\n \n@@ -116,3 +119,6 @@ lws_adns_get_query(lws_async_dns_t *dns, adns_query_type_t qtype,\n \n void\n lws_async_dns_trim_cache(lws_async_dns_t *dns);\n+\n+int\n+lws_async_dns_get_new_tid(struct lws_context *context, lws_adns_q_t *q);\n","s":{"c":1752652521,"u": 8393}} ],"g": 10589,"chitpc": 0,"ehitpc": 0,"indexed":0 , "ab": 0, "si": 0, "db":0, "di":0, "sat":0, "lfc": "0000"}