[Libwebsockets] Maximun size of URL

Andy Green andy at warmcat.com
Wed Sep 20 01:11:50 CEST 2017



On 09/19/2017 11:08 PM, Renan Le Padellec wrote:
> Problem solved
> 
> My server uses an external poll integration. I need to perform a "forced 
> service" action.
> 
> Thanks for your explanation that helped me to diagnose my issue,

Thanks for letting us know the root cause.

The forced service handling is already in the test-server code for 
external poll and the api docs explains it briefly.

On master I added some extra docs in README.coding.md about it:


You also need to take care about "forced service" somehow... these are cases
where the network event was consumed, incoming data was all read, for 
example,
but the work arising from it was not completed.  There will not be any more
network event to trigger the remaining work, Eg, we read compressed 
data, but
we did not use up all the decompressed data before returning to the 
event loop
because we had to write some of it.

Lws provides an API to determine if anyone is waiting for forced service,
`lws_service_adjust_timeout(context, 1, tsi)`, normally tsi is 0.  If it 
returns
0, then at least one connection has pending work you can get done by calling
`lws_service_tsi(context, -1, tsi)`, again normally tsi is 0.

For eg, the default poll() event loop, or libuv/ev/event, lws does this
checking for you and handles it automatically.  But in the external polling
loop case, you must do it explicitly.  Handling it after every normal 
service
triggered by the external poll fd should be enough, since the situations 
needing
it are initially triggered by actual network events.

An example of handling it is shown in the test-server code specific to
external polling.


-Andy

> Renan
> 
> On 16/09/2017 00:36, Andy Green wrote:
> 
>>
>>
>> On 09/15/2017 04:13 PM, Renan Le Padellec wrote:
>>> Hi Andy,
>>>
>>> On 15/09/2017 09:20, Andy Green wrote:
>>>>
>>>>
>>>> On 09/15/2017 03:04 PM, Renan Le Padellec wrote:
>>>>> Hi,
>>>>>
>>>>> With the latest version, what is the limitation of the size of the 
>>>>> URL? It seems that the library is not able to handle an URL with a 
>>>>> quite long query string (~2K).
>>>>
>>>> It is configurable at context creation time
>>>>
>>>>     short max_http_header_data;
>>>>     /**< CONTEXT: The max amount of header payload that can be handled
>>>>      * in an http request (unrecognized header payload is dropped) */
>>>>
>>>> If you leave it at zero, it uses the default of 4096... that is for 
>>>> all the recognized header content.
>>>>
>>> Increasing the default value to 8192 does not fix my issue.
>>>
>>> However, modifying the hard-coded size of the allocated_headers rx 
>>> buffer from 2048 to 4096 does solve my issue.
>>>
>>> Should this rx buffer be based on the max_http_header_data value ?
>>
>> Nope, because it's used as a persistent temp buffer to feed the HTTP 
>> parsing state machine.  The headers won't finish parsing until they 
>> reach the state they saw CRLFCRLF whether they need to fill ah->rx[] 
>> multiple times or not.  It has to be 'sticky' because the peer may 
>> send multiple header sets pipelined, and we may read part or all of 
>> another header set, since we can't know the length of one set of 
>> headers.  So it has to survive going back to the event loop for an 
>> unknown amount of times.  But that's all.  Size of rx[] is just 
>> balancing efficiency of how much to read at once against memory usage, 
>> and has no other meaning AFAIK.  But I could have forgotten something.
>>
>> I tried to reproduce this -- you don't describe your URL or the kind 
>> of failure you experience -- first with
>>
>> wget 
>> http://localhost:7681/012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789.png 
>>
>>
>> (it's 3000 bytes).  However it's not possible on my system to create a 
>> filename of length 3K for it to serve, so it responds with a 404 on 
>> both the original test server and test-server-v2.0. Then I tried
>>
>> wget 
>> http://localhost:7681/libwebsockets.org-logo.png?x=012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789 
>>
>>
>> and it serves the PNG fine.
>>
>> This is with just the defaults (4K header data limit and 2K ah.rx[]).
>>
>> What exactly is the problem you are seeing and how can I try to 
>> reproduce it?
>>
>> -Andy
>>
>>> Thanks a lot for your help,
>>> Renan
>>>
>>>> -Andy
>>>>
>>>>
>>>>> Regards,
>>>>>
>>>>> Renan
>>>>>
>>>>> _______________________________________________
>>>>> Libwebsockets mailing list
>>>>> Libwebsockets at ml.libwebsockets.org
>>>>> https://libwebsockets.org/mailman/listinfo/libwebsockets
>>>
> 


More information about the Libwebsockets mailing list