[Libwebsockets] Lightweight JSON parser in C

Jack Mitchell ml at communistcode.co.uk
Fri Feb 22 17:36:53 CET 2013


On 22/02/13 16:11, "Andy Green (林安廸)" wrote:
> On 22/02/13 23:53, the mail apparently from Jack Mitchell included:
>> On 22/02/13 15:28, "Andy Green (林安廸)" wrote:
>>> Hi -
>>>
>>> Is there any interest in a very lightweight but 100% correct JSON 
>>> parser?
>>>
>>> I don't mean like you have in a browser where it goes through and
>>> spawns objects for everything in the JSON. The idea is it is basically
>>> a stateful stream reader, which can do something when the object path
>>> it is on top of matches what you're interested in.
>>>
>>> It does not recurse, nor malloc anything. You call it with cursor
>>> structs which start off zero'd and hold all parsing state. The cursor
>>> structs have no pointers and may be copied. You can call the multiple
>>> times with the same cursor as new JSON pieces come in and it will
>>> parse on from the previous state; it's a bytewise parser so there are
>>> no fragmentation issues. It means you don't need to hold the JSON all
>>> at once anywhere but can still parse the whole JSON.
>>>
>>> So the style would be parse until the node of interest is held in the
>>> cursor struct, then you can use a copy of that as a starting point for
>>> rummaging around.
>>>
>>> I have the parser working - it is very lightweight, the cursor struct
>>> is the only storage and it's < 256 bytes - but I am a bit stumped
>>> about how to most flexibly expose the wanted results. I know what I
>>> want it for and can hack that up easily, but I am wondering if there
>>> are other real-world uses for very cheap JSON parsing in C... if so,
>>> what does the JSON look like and what are the operations needed on it?
>>>
>>> -Andy
>>> _______________________________________________
>>> Libwebsockets mailing list
>>> Libwebsockets at ml.libwebsockets.org
>>> http://ml.libwebsockets.org/mailman/listinfo/libwebsockets
>>
>> Hi Andy,
>>
>> I have JSON parsing in my application and use the Jansson JSON Library
>> to parse and manage it. A lot of what you said went over my head so I'm
>> not sure if it would be applicable in my case, as I do a lot of parsing
>> of the JSON and I also reply to the recieve method in JSON which I
>> create on the fly.
>>
>> If you want to poke into my use case then a few simple fairly specific
>> questions would probably be enough to coax some useful information out
>> of me ;)
>
> Jansson seems to take the approach to sit down and allocate a parsed 
> model of the JSON, which you can then query, like a browser does it. 
> It's fine but it does not sit well with very resource-constrained 
> devices.  My target is a Cortex-M0 with 32KByte flash and 2KBytes free 
> memory.
>
> This is different, it's a stream parser that does not even need the 
> JSON in one place at one time, and it does not "make notes as it 
> goes", it just goes through the JSON like a text search until it sees 
> it has arrived at the bit you are interested in.  (The structure of 
> where it is, the parsing state, is held in a struct.)  Then, you will 
> be able to get parsed elements from it, skip to the next bit you are 
> interested in etc.
>
> What I was asking is
>
>  - what kind of queries are you doing at the moment in Jansson?

I have a custom RPC style implementation in JSON, so I currently take 
received messages with a method and parameters which I then parse, act 
upon and build a new JSON string to write out to all clients with 
updated information.

I quite often pick parameters out in an odd order and often make 
extensive use of arrays and objects.

>
>  - Find a section like "thing.mylist", and then get strings from 
> "thing.mylist[n]"?

Yes, my main use is like this, for example this is a verbose snippet of 
my JSON in and out from the server point of view:

[946949789:3863] INFO: insert_wsi_socket_into_fds: wsi=0x1d73c8, 
sock=22, fds pos=1
[946949789:4391] INFO: Allocating RX buffer 1046
JSON IN: { "method":"setCard", "card":0 }
JSON IN: {"method":"0","parameters":{"callbackFunction":"test"}}
JSON OUT: 
{"success":0,"method":["callback"],"calledParams":{"callbackFunction":"test"}}
JSON IN: {"method":"1","parameters":{"callbackFunction":"test"}}
JSON OUT: 
{"success":0,"method":["callback"],"calledParams":{"callbackFunction":"test"}}
JSON IN: {"method":"1","parameters":{"callbackFunction":"test"}}
JSON OUT: 
{"success":0,"method":["callback"],"calledParams":{"callbackFunction":"test"}}
[946949790] Card 0: Processing Data
JSON OUT: 
{"method":["updateRegisters"],"parameters":{"10":{"val":4553},"11":{"val":4553},"17":{"val":1637},"12":{"val":22186},"13":{"val":22186},"23":{"val":1637}}}
JSON OUT: 
{"method":["updateData"],"parameters":{"43":{"val":4553},"45":{"val":22186}}}
JSON OUT: 
{"method":["updateData"],"parameters":{"44":{"val":4553},"46":{"val":22186}}}
JSON OUT: {"method":["updateData"],"parameters":{"49":{"val":1637}}}
JSON OUT: {"method":["updateData"],"parameters":{"53":{"val":1637}}}
JSON IN: 
{"method":"3","parameters":{"routeID":2,"routeState":1,"callbackFunction":"finishSetRoute"}}
JSON OUT: 
{"success":0,"method":["callback"],"calledParams":{"routeID":2,"routeState":1,"callbackFunction":"finishSetRoute"}}
JSON IN: 
{"method":"3","parameters":{"routeID":1,"routeState":1,"callbackFunction":"finishSetRoute"}}
JSON OUT: 
{"success":0,"method":["callback"],"calledParams":{"routeID":1,"routeState":1,"callbackFunction":"finishSetRoute"}}
[946949791] Card 0: Processing Data
JSON OUT: 
{"method":["updateRegisters"],"parameters":{"13":{"val":22509},"1":{"val":12424},"10":{"val":4229},"11":{"val":4229},"12":{"val":22509},"17":{"val":1599},"23":{"val":1599}}}
JSON OUT: 
{"method":["updateRouting"],"parameters":{"33":{"val":1},"34":{"val":1},"35":{"val":1},"36":{"val":1}}}
JSON OUT: 
{"method":["updateData"],"parameters":{"43":{"val":4229},"45":{"val":22509}}}
JSON OUT: 
{"method":["updateData"],"parameters":{"44":{"val":4229},"46":{"val":22509}}}
JSON OUT: {"method":["updateData"],"parameters":{"49":{"val":1599}}}
JSON OUT: {"method":["updateData"],"parameters":{"53":{"val":1599}}}
[946949792] Card 0: Processing Data


>
>  - Something different?  Are sub-objects in the results you want? How 
> do you deal with that at the moment?

As you can see above, I do use sub-objects. I'm not so sure my use case 
is something to be fulfilled by a zero-copy speedy parser :)
>
> -Andy

-- 

   Jack Mitchell (jack at embed.me.uk)
   Embedded Systems Engineer
   http://www.embed.me.uk

--




More information about the Libwebsockets mailing list