||| |—|—|—| |cmake| LWS_WITH_CBOR, LWS_WITH_CBOR_FLOAT| |Header| ./include/libwebsockets/lws-lecp.h| |api-test| ./minimal-examples/api-tests/api-test-lecp/| |test app| ./test-apps/test-lecp.c -> libwebsockets-test-lecp|

LECP is the RFC8949 CBOR stream parsing counterpart to LEJP for JSON.

Features

Completely immune to input fragmentation, give it any size blocks of CBOR as they become available; 1 byte, or 100K at a time give identical parsing results
Input chunks discarded as they are parsed, whole CBOR never needed in memory
Nonrecursive, fixed stack usage of a few dozen bytes
No heap allocations at all, just requires ~500 byte context usually on caller stack
Creates callbacks to a user-provided handler as members are parsed out
No payload size limit, supports huge / endless strings or blobs bigger than system memory
Collates utf-8 text and blob payloads into a 250-byte chunk buffer for ease of access
Write apis don't use any heap allocations or recursion either
Write apis use an explicit context with its own lifecycle, and printf style vaargs including sized blobs, C strings, double, int, unsigned long etc
Completely immune to output fragmentation, supports huge strings and blobs into small buffers, api returns to indicates unfinished if it needs to be called again to continue; 1 byte or 100K output buffer give same results
Write apis completely fill available buffer and if unfinished, continues into same or different buffer when called again with same args; no requirement for subsequent calls to be done sequentially or even from same function

Type limits

CBOR allows negative integers of up to 64 bits, these do not fit into a uint64_t. LECP has a union for numbers that includes the types uint64_t and int64_t, but it does not separately handle negative integers. Only -2^63.. 2^64 -1 can be handled by the C types, the oversize negative numbers wrap and should be avoided.

Floating point support

Floats are handled using the IEEE memory format, it means they can be parsed from the CBOR without needing any floating point support in the build. If floating point is available, you can also enable LWS_WITH_CBOR_FLOAT and a float and double types are available in the number item union. Otherwise these are handled as ctx->item.u.u32 and ctx->item.u.u64 union members.

Half-float (16-bit) is defined in CBOR and always handled as a uint16_t number union member ctx->item.u.hf.

Callback reasons

The user callback does not have to handle any callbacks, it only needs to process the data for the ones it is interested in.

Callback reason	CBOR structure	Associated data
`LECPCB_CONSTRUCTED`	Created the parse context
`LECPCB_DESTRUCTED`	Destroyed the parse context
`LECPCB_COMPLETE`	The parsing completed OK
`LECPCB_FAILED`	The parsing failed
`LECPCB_VAL_TRUE`	boolean true
`LECPCB_VAL_FALSE`	boolean false
`LECPCB_VAL_NULL`	explicit NULL
`LECPCB_VAL_NUM_INT`	signed integer	`ctx->item.u.i64`
`LECPCB_VAL_STR_START`	A UTF-8 string is starting
`LECPCB_VAL_STR_CHUNK`	The next string chunk	`ctx->npos` bytes in `ctx->buf`
`LECPCB_VAL_STR_END`	The last string chunk	`ctx->npos` bytes in `ctx->buf`
`LECPCB_ARRAY_START`	An array is starting
`LECPCB_ARRAY_END`	An array has ended
`LECPCB_OBJECT_START`	A CBOR map is starting
`LECPCB_OBJECT_END`	A CBOR map has ended
`LECPCB_TAG_START`	The following data has a tag index	`ctx->item.u.u64`
`LECPCB_TAG_END`	The end of the data referenced by the last tag
`LECPCB_VAL_NUM_UINT`	Unsigned integer	`ctx->item.u.u64`
`LECPCB_VAL_UNDEFINED`	CBOR undefined
`LECPCB_VAL_FLOAT16`	half-float available as host-endian `uint16_t`	`ctx->item.u.hf`
`LECPCB_VAL_FLOAT32`	`float` (`uint32_t` if no float support) available	`ctx->item.u.f`
`LECPCB_VAL_FLOAT64`	`double` (`uint64_t` if no float support) available	`ctx->item.u.d`
`LECPCB_VAL_SIMPLE`	CBOR simple	`ctx->item.u.u64`
`LECPCB_VAL_BLOB_START`	A binary blob is starting
`LECPCB_VAL_BLOB_CHUNK`	The next blob chunk	`ctx->npos` bytes in `ctx->buf`
`LECPCB_VAL_BLOB_END`	The last blob chunk	`ctx->npos` bytes in `ctx->buf`

CBOR indeterminite lengths

Indeterminite lengths are supported, but are concealed in the parser as far as possible, the CBOR lengths or its indeterminacy are not exposed in the callback interface at all, just chunks of data that may be the start, the middle, or the end.

Handling CBOR UTF-8 strings and blobs

When a string or blob is parsed, an advisory callback of LECPCB_VAL_STR_START or LECPCB_VAL_BLOB_START occurs first. The _STR_ callbacks indicate the content is a CBOR UTF-8 string, _BLOB_ indicates it is binary data.

Strings or blobs may have indeterminite length, but if so, they are composed of logical chunks which must have known lengths. When the _START callback occurs, the logical length either of the whole string, or of the sub-chunk if indeterminite length, can be found in ctx->item.u.u64.

Payload is collated into ctx->buf[], the valid length is in ctx->npos.

For short strings or blobs where the length is known, the whole payload is delivered in a single LECPCB_VAL_STR_END or LECPCB_VAL_BLOB_END callback.

For payloads larger than the size of ctx->buf[], LECPCB_VAL_STR_CHUNK or LECPCB_VAL_BLOB_CHUNK callbacks occur delivering each sequential bufferload. If the CBOR indicates the total length, the last chunk is delievered in a LECPCB_VAL_STR_END or LECPCB_VAL_BLOB_END.

If the CBOR indicates the string end after the chunk, a zero-length ..._END callback is provided.

Handling CBOR tags

CBOR tags are exposed as LECPCB_TAG_START and LECPCB_TAG_END pairs, at the _START callback the tag index is available in ctx->item.u.u64.

CBOR maps

You can check if you are on the "key" part of a map "key:value" pair using the helper api lecp_parse_map_is_key(ctx).

Parsing paths

LECP maintains a "parsing path" in ctx->path that represents the context of the callback events. As a convenience, at LECP context creation time, you can pass in an array of path strings you want to match on, and have any match checkable in the callback using ctx->path_match, it's 0 if no active match, or the match index from your path array starting from 1 for the first entry.

CBOR element	Representation in path
CBOR Array	`[]`
CBOR Map	`.`
CBOR Map entry key string	`keystring`

Accessing raw CBOR subtrees

Some CBOR usages like COSE require access to selected raw CBOR from the input stream. lecp_parse_report_raw(ctx, on) lets you turn on and off buffering of raw CBOR and reporting it in the parse callback with LECPCB_LITERAL_CBOR callbacks. The callbacks mean the temp buffer ctx->cbor[] has ctx->cbor_pos bytes of raw CBOR available in it. Callbacks are triggered when the buffer fills, or reporting is turned off and the buffer has something in it.

By turning the reporting on and off according to the outer CBOR parsing state, it's possible to get exactly the raw CBOR subtree that's needed.

Capturing and reporting the raw CBOR does not change that the same CBOR is being passed to the parser as usual as well.

Comparison with LEJP (JSON parser)

LECP is based on the same principles as LEJP and shares most of the callbacks. The major differences:

LEJP value callbacks all appear in ctx->buf[], ie, floating-point is provided to the callback in ascii form like "1.0". CBOR provides a more strict typing system, and the different type values are provided either in ctx->buf[] for blobs or utf-8 text strtings, or the item.u union for converted types, with additional callback reasons specific to each type.
CBOR "maps" use _OBJECT_START and _END parsing callbacks around the key / value pairs. LEJP has a special callback type PAIR_NAME for the key string / integer, but in LECP these are provided as generic callbacks dependent on type, ie, generic string callbacks or integer ones, and the value part is represented according to whatever comes.

Writing CBOR

CBOR is written into a lws_lec_pctx_t object that has been initialized to point to an output buffer of a specified size, using printf type formatting.

Output is paused if the buffer fills, and the write api may be called again later with the same context object, to resume emitting to the same or different buffer.

This allows bufferloads of encoded CBOR to be produced on demand, it's designed to fit usage in WRITEABLE callbacks and Secure Streams tx() callbacks where the buffer size for one packet is already fixed.

CBOR array and map lengths are deduced from the format string, as is whether to use indeterminite length formatting or not. For indeterminite text or binary strings, a container of < >

Format	Arg(s)	Meaning
`123`		unsigned literal number
`-123`		signed literal number
`u`	`unsigned int`	number
`lu`	`unsigned long int`	number
`llu`	`unsigned long long int`	number
`d`	`signed int`	number
`ld`	`signed long int`	number
`lld`	`signed long long int`	number
`f`	`double`	floating point number
`123(...)`		literal tag and scope
`t(...)`	`unsigned int`	tag and scope
`lt(...)`	`unsigned long int`	tag and scope
`llt(...)`	`unsigned long long int`	tag and scope
`[...]`		Array (fixed len if `]` in same format string)
`{...}`		Map (fixed len if `}` in same format string)
`<t...>`		Container for indeterminite text string frags
`<b...>`		Container for indeterminite binary string frags
‘'string’`\ilinebr </td> <td class="markdownTableBodyNone"> Literal text of known length \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone">`s`\ilinebr </td> <td class="markdownTableBodyNone">`const char `\ilinebr </td> <td class="markdownTableBodyNone"> NUL-terminated string \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone">`%.s`\ilinebr </td> <td class="markdownTableBodyNone">`int`,`const char `\ilinebr </td> <td class="markdownTableBodyNone"> length-specified string \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone">`%.b`\ilinebr </td> <td class="markdownTableBodyNone">`int`,`const uint8_t *`\ilinebr </td> <td class="markdownTableBodyNone"> length-specified binary \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone" colspan="2">`:`\ilinebr </td> <td class="markdownTableBodyNone"> separator between Map items (a:b) \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone" colspan="2">`,`		separator between Map pairs or array items

Backslash is used as an escape in ‘’...'literal strings, so'\'represents a string consisting of a single backslash, and'\''` a string consisting of a single single-quote.

For integers, various natural C types are available, but in all cases, the number is represented in CBOR using the smallest valid way based on its value, the long or long-long modifiers just apply to the expected C type in the args.

For floats, the C argument is always expected to be a double type following C type promotion, but again it is represented in CBOR using the smallest valid way based on value, half-floats are used for NaN / Infinity and where possible for values like 0.0 and -1.0.

Examples

Literal ints

uint8_t buf[128];
lws_lec_pctx_t cbw;
 
lws_lec_init(&cbw, buf, sizeof(buf));
lws_lec_printf(ctx, "-1");

||| |—|—| |Return| LWS_LECPCTX_RET_FINISHED| |ctx->used|1| |buf[]|20|

Dynamic ints

uint8_t buf[128];
lws_lec_pctx_t cbw;
int n = -1; /* could be long */
 
lws_lec_init(&cbw, buf, sizeof(buf));
lws_lec_printf(ctx, "%d", n); /* use %ld for long */

||| |—|—| |Return| LWS_LECPCTX_RET_FINISHED| |ctx->used|1| |buf[]|20|

Maps, arrays and dynamic ints

...
int args[3] = { 1, 2, 3 };
 
lws_lec_printf(ctx, "{'a':%d,'b':[%d,%d]}", args[0], args[1], args[2]);

||| |—|—| |Return| LWS_LECPCTX_RET_FINISHED| |ctx->used|9| |buf[]|A2 61 61 01 61 62 82 02 03|

String longer than the buffer

Using s and the same string as an arg gives same results

uint8_t buf[16];
lws_lec_pctx_t cbw;
 
lws_lec_init(&cbw, buf, sizeof(buf));
lws_lec_printf(ctx, "'A literal string > one buf'");
/* not required to be in same function context or same buf,
 * but the string must remain the same */
lws_lec_setbuf(&cbw, buf, sizeof(buf));
lws_lec_printf(ctx, "'A literal string > one buf'");

First call

||| |—|—| |Return| LWS_LECPCTX_RET_AGAIN| |ctx->used|16| |buf[]|78 1A 41 20 6C 69 74 65 72 61 6C 20 73 74 72 69|

Second call

||| |—|—| |Return| LWS_LECPCTX_RET_FINISHED| |ctx->used|12| |buf[]|6E 67 20 3E 20 6F 6E 65 20 62 75 66|

Binary blob longer than the buffer

uint8_t buf[16], blob[] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18 };
lws_lec_pctx_t cbw;
 
lws_lec_init(&cbw, buf, sizeof(buf));
lws_lec_printf(ctx, "%.*b", (int)sizeof(blob), blob);
/* not required to be in same function context or same buf,
 * but the length and blob must remain the same */
lws_lec_setbuf(&cbw, buf, sizeof(buf));
lws_lec_printf(ctx, "%.*b", (int)sizeof(blob), blob);

First call

||| |—|—| |Return| LWS_LECPCTX_RET_AGAIN| |ctx->used|16| |buf[]|52 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F|

Second call

||| |—|—| |Return| LWS_LECPCTX_RET_FINISHED| |ctx->used|3| |buf[]|10 11 12|