libwebsockets
Lightweight C library for HTML5 websockets
Loading...
Searching...
No Matches
Porting lws' sansIO half

This is for whoever ports lws' protocol half to another language, first to safe Rust as the lws crates (repo lws-rs). It says what in the C tree is the source of the port, what the port translates, what it redesigns because C's shape does not carry over, what it leaves behind, and how it is checked against C. READMEs/README.sans-io-split.md is how the C tree was split to make this possible; read its "The contract" and "Time" first.

What is ported, and in what order

The port is of the sansIO half: lib/sansio, plus the substrate neither half owns that it uses (lib/core, lib/misc, the neither-half files of lib/core-net, the generic crypto). IO (lib/io, lib/plat, lib/event-libs, the tls library glue in lib/tls, lib/drivers) is not ported: the port's IO is written fresh for its own runtime, against the same contract.

The order follows what the C harness can already check, since a role is only ported once there are transcripts to replay against it:

stage C source size in C (lines) checked by
1 core: the connection state machine (lib/sansio/wsi-state.c, wsi.c, close.c), http (lib/sansio/http), h1 (lib/sansio/h1), ws (lib/sansio/ws) http ~16.5k, h1 ~1.4k, ws ~6.2k transcripts h1-ws-server, h1-client-get, ws-client; fuzz corpora fuzz-h1, fuzz-ws, fuzz-ws-pmd
2 h2 (lib/sansio/h2: frames, hpack.c) ~8.8k h2 transcripts (not written yet); fuzz-h2
3 quic and h3 (lib/sansio/quic, lib/sansio/h3, qpack.c) ~13.5k datagram transcripts (not written yet); fuzz-qpack
4 mqtt (lib/sansio/mqtt) ~4.4k mqtt transcripts (not written yet)

Stage 1 can start now. The later stages wait for their transcripts in C (README.sans-io-split.md, "Open before it is done", item 2) and, for quic, for the C behaviour debts under "Do not copy" below to be fixed first.

The layout agreed for the Rust side: one repo, one Cargo workspace, member crates under crates/: lws-core (the connection machine, time, the substrate), one crate per protocol (lws-h1, lws-ws, lws-h2, lws-h3, lws-wt), lws-io (sockets, tls, the loop), and lws, a facade that re-exports them behind features. No crate has a -rs suffix. quinn / quinn-proto is the model for the split between a sans-IO protocol crate and its IO.

What is translated

These are the parts whose behaviour is the specification. They are translated closely, keeping their names where that helps someone read both:

  • The state tables. A connection's state is four machines in one word (transport, carrier, transaction, close), and every transition is a row of a table: the events in lws_wsi_event_edges[] in lib/sansio/wsi-state.c, described in READMEs/README.wsi-state-machines.md. A transition not in the table is a bug in C (a build with LWS_WITH_STATE_CHECK aborts on it). The port's machines are these tables, as enums and match arms, and a transition the table does not have is unrepresentable rather than checked.
  • The parsers and framing. The h1 request and response parser (lib/sansio/http/parsers.c, its header-name table lextable.h generated by minilex.c), chunked coding, ws framing and masking (lib/sansio/ws/ops-ws.c, client-parser-ws.c) and permessage-deflate (lib/sansio/ws/ext), h2 frames and hpack, quic packets and frames, qpack. Their limits (header sizes, frame sizes, table sizes) are part of the behaviour: keep them, and where C refuses input, refuse it the same way. The fuzz corpora (fuzz/fuzz-*/seeds/) are the inputs that found C's bugs: use them as the port's fuzz seeds and regression tests.
  • The contract with IO. What sansIO asks of IO is lws_io_ops_t (include/libwebsockets/lws-io-ops.h, every member documented), grouped by theme: the loop (want write, want read, the deadline moved, close by phase), the connection object, a client's transport, the tls session, tx, service, the vhost. In the port that is the IO side's trait, or, quinn-proto style, requests the protocol side returns for its caller to act on. The requests of lws' own features that no other IO has (a cgi child, jit trust: lws_io_plumbing_t) are not part of it.
  • Time and random as inputs. C's sansIO asks the service thread's time (lws_wsi_now(), lws_wsi_now_wall()) and never the clock; an embedder gives it with lws_service_set_now(). In the port, now (monotonic and wall) is an argument of every entry point that can decide by it, as quinn-proto's handle_timeout(now) is, and the next deadline is a query. Random is an injected source; see "Checking", below, for why the port may want to draw it the way C does.

What is redesigned

C's shape here does not carry over to safe Rust, and translating it literally would be the wrong port:

  • Callbacks become events. C tells the application what happened by calling its protocol callback (125 reasons, in include/libwebsockets/lws-callbacks.h) from inside the protocol code, about a hundred call sites in lib/sansio. The callback may call back into lws (write, ask for writeable, close), and may close the very connection it was called for, so the C code checks after many calls whether the connection still exists (the ..._ALREADY_DIED results, the "1: the wsi was closed" returns of the service calls). Safe Rust cannot hold the connection mutably across such a call. The port returns events instead (the connection's poll_event()), and takes the application's writes, closes and write requests as calls made between them. So what is translated is the state table and the byte handling around each callback site, not the callback API. The C reasons map to events; the transcripts' app_rx is the first of them.
  • The object graph becomes an arena. C's struct lws points to its parent, its mux children and siblings, its network connection, its vhost and context, and sits on intrusive lws_dll2 lists (timers, pending rx, pending writes). In the port, connections are entries in a slab owned by the endpoint, the links are indexes, and the timer and pending lists are ordinary collections of indexes. A connection's IO half (C's lws_io_adjunct, held by pointer) is the IO side's own state keyed by the same index.
  • One role at a time is an enum. C keeps each role's state in a union in struct lws and dispatches through a table of role ops (27 of them, LWS_COUNT_ROLE_OPS), with role changes (h1 to ws, h1 to h2 by upgrade) swapping the table. The port's connection holds an enum of role states, and a role change is a transition of it, as in the state tables.
  • tx is a pull. In C the application's lws_write() frames in place in the LWS_PRE bytes before its payload and pushes the frame to IO; the protocols' own packets (h2's, quic's) are already pulled by IO. The port pulls all of it: the application queues payload, and the connection produces frames when IO asks for bytes. LWS_PRE and the per-thread scratch buffers (serv_buf, compose_buf) are C's answer to allocation and copying and do not carry over.
  • Build options become a few features. C builds in and out a great many options (roles, client, server, extensions, logging), guarding code with the preprocessor. The port has cargo features at crate granularity, not an option per C #if.
  • No locks. C's sansIO takes the service thread, context and vhost locks about 38 times so connections can be touched from other threads. A sans-IO connection has one owner; moving work between threads is the application's and the IO side's business, through Send.

What is left behind

  • IO: sockets, the poll set, the event library adapters, the platforms, the tls library glue. The port's IO is written for its runtime.
  • Secure streams, lwsws, the plugins, cgi and jit trust are layers over both halves; they are not part of the sansIO port.
  • The logging, latency measurement and metrics plumbing: the port uses its ecosystem's.

Do not copy

C behaviour known to be wrong and not yet fixed (README.sans-io-split.md, "Open before it is done", item 3), which the port must not reproduce, and whose fixes in C will change transcripts when they land:

  • a quic frame in flight counts as buffered output, so an h3 file response goes one fragment per round trip, and the loop spins after the client vanishes;
  • a mux stream holding parked rx may not get the service pass that should drain it (unverified);
  • the status a kept-warm connection's joiner sees when it is established (parked).

Checking the port against C

  • Transcripts. minimal-examples-lowlevel/api-tests/api-test-sansio/transcripts/ holds each connection the C harness drives as data: the times, the bytes in and out, what the application was given, the close. Its README specifies the format, how a replay runs, and the application each connection needs. The port replays them in its tests: the same bytes at the same times must produce the same bytes and events. C checks itself against the same files in every build, so they only change with a deliberate change to lws' behaviour, whose commit says why.
  • Seeded random. A transcript with a nonzero seed depends on lws' random (a ws client's key and masks). C draws it from xoshiro256** seeded by splitmix64, each 64-bit result little-endian (lws_fi_random_seed(), the vector in api-test-random-prng). The port either draws the same stream in the same order, which makes those transcripts replay byte for byte, or treats the drawn values as free: taking them from its own output and checking what follows from them.
  • Fuzzing. The port's fuzz targets start from C's corpora (fuzz/fuzz-*/seeds/, fuzz/README.md), which include the inputs of every parser bug C has had.
  • Differential runs and conformance. Beyond the transcripts, the port can be run against C lws over real connections, and against the protocol conformance suites (autobahn for ws, h2spec, h3spec).

Words C uses

C meaning
wsi a connection object, struct lws; also a mux stream, a listener, a pipe
network wsi (nwsi) the connection that owns the transport under mux streams; a server quic connection is itself a mux child of its udp listener, and the walk up to the nwsi stops below a listener (mux_listener, sansIO's own record, set when quic hangs a connection off it)
mux h2, h3 and mqtt streams multiplexed on one network connection
pt a service thread's state: its timers, its scratch buffers, its poll set
vhost a virtual host: listen sockets, protocols, tls config
context the whole lws instance
sul a timer: an entry on a thread's sorted list of deadlines
ah an allocated header table, borrowed from a pool while headers are parsed
role the protocol a connection speaks now (h1, h2, ws, ...), with its role ops
lwsi_state the connection's state word (README.wsi-state-machines.md)
seam lib/sansio/private-lib-sansio-seam.h: sansIO's spelling of its requests of IO