[Libwebsockets] Help wanted: V4L2 / mp4 / h.264

Andy Green andy at warmcat.com
Mon Apr 12 07:53:45 CEST 2021


Hi -

I spent the last couple of weekends on a minimal example that works with 
v4l2 and libav*.  What I have is in a couple of patches ahead of main here

https://libwebsockets.org/git/libwebsockets/log?h=_v4l2

In the first patch, it can use MJPG from a webcam or cheapo HDMI - uvc 
adapter (whatever is at /dev/video0) to sequentially capture JPG stills 
and distribute them to ws clients.  It's integrated into lws event loop 
cleanly without threads.  It works successfully, but when I adapted it 
to send jpegs back-to-back, although Chrome is OK with it, firefox 
garbage collection cannot keep up with it.

So the second patch adds h.264 transcoding via libav*, this works as far 
as it goes and produces a valid h.264 TS stream.

However I cannot tie that up to MediaSource apis on the browser side, 
these demand an MP4 wrapper I don't know how to produce with libav*. 
MediaSource seems to want to offer dynamic streaming, but they want the 
input to have basically mp4 file semantics with a header from what I 
understand.

This is quite far along, and is going to be quite useful when it is 
complete, so I am putting it out there in case anyone here with more 
experience than me with either the browser or libav* either has 
suggestions or would like to cooperate to move it on.

-Andy


More information about the Libwebsockets mailing list