New Backend: libavfilter virtual device (ffmpeg/ffplay)?

[Vipitis]

Currently more of a goal, I don't have a working POC yet.

While looking into solutions for recording directly to video, I wanted to still have a GUI for interactivity (for pygfx/shadertoy#55). One thing I read in the documentation is https://www.ffmpeg.org/ffmpeg-devices.html#lavfi if it works could open up some potential. Like writing frames to a encoder (which could also be the streamed remotely).

There is a GUI to this to via ffplay but I haven't had the time to look into potential python bindings.
Worst case it's a worse offscreencanvas, which wouldn't make sense to upstream.

Opening this issue to invite suggestions, I hope to have a prototype sometime later this week.

[Vipitis]

I am part way to a proof of concept... managed to stream locally to my laptop (albeit awful delay and quality).

[Vipitis]

few evenings of reading through ffmpeg documentation later and I can now say: it totally doesn't work the way I thought it might.
What I ended up is more or less just a sheduled offscreen canvas with the file encoding ontop. I also have ffplay to show what is being recorded via subprocess.PIPE. but that is like way worse than a real screen-gui and not interactive.
For GLFW specifically I had something that works, but it's not a good approach. I think to make this work you would likely just have a texture download method for gui canvases and encode that frame on the CPU. Sharing buffers didn't seem possible from my reading of the source.

I tried to keep the commits as sorta useful steps if anyone wants to have a look at it (PR linked above) - but I don't think it's worth upstream something like that... maybe as an example? otherwise feel free to close

[hmaarrfk]

I think it is useful to upload your sample code to a zip here