Background Features in Google Meet, Powered by Web ML
Whether you’re sat in Zoom calls as part of your day job, or face-timing your relatives, it’s likely that you’ll have spent a significant amount of time using video conferencing software this year. This blog post looks at how the Google Meet team implemented the popular background-blur effect using WebAssembly.
The blur effect is built using MediaPipe, an open source cross-platform, machine learning library for creating processing pipelines for live and streaming media. Using this library you can perform tasks such as face detection, iris tracking or even hair segmentation. Earlier this year they announced browser support via WebAssembly, the online demos are great - for example take a look at the hair segmentation demo. The MediaPipe ML models are written in C++ and compiled to WebAssembly using Emscripten.
It’s great to see WebAssembly being used in another production application. Notably the new WebAssembly SIMD (single instruction multiple data) feature doubles the model performance.
GLAS = WebGL + Assembly Script(WASM);
“The future of game engines is coming soon, and it’s in the form of web assembly (wasm)” - a cryptic title and a bold claim kick of this article! This blog post introduces GLAS, a WebGL and AssemblyScript powered 3D games engine.
EnvoyCon 2020 Conference videos
Envoy is a service proxy which allows you to write extensions using WebAssembly. The EnvoyCon 2020 conference took place around a month ago and the talk videos have started appearing online. Quite a bit of wasm-focussed content here.
On a related note, a recent talk by Luke Wagner on Interface Types has just been published,
And finally …
I asked for examples of practical / production WebAssembly applications written in Rust and C++ on Twitter - quite a few interesting responses.