Background Features in Google Meet, Powered by Web ML

GOOGLEBLOG.COM

Whether you’re sat in Zoom calls as part of your day job, or face-timing your relatives, it’s likely that you’ll have spent a significant amount of time using video conferencing software this year. This blog post looks at how the Google Meet team implemented the popular background-blur effect using WebAssembly.

The blur effect is built using MediaPipe, an open source cross-platform, machine learning library for creating processing pipelines for live and streaming media. Using this library you can perform tasks such as face detection, iris tracking or even hair segmentation. Earlier this year they announced browser support via WebAssembly, the online demos are great - for example take a look at the hair segmentation demo. The MediaPipe ML models are written in C++ and compiled to WebAssembly using Emscripten.

It’s great to see WebAssembly being used in another production application. Notably the new WebAssembly SIMD (single instruction multiple data) feature doubles the model performance.

FFMPEG.WASM

GITHUB.IO

FFmpeg, which launched almost 20 years ago, is an open source project with a suite of tools for handling video, audio, and other multimedia files and streams. It is commonly used for video scaling and transcoding. This project is a WebAssembly port of FFmpeg, using Emscripten to compile the core libraries C++ codebase to wasm. Together with a JavaScript wrapper that provides a friendly API for use within the browser. The project also has a number of blog posts that describe the development process, which make interesting reading.

GLAS = WebGL + Assembly Script(WASM);

DEV.TO

“The future of game engines is coming soon, and it’s in the form of web assembly (wasm)” - a cryptic title and a bold claim kick of this article! This blog post introduces GLAS, a WebGL and AssemblyScript powered 3D games engine.

EnvoyCon 2020 Conference videos

YOUTUBE.COM

Envoy is a service proxy which allows you to write extensions using WebAssembly. The EnvoyCon 2020 conference took place around a month ago and the talk videos have started appearing online. Quite a bit of wasm-focussed content here.

On a related note, a recent talk by Luke Wagner on Interface Types has just been published,

And finally …

I asked for examples of practical / production WebAssembly applications written in Rust and C++ on Twitter - quite a few interesting responses.