Enjoying The SHOW
Real-time AI audience emotion tracking for speakers
What is This?
Enjoying The Show
This is an experiment for connecting speakers and audiences using AI. Any speaker can allocate a room URL for their audience, and regardless of the crowd being real or virtual, their webcams will share their dominant facial expression with the speaker in an audience summary. This helps the speaker gauge feedback from the audience just as they would by looking out at the faces in the crowd.
How it works
- Speaker: type a room name under Watch a Room. You'll land on
/watch/your-room, a live pie chart of the room's mood. - Audience: share
enjoyingthe.show/room/your-room. Each person allows webcam access and their expression is detected right in their browser. - Privacy: video never leaves the device. Only the single word for the dominant expression (like “happy”) is sent.
The Technology used for Enjoying The Show uses elaborate tooling to complete a simple straight-forward task. Each audience member's browser runs their webcam input through TensorFlow.js (via face-api) to identify the emotional sentiment of the person. The dominant detection out of Happy, Neutral, Surprised, Sad, Fearful, Disgusted, and Angry, is sent to an AWS AppSync GraphQL API. On the other side, the "watching room" URL has a GraphQL subscription and updates whenever a new audience member arrives, leaves, or changes their facial expression. Hundreds of client models can be managed and displayed on the dashboard in real-time.
Questions
Is Enjoying The Show free?
Yes. It is free to use and there is no account or sign-up. Pick a room name and share the link.
Is my webcam video uploaded anywhere?
No. Face detection runs in your own browser with TensorFlow.js. Only one word per detected face, such as "happy", is sent to the room. Video and images never leave your device.
Which emotions can it detect?
Happy, neutral, surprised, sad, fearful, disgusted and angry. The speaker sees a live pie chart of how many faces show each one.
Does it work for virtual talks on Zoom, Google Meet or a livestream?
Yes. Audience members open the room link in a browser tab while they watch. Some computers only let one app use the camera at a time, so a phone or a second device works well too.
What does the audience need?
A modern browser with JavaScript and WebGL, a webcam, and permission to use it. Nothing to install.
Can I run my own copy?
Yes. The source is MIT licensed on GitHub at github.com/GantMan/enjoytheshow, with instructions for deploying your own backend on AWS.
Watch the talk
Gant Laborde walks through how Enjoying The Show was built on Made with TensorFlow.js, from the TensorFlow channel.
Special Thanks
To Nader Dabit, who has done a great job advocating AWS Amplify and answering questions on this awesome technology. To Vincent Mühler, who has done all the hard work in TensorFlow.js with his amazing face-api.js library. To Infinite Red, who is the best damn consulting company in the world; their support allows me push the boundaries of AI, Machine Learning, and JavaScript.
And THANK YOU for sharing this with others. Feel free to use this service with your audience. I'll see you on X / Twitter.