Peter Yang - Zoom Plus

Interaction DesignComputational Design

Overview

Role

Led the project, programmed and tested interactions, trained the Teachable Machine model, and helped define the concept and experience

Product Name

Zoom Plus

Timeline

3 weeks, Fall 2020

Collaborated with

Lucy Du, Connie Yumih Chang

Built with

Teachable Machine, ml5, p5.js, Socket.io, Processing, OBS, Zoom

Deliverables

Concept design, interaction prototype, machine-learning gesture model, shared canvas interactions, visual filters, introduction video

Where I Started

Problem Statement

Remote meetings can feel disconnected because facial expressions, gestures, and body language are harder to read through delayed video communication.
Standard video conferencing reactions are limited, making it difficult for participants to express mood, energy, and spontaneous responses naturally.

Context

During the rise of remote work and online classes in 2020, video conferencing became a primary space for collaboration, critique, and social interaction.
The project explored how expressive visual tools could reduce fatigue and make online meetings feel more active, playful, and present.

What I Did

Responsibilities & Contributions

Designed and prototyped shared Zoom background canvases with color, drawing, gesture, emoji, and filter interactions.
Trained gesture and expression models with Teachable Machine, then connected the model outputs to p5.js and ml5 visual effects.
Tested the prototype in Zoom using Processing, chromakey screens, OBS, and real-time video streams.

Impact

Expression

Turned subtle gestures and moods into visible shared reactions.

Participation

Made remote meetings feel more playful and active through collaborative background tools.

Prototype direction

Proposed a plug-in style experience that could extend video conferencing beyond standard emoji reactions.

What I Learned

Key Insights

Small visual reactions can make remote communication feel more immediate when normal body language cues are weakened.
Machine-learning prototypes need simple controls and clear feedback so expressive features feel lightweight rather than distracting.

Critical Constraints

The prototype depended on experimental tooling and real-time video routing, so stability and setup complexity would need refinement before productization.
A production version would require deeper integration with conferencing platforms and careful controls for privacy, performance, and accessibility.