HeyDancing: An AR/VR-Based Dancing Coach
Published:
Dance Anytime, Anywhere – Immersive Learning with VR/AR, HeyDancing!
Team members: Yanling Hua, Yuxi Zhao, Jiuming Zeng
Tools: Unity, Blender, OpenXR, Meta Quest 3, MQTT, Websockets, Python
Course: Augmented Reality - Interaction - MAMN60, Lund University
Motivation
Imagine turning your living room into a high-tech dance studio! This project brings professional dance training home using cutting-edge VR/AR, letting you groove with a virtual coach and get real-time guidance—no commute, no pressure, just pure rhythm.
Objectives
- Recreate the Studio Vibe: Simulate tutor–learner interaction in a fully digital, immersive space
- Level Up Faster: Get instant, real-time feedback on every move to sharpen your skills
- Put You in Control: Learn at your own pace, your own way—with tech that adapts to your needs
System Design
- Real-Time Motion Capture: Markerless 3D pose estimation from video input
- Data Communication Module:
- Uses Mosquitto to stream motion data from capture system to PC
- Employs WebSockets to sync motion data between Unity and the VR headset
- Animation & Feedback Engine:
- Visualizes the user’s avatar and virtual coach in real-time
- Provides instant feedback on body part accuracy
- VR & AR Display Module:
- Combines VR for deep immersion
- Integrates AR for real-world practice with virtual guidance
The Future of Learning is Here—And It’s Got Rhythm.
Want to dive deeper? Check out the full paper here (PDF).