Sound My Vision: Real-time Video Analysis On Mobile Platforms For Controlling Multimedia Performances

This paper presents Sound My Vision, an Android application for controlling music expression and multimedia projects. Unlike other similar applications which collect data only from sensors and input devices, Sound My Vision also analyses input video in real time and extracts low-level video features. Such a versatile controller can be used in various scenarios from entertainment and experimentation to live music performances, installations and multimedia projects. The application can replace complex setups that are usually required for capturing and analyzing a video signal in live performances. Additionally, mobility of smartphones allows perspective changes in sense that the performer can become either an object or a subject involved in controlling the expression. The most important contributions of this paper are selection of general and low-level video feature and the technical solution for seamless real-time video extraction on the Android platform.

Published in:
Proceedings of the 12th International Conference on Sound and Music Computing
Presented at:
The 12th Sound and Music Computing Conference, Maynooth University, Maynooth, Co. Kildare, Ireland, July 26 - August 1, 2015

Note: The status of this file is:

 Record created 2015-11-16, last modified 2019-12-05

Publisher's version:
Download fulltextPDF
External link:
Download fulltextURL
Rate this document:

Rate this document:
(Not yet reviewed)