We're excited to announce that the Music Intelligence Lab at AUB, together with biomechanics and movement expert Olivier Chiniara as co-grantee, has been awarded a two-year Creative Labs grant for a project called Music in Motion. The grant is part of Ecologies of Culture (EoC), a four-year program co-funded by the European Union and led by the Arab Fund for Arts and Culture (AFAC), in partnership with Oxfam, Echos Electrik, and Megaphone. The project is co-led with professor Elsa Maalouf.
What we're doing
Music and dance have grown up next to each other for millennia as separate but intertwined art forms. Musicians create sound, dancers respond to it, and the relationship is mostly one-way. Music in Motion is our attempt to challenge that separation by making movement itself a musical instrument, through wearable sensors, video pose estimation, and machine learning models that discover natural correspondences between motion and sound.
Our starting point is the Musical Flow Rope: a weighted rope with an IMU at the handles that streams motion into a live pipeline, discovers a small movement vocabulary, and turns each cycle into music through a DAW. The rope is a good first instrument because the object constrains the body into a limited, learnable vocabulary. Music in Motion generalizes that idea toward the space of free human movement, dance included.
The team
- Joseph Bakarji, PI, on supervision, mechatronics, machine learning, music composition, and systems integration.
- Olivier Chiniara, biomechanics and movement expert, on the vocabulary of the moving body (and the reason the rope-flow work exists in the first place).
- Elsa Maalouf, co-PI, dancer and chemical engineer at AUB, on dance as a full-body interface, machine learning.
- Samir Shaker, research assistant, on the live pipeline, app development.
We're also fortunate to be supported by students and collaborators, including: Joelle Khoury, on composition and performance (as part of the Science meets Arts internal stimulus fund), Nareg Karaoghlanian, on mechatronics and design, Khaled Yassine, an advisor on percussion and rhythm, Fouad Bechwati (21db), on acoustics and set design for the public performances, Khyam Allami, who has brought his deep expertise in Arabic music (having developed DiArMaqAr), Helene Jabbour, Mechanical Engineering MS student developing the glove instrument, dancers and musicians we're inviting to be involved in the design and testing of the instruments, and others.
The lab is also fortunate to be supported by AUB's Center for Advanced Mathematical Sciences (CAMS) on various initiatives related to music, mathematics, and machines. CAMS's ongoing institutional support has been an important enabler of this project.
Get involved
If you are working on any part of this space (motion sensing, movement-to-sound mapping, dance practice, real-time music systems), or if you are a musician, dancer, or engineer who would like to be involved, please reach out at joseph.bakarji@aub.edu.lb.
Co-funded by
the European Union
This Project is implemented with the support of the Arab Fund for Arts and Culture – AFAC, in partnership with Oxfam, Echos Electrik, and Megaphone and co-funded by the European Union.
This document has been produced with the financial assistance of the European Union. The contents of this document are the sole responsibility of Music Intelligence Lab, American University of Beirut & Olivier Chiniara, and can under no circumstances be regarded as reflecting the position of the European Union.