Predicting and Critiquing Machine Virtuosity: Mawwal Accompaniment as Case Study
Résumé
The evaluation of machine virtuosity is critical to improving the quality of virtual instruments, and may also help predict future impact. In this contribution, we evaluate and predict the virtuosity of a statistical machine translation model that provides an automatic responsive accompaniment to mawwal, a genre of Arab vocal improvisation. As an objective evaluation used in natural language processing (BLEU score) did not adequately assess the model's output, we focused on subjective evaluation. First, we culturally locate virtuosity within the particular Arab context of tarab, or modal ecstasy. We then analyze listening test evaluations, which suggest that the corpus size needs to increase to 18K for machine and human accompaniment to be comparable. We also posit that the relationship between quality and inter-evaluator disagreement follows a higher order polynomial function. Finally, we gather suggestions from a musician in a user experience study for improving machine-induced tarab. We were able to infer that the machine's lack of integration into tarab may be due, in part, to its dependence on a tri-gram language model, and instead suggest using a four-or five-gram model. In the conclusion, we note the limitations of language models for music translation.
Origine : Fichiers produits par l'(les) auteur(s)