TRIAD: capturing harmonics with 3D convolutions

Perez, Miguel; Kirchhoff, Holger; Serra, Xavier

TRIAD: capturing harmonics with 3D convolutions

License

Citation

Perez, M, Kirchhoff, H, Serra, X. TRIAD: capturing harmonics with 3D convolutions. 2023. handle: http://hdl.handle.net/10230/58109

This citation was generated automatically.

Abstract

Thanks to advancements in deep learning (DL), automatic music transcription (AMT) systems recently outperformed previous ones fully based on manual feature design. Many of these highly capable DL models, however, are computationally expensive. Researchers are moving towards smaller models capable of maintaining state-ofthe- art (SOTA) results by embedding musical knowledge in the network architecture. Existing approaches employ convolutional blocks specifically designed to capture the harmonic structure. These approaches, however, require either large kernels or multiple kernels, with each kernel aiming to capture a different harmonic. We present TriAD, a convolutional block that achieves an unequally distanced dilation over the frequency axis. This allows our method to capture multiple harmonics with a single yet small kernel. We compare TriAD with other methods of capturing harmonics, and we observe that our approach maintains SOTA results while reducing the number of parameters required. We also conduct an ablation study showing that our proposed method effectively relies on harmonic information.

Description

This work has been accepted at the 24th International Society for Music Information Retrieval Conference (ISMIR 2023), at Milan, Italy. October 5-9, 2023.