Abstract:
In this paper, we propose a new evaluation metric for sound event detection (SED) and discuss a problem frequently encountered in conventional metrics. In conventional evaluation metrics, misdetected sound events are treated equally, e.g., the misdetected sound event “car” in the acoustic scenes “office” and “street” are regarded as the same type of misdetection. However, the misdetected event“car” in “office”is as evere mistake compared with its misdetection in“street.” The event “car” rarely occurs in the “office.” SED systems that are evaluated using conventional metrics may cause severe/catastrophic problems and lead to confusion in practice owing to lack of consideration of the relationship between sound events and scenes. Our evaluation metric for SED considers severe misdetections on the basis of the relationship between sound events and scenes. We demonstrate the utility of our proposed method by com-paring it with the conventional evaluation metrics on two datasets with events and scenes. Experimental results show that the pro-posed metric can accurately evaluate whether SED systems appropriately consider the relationship between sound events and scenes,i.e., realistic situations.
