Abstract: We propose a new method to generate explanations for end-to-end classification models. The explanations consist of meaningful features to the user, namely contextual features. We instantiate our approach in the scenario of automated electrocardiogram (ECG) diagnosis and analyze the explanations generated in terms of interpretability and robustness. The proposed method uses a noise-insertion strategy to quantify the impact of intervals and segments of the ECG signals on the automated classification outcome. These intervals and segments and their impact on the diagnosis are common place to cardiologists, and their usage in explanations enables a better understanding of the outcomes and also the identification of sources of mistakes. The proposed method is particularly effective and useful for modern deep learning models that take raw data as input. We demonstrate our method by explaining diagnoses generated by a deep convolutional neural network.