This paper proposes an audio event classification method based on convolutional neural networks (CNNs). CNN has great advantages of distinguishing complex shapes of image. Proposed system uses the features of audio sound as an input image of CNN. Mel scale filter bank features are extracted from eac...
This paper proposes an audio event classification method based on convolutional neural networks (CNNs). CNN has great advantages of distinguishing complex shapes of image. Proposed system uses the features of audio sound as an input image of CNN. Mel scale filter bank features are extracted from each frame, then the features are concatenated over 40 consecutive frames and as a result, the concatenated frames are regarded as an input image. The output layer of CNN generates probabilities of audio event (e.g. dogs bark, siren, forest). The event probabilities for all images in an audio segment are accumulated, then the audio event having the highest accumulated probability is determined to be the classification result. This proposed method classified thirty audio events with the accuracy of 81.5% for the UrbanSound8K, BBC Sound FX, DCASE2016, and FREESOUND dataset.
This paper proposes an audio event classification method based on convolutional neural networks (CNNs). CNN has great advantages of distinguishing complex shapes of image. Proposed system uses the features of audio sound as an input image of CNN. Mel scale filter bank features are extracted from each frame, then the features are concatenated over 40 consecutive frames and as a result, the concatenated frames are regarded as an input image. The output layer of CNN generates probabilities of audio event (e.g. dogs bark, siren, forest). The event probabilities for all images in an audio segment are accumulated, then the audio event having the highest accumulated probability is determined to be the classification result. This proposed method classified thirty audio events with the accuracy of 81.5% for the UrbanSound8K, BBC Sound FX, DCASE2016, and FREESOUND dataset.
* AI 자동 식별 결과로 적합하지 않은 문장이 있을 수 있으니, 이용에 유의하시기 바랍니다.
제안 방법
There was a study on distinguishing four different events: traffic, music, crowd, and applause using an RBM [10]. In this study, RBM was trained so that the hidden layer could generate an output feature vector for an input feature vector; then the feed forward neural network (FFNN) was configured for the highest output layer so that the sound event could be produced. DNNs using RBM were evaluated together with SVM and GMM, and it was found that RBM had better classification performance than GMM and SVM.
In this study, a CNN-based audio event classifier was proposed and its performance was verified through experimentation. Proposed system used the features of audio sound as an input image of CNN.
대상 데이터
Tagging results show that a total of 160 audio events exist. Among them, 10 audio events, which belong to the events selected in Table 1 and contain enough quantity to be used as training data, are selected, and the corresponding tagging results are selected as training data.
One-tenth of the training data is used as validation data. Test data consists of a total of 600 audio segments by selecting 20 segments-per-event from all the datasets. All the data are converted to 16 kHz sampling and 16 bit-mono.
The data corresponding to the list of the audio events in Table 1 were collected from four datasets: UrbanSound8K, BBC Sound FX, DCASE2016, FREESOUND.
To conduct audio event analysis in as many segments as possible, the output class of CNNs should match the sounds from video as much as possible. To analyze this, total 11 hours of youtube videos were collected and audio events are tagged by annotators. Afterwards, 30 events were selected in the order of highest occurrence.
이론/모형
However, as there are cases with various events, the event sequence was classified. After modeling each sound event by GMM, a 3-state hidden markov model (HMM) was used to classify the sound event sequence. While this model exhibits high performance when sound events appear in a sequence, it is difficult to collect training data with answer scripts.
Segment level accuracy indicates the segment level accuracy for 600 segments. Segment level classification tests were conducted by using two decision making methods: probability accumulation method and voting method. In the probability accumulation method, the event probability of an audio segment is calculated as the accumulation of frame level probabilities over all frames.
성능/효과
Supervised learning is finally performed for output layers only using a small amount of learning data with answer scripts. According to the experimental results, an approximately 73 % classification performance was observed in the five music genre classification experiments, and an approximately 80 % classification performance was observed in the four musician classification experiments. This study shows the advantage of using a large amount of training data without answer scripts.
In this study, RBM was trained so that the hidden layer could generate an output feature vector for an input feature vector; then the feed forward neural network (FFNN) was configured for the highest output layer so that the sound event could be produced. DNNs using RBM were evaluated together with SVM and GMM, and it was found that RBM had better classification performance than GMM and SVM. This study also had a limitation that the number of sound events was small.
The performance of DNN-based and CNN-based classifiers with various model structures were measured. Experimental results exhibited a maximum performance of 81.5 %, which is approximately 20 % higher than the performance of the baseline classifier.
Mel scale filter bank features were extracted from each frame, then the concatenated features over 40 consecutive frames were regarded as an input image. The event probabilities of output layer of CNN for all images in an audio segment were accumulated, then the audio event having the highest accumulated probability was determined to be the classification result. The performance of DNN-based and CNN-based classifiers with various model structures were measured.
When one sample input was classified, the probabilities of each event for all the frames were added, and the event with the highest probability was determined as the classification result. The number of audio events was 10, and the audio event classification accuracy of sample units showed more than 70 % performance showing better performance than SVM.
참고문헌 (22)
K. Kim and H. Kim, "Storytelling Strategy of Visual-Image Contents base on Rhetoric Metaphors," Journal of Digital Content Society, vol. 14, no. 4, pp. 481-491, December, 2013.
L. Lu, H. Jiang and H. Zhang, "A robust audio classification and segmentation method," in Proc. of ACM International Conference on Multimedia, pp. 203-211, September 30-October 5, 2001.
M. Xu, N. Maddage, C. Xu, M. Kankanhalli and Q. Tian, "Creating audio keywords for event detection in soccer video," in Proc. of IEEE International Conference on Multimedia and Expo, pp.281-284, July 6-9, 2003.
W. Cheng, W. Chu and J. Wu, "Semantic context detection based on hierarchical audio models," in Proc. of ACM SIGMM International Workshop on Multimedia Information Retrieval, pp.109-115, November 7-7, 2003.
H. Lee, P. Pham, Y. Largman and Y. Ng, "Unsupervised feature learning for audio classification using convolutional deep belief networks," in Proc. of Advances in Neural Information Processing Systems, pp.1096-1104, December 7-10, 2009.
Y. Bengio and Y. LeCun, "Large-scale Kernel Machines," MIT Press, 2007.
J. Portelo, M. Bugalho, I. Trancoso, J. Neto, A. Abad and A. Serralheiro, "Non-speech audio event detection," in Proc. of Internationa Conference on Acoustics, Speech and Signal Processing, pp.1973-1976, April 19-24, 2009.
L. Ballan, A. Bazzica and M. Bertini, A. Bimbo, and G. Serra, "Deep networks for audio event classification in soccer videos," in Proc. of International Conference on Multimedia and Expo, pp.474-477, June 28-3, 2009.
T. Heittola, A. Mesaros, A. Eronen and T. Virtanen, "Context-dependent sound event detection," EURASIP Journal on Audio, Speech, and Music Processing, vol.1, pp.1-13, January, 2013.
K. Zvi and T. Orith, "Audio event classification using deep neural networks," in Proc. of Interspeech, pp.1482-1486, August 25-29, 2013.
S. Downie, et al., "The Music Information Retrieval Evaluation eXchange: Some observations and insights," Advances in Music Information Retrieval, pp. 93-115, 2010.
R. Malkin, "Multimodal Technologies for Perception of Humans," Springer, pp. 323-330, 2007.
F. Smeaton, et al., "Evaluation campaigns and TRECVid," in Proc. of ACM International Workshop on Multimedia Information Retrieval, pp. 321-330, 2006.
E. Vincent, et al., "The signal separation evaluation campaign (2007-2010): Achievements and remaining challenges," Signal Processing, vol. 82, no. 8, pp. 1928-1936, 2012.
H. Larochelle, et al., "An empirical evaluation of deep architectures on problems with many factors of variation," in Proc. of International Conference on Machine Learning, pp.473-480, 2007.
M. Lim and J. Kim, "Audio Event Classification Using Deep Neural Networks," Phonetics and Speech Sciences, vol. 7, no. 4, pp.27-33, January, 2015.
J. Salamon, C. Jacoby and J. Bello, "A dataset and taxonomy for urban sound research," in Proc. of ACM International Conference on Multimedia, pp.1041-1044, November 3-7, 2014.
M. Slaney, "Semantic-audio retrieval," in Proc. of International Conference on Acoustics, Speech and Signal Processing, pp.1408-1411, May 13-17, 2002.
A. Mesaros, T. Heittola, and T. Virtanen, "TUT database for acoustic scene classification and sound event detection," in Proc. of 24th European Signal Processing Conference, pp. 1128-1132, 2016.
S. Young, G. Evermann, M. Gales and P. Woodland, "The HTK book (for HTK version 3.4)," Entropic Cambridge Research Laboratory, 2006.
M. Abadi, A. Agarwal, et al, "Tensorflow: Large-scale machine learning on heterogeneous distributed systems," 2016, Preprint at.
Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, pp.436-444, May, 2015.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.