[논문]Evaluating the Visualization of What a Deep Neural Network Has Learned

Samek, Wojciech; Binder, Alexander; Montavon, Gregoire; Lapuschkin, Sebastian; Muller, Klaus-Robert

doi:10.1109/TNNLS.2016.2599820

Evaluating the Visualization of What a Deep Neural Network Has Learned 원문보기

IEEE transactions on neural networks and learning systems, v.28 no.11, 2017년, pp.2660 - 2673

Samek, Wojciech , Binder, Alexander , Montavon, Gregoire , Lapuschkin, Sebastian , Muller, Klaus-Robert

Abstract ▼ AI-Helper

Deep neural networks (DNNs) have demonstrated impressive performance in complex machine learning tasks such as image classification or speech recognition. However, due to their multilayer nonlinear structure, they are not transparent, i.e., it is hard to grasp what makes them arrive at a particular classification or recognition decision, given a new unseen data sample. Recently, several approaches have been proposed enabling one to understand and interpret the reasoning embodied in a DNN for a single test image. These methods quantify the "importance" of individual pixels with respect to the classification decision and allow a visualization in terms of a heatmap in pixel/input space. While the usefulness of heatmaps can be judged subjectively by a human, an objective quality measure is missing. In this paper, we present a general methodology based on region perturbation for evaluating ordered collections of pixels such as heatmaps. We compare heatmaps computed by three different methods on the SUN397, ILSVRC2012, and MIT Places data sets. Our main result is that the recently proposed layer-wise relevance propagation algorithm qualitatively and quantitatively provides a better explanation of what made a DNN arrive at a particular classification decision than the sensitivity-based approach or the deconvolution method. We provide theoretical arguments to explain this result and discuss its practical implications. Finally, we investigate the use of heatmaps for unsupervised assessment of the neural network performance.

참고문헌 (42)

Itti, Laurent, Koch, Christof. Computational modelling of visual attention. Nature reviews. Neuroscience, vol.2, no.3, 194-203.

상세보기
Simoncelli, Eero P, Olshausen, Bruno A. NATURAL IMAGE STATISTICS AND NEURAL REPRESENTATION. Annual review of neuroscience, vol.24, 1193-1216.

상세보기
Rumelhart, David E., Hinton, Geoffrey E., Williams, Ronald J.. Learning representations by back-propagating errors. Nature, vol.323, no.6088, 533-536.

상세보기
Learning multiple layers of features from tiny images krizhevsky 2009
Proc Adv NIPS Learning deep features for scene recognition using places database zhou 2014 487
Russakovsky, Olga, Deng, Jia, Su, Hao, Krause, Jonathan, Satheesh, Sanjeev, Ma, Sean, Huang, Zhiheng, Karpathy, Andrej, Khosla, Aditya, Bernstein, Michael, Berg, Alexander C., Fei-Fei, Li. ImageNet Large Scale Visual Recognition Challenge. International journal of computer vision, vol.115, no.3, 211-252.

상세보기
Heeger, D J, Simoncelli, E P, Movshon, J A. Computational models of cortical visual processing.. Proceedings of the National Academy of Sciences of the United States of America, vol.93, no.2, 623-627.

상세보기
Itti, Laurent, Koch, Christof. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision research, vol.40, no.10, 1489-1506.

상세보기
Explaining nonlinear classification decisions with deep taylor decomposition montavon 2015
J Mach Learn Res The LRP toolbox for artificial neural networks lapuschkin 2016 17 1
Neural Networks Tricks of the Trade montavon 2012 10.1007/978-3-642-35289-8 7700
Caffe An Open Source Convolutional Architecture for Fast Feature Embedding jia 2013
Neural Networks Tricks of the Trade Efficient backprop lecun 2012 10.1007/978-3-642-35289-8_3 9

상세보기
J Mach Learn Res An introduction to variable and feature selection guyon 2003 3 1157
J Mach Learn Res On relevant dimensions in kernel feature spaces braun 2008 9 1875
J Mach Learn Res How to explain individual classification decisions baehrens 2010 11 1803

상세보기
Hansen, Katja, Baehrens, David, Schroeter, Timon, Rupp , Matthias, Müller , Klaus‐Robert. Visual Interpretation of Kernel‐Based Prediction Models. Molecular informatics, vol.30, no.9, 817-826.

상세보기
Montavon, G., Braun, M., Krueger, T., Muller, Klaus-Robert. Analyzing Local Structure in Kernel-Based Learning: Explanation, Complexity, and Reliability Assessment. IEEE signal processing magazine, vol.30, no.4, 62-74.

상세보기
Understanding representations learned in deep architectures erhan 2010
J Mach Learn Res Kernel analysis of deep networks montavon 2011 12 2563

상세보기
Proc ECCV Visualizing and understanding convolutional networks zeiler 2014 818
Bach, Sebastian, Binder, Alexander, Montavon, Grégoire, Klauschen, Frederick, Müller, Klaus-Robert, Samek, Wojciech. On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation. PloS one, vol.10, no.7, e0130140-.

상세보기
Proc IEEE CVPR Multi-column deep neural networks for image classification ciregan 2012 3642
10.1109/CVPR.2016.318
Going Deeper with Convolutions szegedy 2014
Proc EMNLP Recursive deep models for semantic compositionality over a sentiment treebank socher 2013 1631
10.1109/CVPR.2010.5539970
J Mach Learn Res Natural language processing (almost) from scratch collobert 2011 12 2493

상세보기
10.1109/CVPR.2011.5995496
Proc ICML 3D convolutional neural networks for human action recognition ji 2010 495
Proc Adv NIPS Deep neural networks segment neuronal membranes in electron microscopy images ciresan 2012 2852
Montavon, Grégoire, Rupp, Matthias, Gobre, Vivekanand, Vazquez-Mayagoitia, Alvaro, Hansen, Katja, Tkatchenko, Alexandre, Müller, Klaus-Robert, Anatole von Lilienfeld, O. Machine learning of molecular electronic properties in chemical compound space. New journal of physics, vol.15, no.9, 095003-.

상세보기
Proc Adv Neural Inf Process Syst (NIPS) ImageNet classification with deep convolutional neural networks krizhevsky 2012 25 1097
Understanding Deep Image Representations by Inverting Them mahendran 2014
Explaining and Harnessing Adversarial Examples goodfellow 2014
Intriguing properties of neural networks szegedy 2013
Deep neural networks are easily fooled High confidence predictions for unrecognizable images nguyen 2014
10.1109/CVPR.2016.522
Object detectors emerge in deep scene cnns zhou 2014
Understanding neural networks through deep visualization yosinski 2015
Proc ICLR Workshop Deep inside convolutional networks: Visualising image classification models and saliency maps simonyan 2014 1
Proc Biosignals Visualization of nonlinear classification models in neuroimaging—Signed sensitivity maps rasmussen 2012 254

LOADING...

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증

Evaluating the Visualization of What a Deep Neural Network Has Learned 원문보기

Abstract ▼ AI-Helper

참고문헌 (42)

이 논문을 인용한 문헌

연구과제 타임라인

관련 콘텐츠

원문 보기

원문 URL 링크

오픈액세스(OA) 유형

이 논문과 함께 이용한 콘텐츠

AI-Helper ※ AI-Helper는 오픈소스 모델을 사용합니다.

선택된 텍스트

연합인증

Evaluating the Visualization of What a Deep Neural Network Has Learned 원문보기

Abstract ▼ AI-Helper

참고문헌 (42)

이 논문을 인용한 문헌

연구과제 타임라인

전체(0) 논문(0) 특허(0) 보고서(0)

전체(0) 논문(0) 특허(0) 보고서(0)

관련 콘텐츠

원문 보기

원문 URL 링크

오픈액세스(OA) 유형

이 논문과 함께 이용한 콘텐츠

AI-Helper ※ AI-Helper는 오픈소스 모델을 사용합니다.

선택된 텍스트