[논문]Artificial Intelligence-based Echocardiogram Video Classification by Aggregating Dynamic Information

Ye, Zi; Kumar, Yogan J.; Sing, Goh O.; Song, Fengyan; Ni, Xianda; Wang, Jin

doi:10.3837/tiis.2021.02.007

Artificial Intelligence-based Echocardiogram Video Classification by Aggregating Dynamic Information 원문보기

KSII Transactions on internet and information systems : TIIS, v.15 no.2, 2021년, pp.500 - 521

Ye, Zi (Department of Information Technology, Wenzhou Polytechnic) , Kumar, Yogan J. (Centre for Advanced Computing Technology, Faculty of Information and Communication Technology Universiti Teknikal Malaysia Melaka) , Sing, Goh O. (Centre for Advanced Computing Technology, Faculty of Information and Communication Technology Universiti Teknikal Malaysia Melaka) , Song, Fengyan (Shanghai Gen Cong Information Technology Co. Ltd) , Ni, Xianda (Department of Ultrasonography, the First Affiliated Hospital of Wenzhou Medical University) , Wang, Jin (School of Computer & Communication Engineering, Changsha University of Science & Technology)

Abstract ▼ AI-Helper

Echocardiography, an ultrasound scan of the heart, is regarded as the primary physiological test for heart disease diagnoses. How an echocardiogram is interpreted also relies intensively on the determination of the view. Some of such views are identified as standard views because of the presentation and ease of the evaluations of the major cardiac structures of them. However, finding valid cardiac views has traditionally been time-consuming, and a laborious process because medical imaging is interpreted manually by the specialist. Therefore, this study aims to speed up the diagnosis process and reduce diagnostic error by providing an automated identification of standard cardiac views based on deep learning technology. More importantly, based on a brand-new echocardiogram dataset of the Asian race, our research considers and assesses some new neural network architectures driven by action recognition in video. Finally, the research concludes and verifies that these methods aggregating dynamic information will receive a stronger classification effect.

주제어

표/그림 (16)

그림 Fig. 1. In transthoracic echocardiography, nine cardiac views are acquired at arbitrary stages of the heart cycle. Examples of PLAX, PSAX-AV, PSAX-AP, PSAX-MV, PSAX-MID, A2C, A3C, A4C, and A5C, as well as non-assignable samples labeled 'OTHERS' are illustrated.
그림 Fig. 2. The three classes of neural network architectures used in this study
그림 Fig. 3. The procedure of video classification for classic 2D CNNs
그림 Fig. 4. The structure of CNN+BiLSTM architectures for video classification
그림 Fig. 5. The structure of Spatiotemporal-BiLSTM architecture for video classification
표 Table 1. The overview of the dataset
그림 Fig. 6. The numbers of patients for every ten viewpoints in the database and applied for training, validation, and testing correspondingly
그림 Fig. 7. The numbers of videos for every ten viewpoints in the database and applied for training, validation, and testing correspondingly
그림 Fig. 8. Procedures used for removing the class label and segmenting relevant visual structures
표 Table 2. Table demonstrating the overall accuracy for each of the networks on the test set
그림 Fig. 9. Echocardiogram view classification through Xception. Confusional matrix demonstrating actual view labels on the y-axis, and neural network-predicted view labels on the x-axis by view category for video classification
그림 Fig. 10. Echocardiogram view classification by Xception+BiLSTM
그림 Fig. 11. Echocardiogram view classification by Two-Stream Model
그림 Fig. 12. Examples of the short-axis view of the papillary muscle, and that of the apical level
표 Table 3. Classification accuracy on various video frames
표 Table 4. Comparison of CNN+BiLSTM structure and Spatiotemporal-BiLSTM structure

AI 본문요약
AI-Helper

제안 방법

In this paper, we first employed classic neural networks that were proven successful in quite a few different fields of medical image classification to classify individual video frames. However, these methods omitted the ability to use dynamic knowledge about how features such as ventricular walls or heart valves behave during the cardiac cycle.
To assist echocardiographers in speeding up the diagnosis process by improving the use of echocardiography for precision medicine, the key objective of this research is to improve supervised deep learning to assess the capability of the state-of-the-art scene understanding methods proposed in computer vision to identify the standard cardiac views being visualized in echocardiograms.

대상 데이터

Methods were performed following relevant regulations and guidelines. There were echocardiogram studies from 267 patients being chosen randomly. Besides, 7994 were extracted from the hospital's echocardiogram database in DICOM format.

이론/모형

The CNN+BiLSTM structure comprised the trained classic 2D network which placed multiple frames of each echocardiogram. Before training, the CNN part used for feature extraction and representation received model weights from their trained Classic 2D CNNs models, whereas the Uniform Initialization was used on the Bi-LSTM network part.

참고문헌 (32)

H. Chen, D. Ni, J. Qin, S. Li, X. Yang, T. Wang, and P. A. Heng, "Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks," IEEE Journal Biomedical Health Informatics, vol. 19, no. 5, pp. 1627-1636, Sep. 2015.

상세보기
R. G. Dantas, E. T. Costa, and S. Leeman, "Ultrasound speckle and equivalent scatterers," Ultrasonics, vol. 43, no. 6, pp. 405-420, May 2005.

상세보기
X. Gao, W. Li, M. Loomes, and L. Wang, "A fused deep learning architecture for viewpoint classification of echocardiography," Inform Fusion, vol. 36, pp. 103-113, July 2017.

상세보기
A. M. El Missiri, K. A. L. El Meniawy, S. A. S. Sakr, and A. S. E. D. Mohamed, "Normal reference values of echocardiographic measurements in young Egyptian adults," Egypt Heart Journal, vol. 68, no. 4, pp. 209-215, Dec. 2016.

상세보기
A. A. M. Jamel and B. Akay, "A Survey and systematic categorization of parallel K-means and Fuzzy-c-Means algorithms," Computer Systems Science and Engineering, vol. 34, no. 5, pp. 259-281, Sep. 2019.

상세보기
L. Aguilar, S. W. Nava-Diaz, and G. Chavira, "Implementation of decision trees as an alternative for the support in the decision-making within an intelligent system in order to automatize the regulation of the VOCs in non-industrial inside environments," Computer Systems Science Engineering, vol. 34, no. 5, pp. 297-303, 2019.

상세보기
Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, May 2015.

상세보기
F. Chollet, "Xception: Deep learning with depthwise separable convolutions," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 1251-1258, 2017.
X. Hong, X. Zheng, J. Xia, L. Wei, and W. Xue, "Cross-Lingual Non-Ferrous Metals Related News Recognition Method Based on CNN with A Limited Bi-Lingual Dictionary," Computers, Materials, and Contiuna, vol. 58, no. 2, pp. 379-389, 2019.
S. Y. Tan, "Automated Interpretation of Echocardiograms Technical Milestone Report," to be published.
A. Ostvik, E. Smistad, S. A. Aase, B. O. Haugen, and L. Lovstakken, "Real-time standard view classification in transthoracic echocardiography using convolutional neural networks," Ultrasound in Medicine and Biology, vol. 45, no. 2, pp. 374-384, Feb. 2019.

상세보기
J. P. Howard, J. Tan, M. J. Shun-Shin, D. Mahdi, A. N. Nowbar, A. D. Arnold, Y. Ahmad, P. McCartney, M. Zolgharni, N. W. F. Linton, N. Sutaria, B. Rana, J. Mayet, D. Rueckert, G. D. Cole, and D. P. Francis, "Improving ultrasound video classification: an evaluation of novel deep learning methods in echocardiography," Journal of Medical Artificial Intelligence, vol. 3, no. 4, Mar. 2020.
S. Ebadollahi, S. F. Chang, and H. Wu, "Automatic view recognition in echocardiogram videos using parts-based representation," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR), pp. 2-9, 2004.
G. N. Balaji, T. S. Subashini, and N. Chidambaram, "Automatic classification of cardiac views in echocardiogram using histogram and statistical features," Procedia Computer Science, vol. 46, pp.1569-1576, 2015.

상세보기
H. Khamis, G. Zurakhov, V. Azar, A. Raz, Z. Friedman, and D. Adam, "Automatic apical view classification of echocardiograms using a discriminative learning dictionary," Medical Image Analysis, vol. 36, pp. 15-21, Feb. 2017.

상세보기
Z. Liu, B. Xiang, Y. Song, H. Lu, and Q. Liu, "An Improved Unsupervised Image Segmentation Method Based on Multi-Objective Particle, Swarm Optimization Clustering Algorithm," Computers, Materials and Continua, vol. 58, no. 2, pp. 451-461, 2019.
M. Long and Y. Zeng, "Detecting Iris Liveness with Batch Normalized ConVolutional Neural Network," Computers, Materials and Continua, vol. 58, no. 2, pp. 493-504, 2019.
W. Lu, L. Quan, and L. Ping, "Image Classification using Optimized MKL for SSPM," Intelligent Automation and Soft Computing, vol. 25, no. 2, pp. 249-257, 2019.
A. Madani, R. Arnaout, M. Mofrad, and R. Arnaout, "Fast and accurate view classification of echocardiograms using deep learning," NPJ Digital Medicine, vol. 1, no. 1, pp. 1-8, Mar. 2018.

상세보기
J. Zhang, S. Gajjala, P. Agrawal, G. H. Tison, L. A. Hallock, L. Beussink-Nelson, M. H. Lassen, E. Fan, M. A. Aras, C. Jordan, K. E. Fleischmann, M. Melisko, A. Qasim, S. J. Shah, R. Bajcsy, and R. C. Deo, "Fully automated echocardiogram interpretation in clinical practice: feasibility and diagnostic accuracy," Circulation, vol. 138, no. 16, pp. 1623-1635, Sep. 2018.

상세보기
K. Simonyan and A. Zisserman, "Two-stream convolutional networks for action recognition in videos," Advances in Neural Information Processing Systems, pp. 568-576, 2014.
T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, "High accuracy optical flow estimation based on a theory for warping," in Proc. of Eupopean Conference on Computer Vision, pp. 25-36, 2004.
A. Sengupta, Y. Ye, R. Wang, C. Liu, and K. Roy, "Going deeper in spiking neural networks: Vgg and residual architectures," Fronters in Neuroscienc, vol. 13, Mar. 2019.
C. Szegedy, S. Ioffe, V. Vanhoucke, and A. Alemi, "Inception-v4, inception-resnet and the impact of residual connections on learning," arXiv preprint arXiv:1602.07261, Aug. 2016.
C. Zhang and Y. Tian, "Automatic video description generation via lstm with joint two-stream encoding," in Proc. of the 23rd International Conference on Pattern Recognition(ICPR), pp. 2924-2929, 2016.
H. Wu, Q. Liu, and X. Liu, "A review on deep learning approaches to Image classification And object segmentation," Computers, Maerial and Continua, vol. 60, no. 2, pp. 575-597, 2019.
S. Zhou, L. Chen, and V. Sugumaran, "Hidden two-stream collaborative learning network for action recognition," Computers, Maerial and Continua, vol. 63, no. 3, pp. 1545-1561, 2020.

상세보기
A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Communications of the ACM, vol. 60, no. 6, pp. 84-90, May 2017.

상세보기
D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," in Proc. of the 3rd International Conference for Learning Representations, 2015.
Z. Xu, Q. Zhou, and Z. Yan, "Special Section on Recent Advances in Artificial Intelligence for Smart Manufacturing - Part I," Intellignet Automaiton and Soft Computing, vol. 25, no. 4, pp. 693-694, 2019.
F. Duran and M. Teke, "Design and Implementation of an Intelligent Ultrasonic Cleaning Device," Intellignet Automaiton and Soft Computing, vol. 25, no. 3, pp. 441-449, 2019.
D. Ouyang, B. He, A. Ghorbani, N. Yuan, J. Ebinger, C. P. Langlotz, P. A. Heidenreich, R. A. Harrington, D. H. Liang, E. A. Ashley, and J. Y. Zou, "Video-based AI for beat-to-beat assessment of cardiac function," Nature, vol. 580, pp. 252-256, Mar. 2020.

상세보기

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증