[논문]A Text Sentiment Classification Method Based on LSTM-CNN

Wang, Guangxing; Shin, Seong-Yoon; Lee, Won Joo

doi:10.9708/jksci.2019.24.12.001

초록
AI-Helper

머신 러닝의 심층 개발로 딥 러닝 방법은 특히 CNN(Convolution Neural Network)에서 큰 진전을 이루었다. 전통적인 텍스트 정서 분류 방법과 비교할 때 딥 러닝 기반 CNN은 복잡한 다중 레이블 및 다중 분류 실험의 텍스트 분류 및 처리에서 크게 발전하였다. 그러나 텍스트 정서 분류를 위한 신경망에도 문제가 있다. 이 논문에서는 LSTM (Long-Short Term Memory network) 및 CNN 딥 러닝 방법에 기반 한 융합 모델을 제안하고, 다중 카테고리 뉴스 데이터 세트에 적용하여 좋은 결과를 얻었다. 실험에 따르면 딥 러닝을 기반으로 한 융합 모델이 텍스트 정서 분류의 예측성과 정확성을 크게 개선하였다. 본 논문에서 제안한 방법은 모델을 최적화하고 그 모델의 성능을 개선하는 중요한 방법이 될 것이다.

Abstract ▼ AI-Helper

With the in-depth development of machine learning, the deep learning method has made great progress, especially with the Convolution Neural Network(CNN). Compared with traditional text sentiment classification methods, deep learning based CNNs have made great progress in text classification and proc...

With the in-depth development of machine learning, the deep learning method has made great progress, especially with the Convolution Neural Network(CNN). Compared with traditional text sentiment classification methods, deep learning based CNNs have made great progress in text classification and processing of complex multi-label and multi-classification experiments. However, there are also problems with the neural network for text sentiment classification. In this paper, we propose a fusion model based on Long-Short Term Memory networks(LSTM) and CNN deep learning methods, and applied to multi-category news datasets, and achieved good results. Experiments show that the fusion model based on deep learning has greatly improved the precision and accuracy of text sentiment classification. This method will become an important way to optimize the model and improve the performance of the model.

Keyword

표/그림 (6)

그림 Fig. 1. LSTM-CNN model structure
표 Table 1. LSTM-CNN Model Parameters Configuration Table
표 Table 2. Neural network model training and testing results. Where Improvement rate in test_acc is the percentage of the LSTM-CNN model test accuracy compared to the test accuracy of the corresponding model.
그림 Fig. 2. Confusion matrix for the prediction results of the four neural network models.
그림 Fig. 3. Training accuracy curve of the LSTM-CNN model.
그림 Fig. 4. Training loss rate graph of the LSTM-CNN model.

AI 본문요약
AI-Helper

* AI 자동 식별 결과로 적합하지 않은 문장이 있을 수 있으니, 이용에 유의하시기 바랍니다.

문제 정의

Natural language processing(NLP) has always been the focus of attention in the field of science and technology. The study of natural language processing helps scientists to extract useful features from natural language and analyze the meaning and sentiment expressed by these semantic features. Further explore and predict people's behavioral information.
This paper proposed a text sentiment classification method based on deep learning method model fusion. The method is compared with CNN, RNN, LSTM and two traditional text classification methods.

제안 방법

In this paper, we start with the basic neural network CNN model, briefly introduce the CNN model, LSTM model, and conduct text sentiment classification experiments based on THUC's THUCNews dataset.
Natural language processing is mainly used in spell checking, network information extraction, text sentiment classification, machine translation, spoken dialogue, and complex question and answer systems[3]. The research of NLP promotes the development of computer bionics, combining biological neural network systems with computer technology, enabling computer model biology neural network systems to input, feature, analyze, and output the result. Research on computer information processing models of neural network systems based on deep learning has become popular in recent years, including deep neural network (DNN), convolution neural network (CNN), recurrent neural network (RNN), and variants of RNN, that is, Long-Short Term Memory networks (LSTM).
The maxpooling layer extracts the most important feature information of the text, and finally performs the category output of the text sentiment through the Softmax classification function. Dropout is set between the maxpooling layer and the fully connected layer to prevent over-fitting, and it is beneficial to the rapid convergence of the model and improves the accuracy of classification. The parameters of the LSTM-CNN model are shown in Table 1.
The same data set is used for training and testing in CNN, RNN, LSTM neural networks. The model is evaluated in terms of the loss rate and accuracy of the training, the loss rate and accuracy of the testing, the training time, and the number of sample batches to achieve the best training accuracy, and the final result is presented graphically and in tabular form.
A comparative experiment of text categorization was performed on the dataset THUCNews. The proposed method is to fuse the convolution neural network with the long-short term memory network to form the LSTM-CNN neural network model. The model exploits the performance of long-short term memory networks to process time-text sequences, which solves the problem of gradient disappearance and gradient explosion caused by RNN model, and draws on the advantages of CNN feature extraction in convolution neural networks.
The proposed method is to fuse the convolution neural network with the long-short term memory network to form the LSTM-CNN neural network model. The model exploits the performance of long-short term memory networks to process time-text sequences, which solves the problem of gradient disappearance and gradient explosion caused by RNN model, and draws on the advantages of CNN feature extraction in convolution neural networks. Compared with CNN, RNN and LSTM models, the performance of LSTM-CNN model text sentiment classification has been further improved.
In this paper, we preprocessed the data and transformed text data into word vectors, which is also an important method to improve the prediction performance of the model. In addition, the model can be optimized from bidirectional LSTM, network depth, convolution kernel size, data expansion, gradient descent algorithm selection, etc.

대상 데이터

The dataset used in the experiment is the THUCNews dataset[6]. THUCNews dataset is based on the historical data filtering and filtering of Sina News RSS subscription channel from 2005 to 2011.
The dataset used in the experiment is the THUCNews dataset[6]. THUCNews dataset is based on the historical data filtering and filtering of Sina News RSS subscription channel from 2005 to 2011. It contains 74 million news documents (2.
THUCNews dataset is based on the historical data filtering and filtering of Sina News RSS subscription channel from 2005 to 2011. It contains 74 million news documents (2.19 GB), all in UTF-8 plain text format. After re-integration and division of 14 candidate classification categories.
The experimental data training has a batch size of 64, and each batch is trained in 100 batches, which are divided into 79 training batches. In the process of training, the best recording algorithm is used, that is, the performance of the current training result and the previous training result are checked and compared.

이론/모형

Secondly, the model training. The pre-trained data is entered into the LSTM-CNN model for training. In the LSTM-CNN model, the data is indexed by the word embedding layer, and then the LSTM unit is used to generate the time text sequence, and then enters the convolution layer and the maximum pooling layer to extract the text features.

성능/효과

The experimental result is shown in Table 2. From Table 2, it can be concluded that in this experiment the LSTM-CNN model performed best in terms of loss rate and accuracy, and the accuracy of the test was as high as 97.68%. The test accuracy is improved by 2.
84% compared to the RNN model. The optimal training accuracy and the lowest loss rate were achieved when the training reached 300 batches, while the training batches of other models achieving the optimal accuracy and the lowest loss rate were more than two thousand. The prediction results are much better than traditional classifiers.
00. From the confusion matrix of LSTM-CNN, it can be seen that the predicted classification of finance, science and technology, fashion, and game is the most accurate, reaching 1.00, while the classification of home life is the lowest, only 92%. Figs.
2) Neural network model is generally better for text sentiment classification. The accuracy of LSTM-CNN based on model fusion is up to 97.
3) The best training accuracy was achieved when LSTM-CNN was trained about 300 times. Figs.
4) LSTM-CNN confusion matrix are shown in Fig. 4 intuitively illustrates that the classification of finance, science and technology, fashion, and game is the best, reaching 100%. But the lowest classes of home life classification, only 92%.
Moreover, the LSTM-CNN model converges quickly and can obtain the best prediction accuracy faster, which greatly improves the efficiency of text classification. Compared with the traditional machine learning text sentiment classification method, the advantage of LSTM-CNN convolution neural network is more obvious, and the accuracy of the prediction has increased by 28.12%.

참고문헌 (6)

Chetan Arora, Mehrdad Sabetzadeh, Lionel Briand, Frank Zimmer, "Automated Checking of Conformance to Requirements Templates Using Natural Language Processing," IEEE Transactions on Software Engineering, Vol. 41, Issue 10, pp. 944-968, May, 2015. DOI: 10.1109/TSE.2015.2428709.

상세보기
Mohd Ibrahim, Rodina Ahmad, "Class Diagram Extraction from Textual Requirements Using Natural Language Processing (NLP) Techniques," 2010 Second International Conference on Computer Research and Development, pp. 200-204, 2010. DOI: 10.1109/ICCRD.2010.71.
Sweta P. Lende, M. M. Raghuwanshi, "Question answering system on education acts using NLP techniques," 2016 World Conference on Futuristic Trends in Research and Innovation for Social Welfare(Startup Conclave), pp. 1-6, 2016. DOI: 10.1109/STARTUP.2016.7583963.
C. Janarish Saju, A. S. Shaja, "A Survey on Efficient Extraction of Named Entities from New Domains Using Big Data Analytics," 2017 Second International Conference on Recent Trends and Challenges in Computational Models (ICRTCCM), pp. 170-175, 2017. DOI: 10.1109/ICRTCCM.2017.34.
Tom Young, Devamanyu Hazarika, Soujanya Poria, Erik Cambria, "Recent Trends in Deep Learning Based Natural Language Processing," IEEE Computational Intelligence Magazine, Vol. 13, pp. 55-75, Nov. 2018. DOI: 10.1109/MCI.2018.2840738.

상세보기
THUCTC: An efficient Chinese text classification toolkit. [Online]. Available: http://thuctc.thunlp.org.

이 논문을 인용한 문헌

활용도 분석정보

상세보기

다운로드

내보내기

활용도 Top5 논문

해당 논문의 주제분야에서 활용도가 높은 상위 5개 콘텐츠를 보여줍니다.
더보기 버튼을 클릭하시면 더 많은 관련자료를 살펴볼 수 있습니다.

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증

[국내논문] A Text Sentiment Classification Method Based on LSTM-CNN 원문보기

초록
AI-Helper

Abstract ▼ AI-Helper

Keyword

표/그림 (6)

표/그림 (6)

AI 본문요약
AI-Helper

문제 정의

제안 방법

대상 데이터

이론/모형

성능/효과

참고문헌 (6)

이 논문을 인용한 문헌

활용도 분석정보

활용도 Top5 논문

관련 콘텐츠

원문 보기

원문 URL 링크

오픈액세스(OA) 유형

이 논문과 함께 이용한 콘텐츠

AI-Helper ※ AI-Helper는 오픈소스 모델을 사용합니다.

선택된 텍스트

연합인증

[국내논문] A Text Sentiment Classification Method Based on LSTM-CNN 원문보기

초록 용어보기논문에서 용어와 풀이말을 자동 추출한 결과로, 시범 서비스 중입니다. AI-Helper

Abstract ▼ AI-Helper

Keyword

표/그림 (6) 모든 표/그림 보기

표/그림 (6) 슬라이드로 보기

AI 본문요약 엑셀 다운로드 AI-Helper

문제 정의

제안 방법

대상 데이터

이론/모형

성능/효과

참고문헌 (6)

이 논문을 인용한 문헌

활용도 분석정보

활용도 Top5 논문 더보기

관련 콘텐츠

원문 보기

원문 URL 링크

오픈액세스(OA) 유형

이 논문과 함께 이용한 콘텐츠

AI-Helper ※ AI-Helper는 오픈소스 모델을 사용합니다.

선택된 텍스트

초록
AI-Helper

표/그림 (6)

표/그림 (6)

AI 본문요약
AI-Helper

활용도 Top5 논문