[논문]Zero-Shot 기반 기계번역 품질 예측 연구

어수경; 박찬준; 서재형; 문현석; 임희석

doi:10.15207/jkcs.2021.12.11.035

Zero-Shot 기반 기계번역 품질 예측 연구
Study on Zero-shot based Quality Estimation 원문보기

한국융합학회논문지 = Journal of the Korea Convergence Society, v.12 no.11, 2021년, pp.35 - 43

어수경 (고려대학교 컴퓨터학과) , 박찬준 (고려대학교 컴퓨터학과) , 서재형 (고려대학교 컴퓨터학과) , 문현석 (고려대학교 컴퓨터학과) , 임희석 (고려대학교 컴퓨터학과)

초록
AI-Helper

최근 다언어모델(Cross-lingual language model)을 활용하여 한 번도 보지 못한 특정 언어의 하위 태스크를 수행하는 제로샷 교차언어 전이(Zero-shot cross-lingual transfer)에 대한 관심이 증가하고 있다. 본 논문은 기계번역 품질 예측(Quality Estimation, QE)을 학습하기 위한 데이터 구축적 측면에서의 한계점을 지적하고, 데이터를 구축하기 어려운 상황에서도 QE를 수행할 수 있도록 제로샷 교차언어 전이를 수행한다. QE에서 제로샷을 다룬 연구는 드물며, 본 논문에서는 교차언어모델을 활용하여 영어-독일어 QE 데이터에 대해 미세조정을 실시한 후 다른 언어쌍으로의 제로샷 전이를 진행했고 이 과정에서 다양한 다언어모델을 활용하여 비교 연구를 수행했다. 또한 다양한 자원 크기로 구성된 언어쌍에 대해 제로샷 실험을 진행하고 실험 결과에 대해 언어별 언어학적 특성 관점으로의 분석을 수행하였다. 실험결과 multilingual BART와 multillingual BERT에서 가장 높은 성능을 보였으며, 특정 언어쌍에 대해 QE 학습을 전혀 진행하지 않은 상황에서도 QE를 수행할 수 있도록 유도하였다.

Abstract ▼ AI-Helper

Recently, there has been a growing interest in zero-shot cross-lingual transfer, which leverages cross-lingual language models (CLLMs) to perform downstream tasks that are not trained in a specific language. In this paper, we point out the limitations of the data-centric aspect of quality estimation (QE), and perform zero-shot cross-lingual transfer even in environments where it is difficult to construct QE data. Few studies have dealt with zero-shots in QE, and after fine-tuning the English-German QE dataset, we perform zero-shot transfer leveraging CLLMs. We conduct comparative analysis between various CLLMs. We also perform zero-shot transfer on language pairs with different sized resources and analyze results based on the linguistic characteristics of each language. Experimental results showed the highest performance in multilingual BART and multillingual BERT, and we induced QE to be performed even when QE learning for a specific language pair was not performed at all.

주제어

표/그림 (5)

그림 Fig. 1. Overall architecture of zero-shot transfer
표 Table 1. Comparison of zero-shot transfer performance of various cross-lingual language models for each language pair. The Pearson correlation coefficient for EN-DE is the result of fine-tuning, not zero-shot
표 Table 2. Zero-shot transfer results for cross-lingual language models by language. The Pearson correlation coefficient for EN-DE is the result of fine-tuning, as shown in Table 1.
표 Table 3. Comparison of language families, word order, and alphabets by language
표 Table 4. Comparison of zero-shot cross-lingual transfer results and fine-tuning results

참고문헌 (33)

T. Ranasinghe, C. Orasan & R. Mitkov. (2020). TransQuest at WMT2020: Sentence-Level Direct Assessment. arXiv preprint arXiv:2010.05318.
Z. Chi, L. Dong, S. Ma, S. H. X. L. Mao, H. Huang & F. Wei. (2021). mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs. arXiv preprint arXiv:2104.08692.
T. Pires, E. Schlinger & D. Garrette. (2019). How multilingual is multilingual BERT?. arXiv preprint arXiv:1906.01502. DOI : 10.18653/v1/p19-1493
G. Chen et al. (2021). Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders. arXiv preprint arXiv:2104.08757.
L. Specia, K. Shah, J. G. De Souza & T. Cohn (2013). QuEst-A translation quality estimation framework. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 79-84.
K. Papineni, S. Roukos, T. Ward & W. J. Zhu. (2002). Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 311-318. DOI : 10.3115/1073083.1073135
S. Banerjee & A. Lavie. (2005). METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, 65-72. DOI : 10.3115/1626355.1626389
S. Eo, C. Park, H. Moon, J. Seo & H. Lim. (2021). Dealing with the Paradox of Quality Estimation. In Proceedings of the 4rd Workshop on Technologies for MT of Low Resource Languages, 1-10.
G. Lample & A. Conneau. (2019). Cross-lingual language model pretraining. arXiv preprint arXiv:1901.07291.
A. Conneau et al. (2019). Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116. DOI : 10.18653/v1/P19-4007
J. Devlin, M. W. Chang, K. Lee & K. Toutanova. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. DOI : 10.18653/v1/N19-1423
Y. Liu et al. (2020). Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8, 726-742.

상세보기
J. Hu, S. Ruder, A. Siddhant, G. Neubig, O. Firat & M. Johnson. (2020, November). Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation. In International Conference on Machine Learning, 4411-4421.
G. Campagna, A. Foryciarz, M. Moradshahi & M. S. Lam. (2020). Zero-shot transfer learning with synthesized data for multi-domain dialogue state tracking. arXiv preprint arXiv:2005.00891. DOI : 10.18653/v1/2020.acl-main.12
A. Lauscher, V. Ravishankar, I. Vulic & G. Glavas, (2020). From zero to hero: On the limitations of zero-shot cross-lingual transfer with multilingual transformers. arXiv preprint arXiv:2005.00633. DOI : 10.18653/v1/2020.emnlp-main.363
L. Zhou, L. Ding & K. Takeda. (2020). Zero-shot translation quality estimation with explicit cross-lingual patterns. arXiv preprint arXiv:2010.04989.
Z. Chi, L. Dong, F. Wei, W. Wang, X. L. Mao & H. Huang. (2020). Cross-lingual natural language generation via pre-training. In Proceedings of the AAAI Conference on Artificial Intelligence, 7570-7577. DOI : 10.1609/aaai.v34i05.6256

상세보기
L. Specia, D. Raj & M. Turchi (2010). Machine translation evaluation versus quality estimation. Machine translation, 24(1), 39-50. DOI : 10.1007/s10590-010-9077-2

상세보기
D. Lee. (2020). Cross-lingual transformers for neural automatic post-editing. In Proceedings of the Fifth Conference on Machine Translation, 772-776.
K. Shah, T. Cohn & L. Specia. (2015). A bayesian non-linear method for feature selection in machine translation quality estimation. Machine Translation, 29(2), 101-125. DOI : 10.1007/s10590-014-9164-x

상세보기
R. Soricut, N. Bach & Z. Wang. (2012). The SDL language weaver systems in the WMT12 quality estimation shared task. In Proceedings of the Seventh Workshop on Statistical Machine Translation, 145-151.
H. Kim, J. H. Lee & S. H. Na. (2017). Predictor-estimator using multilevel task learning with stack propagation for neural quality estimation. In Proceedings of the Second Conference on Machine Translation, 562-568. DOI : 10.18653/v1/W17-4763
R. N. Patel. (2016). Translation quality estimation using recurrent neural network. arXiv preprint arXiv:1610.04841. DOI : 10.18653/v1/W16-2389
H. Kim, J. H. Lim, H. K. Kim & S. H. Na. (2019). QE BERT: bilingual BERT using multi-task learning for neural quality estimation. In Proceedings of the Fourth Conference on Machine Translation (3), 85-89. DOI : 10.18653/v1/W19-5407
S. Eo, C. Park, H. Moon, J. Seo & H. Lim. (2021). Research on Recent Quality Estimation. Journal of the Korea Convergence Society, 12(7), 37-44.

원문보기 상세보기
M. Wang et al. (2020). Hw-tsc's participation at wmt 2020 quality estimation shared task. In Proceedings of the Fifth Conference on Machine Translation, 1056-1061.
S. Eo, C. Park, H. Moon, J. Seo & H. Lim. (2021). Comparative Analysis of Current Approaches to Quality Estimation for Neural Machine Translation. Applied Sciences, 11(14), 6584. DOI : 10.3390/app11146584

상세보기
T. Wolf et al. (2019). Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.
C. Park, Y. Yang, K. Park & H. Lim. (2020). Decoding strategies for improving low-resource machine translation. Electronics, 9(10), 1562. DOI : 10.3390/electronics9101562

상세보기
C. Park, S. Eo, H. Moon & H. S. Lim. (2021). Should we find another model?: Improving Neural Machine Translation Performance with ONE-Piece Tokenization Method without Model Modification. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers, 97-104. DOI : 10.18653/v1/2021.naacl-industry.13
C. Park, J. Seo, S. Lee, C. Lee, H. Moon, S. Eo & H. Lim. (2021). BTS: Back TranScription for Speech-to-Text Post-Processor using Text-to-Speech-to-Text. In Proceedings of the 8th Workshop on Asian Translation, 106-116. DOI : 10.18653/v1/2021.wat-1.10
H. Moon, C. Park, S. Eo, J. Park & H. Lim. (2021). Filter-mBART Based Neural Machine Translation Using Parallel Corpus Filtering. Journal of the Korea Convergence Society, 12(5), 1-7. DOI : 10.15207/JKCS.2021.12.5.001

원문보기 상세보기
C. Park, Y. Lee, C. Lee & H. Lim. (2020). Quality, not quantity?: Effect of parallel corpus quantity and quality on neural machine translation. In The 32st Annual Conference on Human Cognitive Language Technology, 363-368.

저자의 다른 논문 :

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증