[논문]혼동행렬의 상관계수를 이용한 최적분류점

홍종선; 오세현; 최예원

doi:10.5351/kjas.2022.35.1.077

혼동행렬의 상관계수를 이용한 최적분류점
Optimal threshold using the correlation coefficient for the confusion matrix 원문보기

응용통계연구 = The Korean journal of applied statistics, v.35 no.1, 2022년, pp.77 - 91

홍종선 (성균관대학교 통계학과) , 오세현 (성균관대학교 통계학과) , 최예원 (성균관대학교 통계학과)

초록
AI-Helper

의학통계와 신용평가 분야에서 혼합분포함수를 판별하는 최적분류점 추정하기 위하여 판별력을 측정하는 다양한 정확도 측도들이 존재한다. 최근에 혼동행렬 빈도수로 표현되는 Matthews의 상관계수와 정밀도와 재현율의 조화평균인 F1 통계량의 정확도 측도들이 최적분류점을 추정하는데 연구되었다. 본 연구에서는 이런 정확도 측도들 중에서 표본크기에 의존하는 정확도 측도들은 두 표본크기 차이가 많은 경우에 최적분류점을 설정하는데 적절하지 않음을 발견한다. 그리고 대안적인 정확도 측도로 혼동행렬의 비율들의 함수인 상관계수를 정의하고, 이를 최대화하는 분류점을 최적분류점으로 추정하는 방법을 제안하고 이 방법의 유용성과 활용성에 대하여 토론한다.

Abstract ▼ AI-Helper

The optimal threshold estimation is considered in order to discriminate the mixture distribution in the fields of Biostatistics and credit evaluation. There exists well-known various accuracy measures that examine the discriminant power. Recently, Matthews correlation coefficient and the F1 statistic were studied to estimate optimal thresholds. In this study, we explore whether these accuracy measures are appropriate for the optimal threshold to discriminate the mixture distribution. It is found that some accuracy measures that depend on the sample size are not appropriate when two sample sizes are much different. Moreover, an alternative method for finding the optimal threshold is proposed using the correlation coefficient that defines the ratio of the confusion matrix, and the usefulness and utility of this method are also discusses.

주제어

표/그림 (11)

표 Table 1: Five categories for accuracy measures
표 Table 2: Thresholds for various accuracy measures
그림 Figure 1: nMCC-F1 plot.
그림 Figure 2: nMCC and F₁ curves.
그림 Figure 3: Variations of FPR and FNR.
$Table 3: Thresholds when <TEX>$\sigma^2_d\geq\sigma^2_n$</TEX>$ 표 Table 3: Thresholds when $\sigma^2_d\geq\sigma^2_n$
$Figure 4: Thresholds on the CC curve when <TEX>$\sigma^2_d>\sigma^2_n$</TEX>.$ 그림 Figure 4: Thresholds on the CC curve when $\sigma^2_d>\sigma^2_n$.
$Figure 5: Thresholds on the CC curve when <TEX>$\sigma^2_d<\sigma^2_n$</TEX>.$ 그림 Figure 5: Thresholds on the CC curve when $\sigma^2_d<\sigma^2_n$.
$Table 4: Thresholds when <TEX>$\sigma^2_d\leq\sigma^2_n$</TEX>$ 표 Table 4: Thresholds when $\sigma^2_d\leq\sigma^2_n$
그림 Figure 6: The CC curves.
표 Table 5: Thresholds and errors

참고문헌 (39)

Altman DG and Bland JM (1994). Diagnostic tests. 1: sensitivity and specificity, British Medical Journal, 308, 1552.

상세보기
Bamber D (1975). The area above the ordinal dominance graph and the area below the receiver operating characteristic graph, Journal of mathematical psychology, 12, 387-415.

상세보기
Bradley AP (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognitions, 30, 1145-1159.

상세보기
Brasil P (2010). Diagnostic Test Accuracy Evaluation for Medical Professionals, Package DiagnosisMed in R.
Cao C, Chicco D, and Holfman MM (2020). The MCC-F1 Curve: A Performance Evaluation Technique for Binary Classification, arXiv Preprint arXiv:2006,11278.
Chicco D and Jurman G (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation, BMC Genomics, 21, 1-13.

상세보기
Centor RN (1991). Signal detectability: The use of ROC curves and their analyses, Medical Decision Making, 11, 102-106.

상세보기
Connell FA and Koepsell TD (1985). Measures of gain in certainty from a diagnostic test, American Journal of Epidemiology, 121, 744-753.

상세보기
Egan JP (1975). Signal detection theory and ROC-analysis, New York, Academic press.
Engelmann B, Hayden E, and Tasche D (2003). Testing rating accuracy, Risk, 16, 82-86.
Fawcett T (2006). An introduction to ROC analysis, Pattern Recognition Letters, 27, 861-874.

상세보기
Fawcett T and Provost F (1997). Adaptive fraud detection, Data Mining and Knowledge Discovery, 1, 291-316.

상세보기
Green DM and Swets JA (1966). Signal detection theory and psychophysics, 1, New York: Wiley.
Hanley JA and McNeil BJ (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve, Radiology, 143, 29-36.

상세보기
Hong CS (2000). Estimation and Hypothesis Testing, Freedom academy, Seoul.
Hong CS and Jang DH (2020). Validation ratings for the length of the ROC curve, Journal of the Korean Data & Information Science Society, 31, 851-863.

상세보기
Hong CS, Joo JS, and Choi JS (2010). Optimal thresholds from mixture distributions, The Korean Journal of Applied Statistics, 23, 13-28.

원문보기 상세보기
Hong CS and Lee SJ (2018). TROC curve and accuracy measures, Journal of the Korean Data & Information Science Society, 29, 861-872.

상세보기
Hong CS and Lim HS (1997). Comparison analysis of association measures for categorical data, Communications for Statistical Applications and Methods, 4, 645-661.
Hong CS, Lin MH, Hong SW, and Kim GC (2011). Classification accuracy measures with minimum error rate for normal mixture, Journal of the Korean Data & Information Science Society, 22, 619-630.
Hsieh F and Turnbull BW (1996). Nonparametric and semiparametric estimation of the receiver operating characteristic curve, The Annals of Statistics, 24, 25-40.

상세보기
Krzanowski WJ and Hand DJ (2009). ROC Curves for Continuous Data, CRC Press, New York.
Lambert J and Lipkovich I (2008). A macro for getting more out of your ROC curve, SAS Global Forum, 231.
Matthews BW (1975). Comparison of the predicted and observed secondary structure of T4 phage lysozyme, Biochimica et Biophysica Acta (BBA)-Protein Structure, 405, 442-451.

상세보기
McDermott J and Forsyth RS (2016). Diagnosing a disorder in a classification benchmark, Pattern Recognition Letters, 73, 41-43.

상세보기
Metz CE (1978). Basic principles of ROC analysis, In Seminars in nuclear medicine, 8, 283-298.

상세보기
Metz CE and Kronman HB (1980). Statistical significance tests for binormal ROC curves, Journal of Mathematical Psychology, 22, 218-243.

상세보기
Moses LE, Shapiro D, and Littenberg B (1993). Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations, Statistics in Medicine, 12, 1293-1316.

상세보기
Pepe MS (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction, Oxford university press, Oxford.
Perkins NJ and Schisterman EF (2006). The inconsistency of "optimal" cutpoints obtained using two criteria based on the receiver operating characteristic curve, American Journal of Epidemiology, 163, 670-675.

상세보기
Provost F and Fawcett T (2001). Robust classification for imprecise environments, Machine Learning, 42, 203-231.
Spackman KA (1989). Signal detection theory: valuable tools for evaluating inductive learning, The Analytics of Risk Model Validation, San Mateo, 160-163.
Sokolova M, Japkowicz N, and Szpakowicz S (2006). Beyond accuracy, F-score and ROC: a family of discriminant measures for performance evaluation, In Australian Joint Conference on Artificial Intelligence, Springer, Berlin.
Swets JA (1988). Measuring the accuracy of diagnostic systems, Science, 240, 1285-1293.

상세보기
Tasche D (2008). Validation of internal rating systems and PD estimates, The Analytics of Risk Model Validation, 169-196.
Vuk M and Curk T (2006). ROC curve, lift chart and calibration plot, MetodoloskiZvezki, 3, 89-108.
Yoo HS and Hong CS (2011). Optimal criterion of classification accuracy measures for normal mixture, Communications for Statistical Applications and Methods, 18, 343-355.

원문보기 상세보기
Youden WJ (1950). Index for rating diagnostic test, Cancer, 3, 32-35.

상세보기
Zweig MH and Campbell G (1993). Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine, Clinical chemistry, 39, 561-577.

상세보기

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증