[논문]사용자 특성과 ChatGPT 신뢰의 관계 : 인구통계학적 변수와 AI 경험의 영향

박예은; 장정훈

doi:10.17662/ksdim.2023.19.4.053

사용자 특성과 ChatGPT 신뢰의 관계 : 인구통계학적 변수와 AI 경험의 영향
User Factors and Trust in ChatGPT: Investigating the Relationship between Demographic Variables, Experience with AI Systems, and Trust in ChatGPT 원문보기

디지털산업정보학회논문지 = Journal of the Korea Society of Digital Industry and Information Management, v.19 no.4, 2023년, pp.53 - 71

박예은 (연세대학교 혁신학과) , 장정훈 (연세대학교 혁신학과)

Abstract ▼ AI-Helper

This study explores the relationship between various user factors and the level of trust in ChatGPT, a sophisticated language model exhibiting human-like capabilities. Specifically, we considered demographic characteristics such as age, education, gender, and major, along with factors related to previous AI experience, including duration, frequency, proficiency, perception, and familiarity. Through a survey of 140 participants, comprising 71 females and 69 males, we collected and analyzed the data to see how these user factors have a relationship with trust in ChatGPT. Both descriptive and inferential statistical methods, encompassing multiple linear regression models, were employed in our analysis. Our findings reveal significant relationships between user factors such as gender, the perception of prior AI interactions, self-evaluated proficiency, and Trust in ChatGPT. This research not only enhances our understanding of trust in artificial intelligence but also offers valuable insights for AI developers and practitioners in the field.

주제어

표/그림 (6)

표 Variables
표 Descriptive statistics of sample
표 Multiple Linear Regression on Trust and Demographic Variables
그림 Boxplot of Gender vs. Trust in ChatGPT
그림 Boxplot of AI experience vs. Trust in ChatGPT
표 Multiple Linear Regression on Trust and AI Experience Variables

AI 본문요약
AI-Helper

제안 방법

Multiple linear regression analyses were performed to simultaneously examine the impact of demographic factors (age, gender, education level) on trust in ChatGPT. For each of the three trust outcomes (Trust in System, Trust in Information, Relative Trust), we regressed them on the demographic factors, while considering ChatGPT user experience variables as potential additional independent variables to control for their effects. For each model, a stepwise regression approach was employed to select the ChatGPT user experience variables that were highly associated with the outcome and achieve parsimony in modeling.
Multiple linear regression analyses were performed to simultaneously examine the impact of demographic factors (age, gender, education level) on trust in ChatGPT. For each of the three trust outcomes (Trust in System, Trust in Information, Relative Trust), we regressed them on the demographic factors, while considering ChatGPT user experience variables as potential additional independent variables to control for their effects.
The aims of this research is to examine the relationship between user factors including demographic information and past AI experiences and how it can affect users' trust in ChatGPT
To achieve this, quantitative research design was employed. The methodology involves descriptive analysis using box plots to visualize the distributions of variables of interest and Spearman correlation test to quantify relationships of Predictor and primary outcome. To further investigate the research questions, inferential statistics such as t-test and ANOVA were employed to compare means across different groups.
The purpose of this study was to investigate the relationship between user variables, including demographic attributes and AI experience, and their corresponding levels of trust. We employed descriptive statistics and inferential statistics to compare whether the differences in the groups were significant.

이론/모형

For each of the three trust outcomes (Trust in System, Trust in Information, Relative Trust), we regressed them on the demographic factors, while considering ChatGPT user experience variables as potential additional independent variables to control for their effects. For each model, a stepwise regression approach was employed to select the ChatGPT user experience variables that were highly associated with the outcome and achieve parsimony in modeling. Note that all of the demographic factors were forced in each model.

성능/효과

After a detailed examination of the demographic variables, it was found that the level of trust in ChatGPT's system is influenced by gender and the frequency of ChatGPT usage, while other demographic variables such as age, level of education, and major did not show substantial correlations with trust factors in this specific study.
In conclusion, the findings indicate that factors related to AI experience, such as higher proficiency and better experience with prior ai services has positive association with both system and information trust. Furthermore, higher proficiency with previous AI systems was also discovered to have a significant effect on relative trust.
In conclusion, this study underscores the limitations of relying solely on demographic factors for predicting trust in AI language models like ChatGPT. As AI services have become more widespread and familiar, the significance of demographic differences in trust seems to have diminished.
Moreover, the version of ChatGPT used was recognized as a confounding factor in the relationship between AI experience and trust in ChatGPT. Specifically, ChatGPT3.
The final aspect of the primary outcome involved participants rating their trust in ChatGPT compared to human experts, online forums, and traditional search engines ('Relative Trust').
The findings revealed that gender and the frequency of ChatGPT usage significantly influenced trust levels in the system, whereas other demographic variables like age, education level, and major showed no substantial correlations with trust. Specifically, men who used ChatGPT more frequently exhibited higher levels of trust, highlighting potential gender-specific disparities in trust in AI systems.

후속연구

This study's findings may not generalize to diverse cultural and linguistic contexts, necessitating caution in interpretation. Further investigation is required to assess the impact of these variables on user trust in AI systems.

참고문헌 (34)

Altman, S, Twitter, https://twitter.com/sama/status/1599668808285028353, 2022.12.15.
Cramer, H., Evers, V., Ramlal, S., Van Someren,？M., Rutledge, L., Stash, N., & Wielinga, B., "The？effects of transparency on trust in and？acceptance of a content-based art？recommender," User Modeling and？User-Adapted Interaction, Vol.18, No.5, 2008,？pp.455-496.
Yin, M., Wortman Vaughan, J., & Wallach, H.,？"Understanding the effect of accuracy on trust？in machine learning models," In Proceedings of？the 2019 CHI Conference on Human Factors in？Computing Systems, 2019, pp.1-12.
Yuan, S., & Lou, N. M. "Trust in artificial？intelligence in customer service: Factors？influencing its formation and development,"？International Journal of Information？Management, Vol.49, 2019, pp.233-242.
Rousseau, D. M., Sitkin, S. B., Burt, R. S., &？Camerer, C., "Not so different after all: A？cross-discipline view of trust," Academy of？Management Review, Vol.29, No.3, 2004,？pp.393-404.
Lee, J. D., & See, K. A., "Trust in automation:？Designing for appropriate reliance," Human？Factors, Vol.46, No.1, 2004, pp.50-80.

상세보기
Hancock, P. A., Billings, D. R., Schaefer, K. E.,？Chen, J. Y., de Visser, E. J., & Parasuraman, R.,？"A meta-analysis of factors affecting trust in？human-robot interaction," Human Factors,？Vol.53, No.5, 2011, pp.517-527.

상세보기
Hoff, K. A., & Bashir, M., "Trust in automation:？Integrating empirical evidence on factors that？influence trust," Human Factors, Vol.57, No.3,？2015, pp.407-434.

상세보기
Dzindolet, M. T., Peterson, S. A., Pomranky, R.？A., Pierce, L. G., & Beck, H. P., "The role of？trust in automation reliance," International？Journal of Human-Computer Studies, Vol.58,？No.6, 2003, pp.697-718.
Brown, T. B., Mann, B., Ryder, N., Subbiah, M.,？Kaplan, J., Dhariwal, P., & Agarwal, S.,？"Language models are few-shot learners," In？Advances in Neural Information Processing？Systems, Vol.33, 2020, pp.1877-1901.
Radford, A., Wu, J., Child, R., Luan, D.,？Amodei, D., & Sutskever, I., "Language models？are unsupervised multitask learners," OpenAI？Blog, Vol.1, No.8, 2019, p.9
Luger, E., & Sellen, A., "Like having a really？bad PA: The Gulf between user expectation and？experience of conversational agents," In？Proceedings of the 2016 CHI Conference on？Human Factors in Computing Systems, 2016,？pp. 5286-5297.
Siau, K., & Wang, W., "Building trust in？artificial intelligence, machine learning, and？robotics," Cutter Business Technology Journal,？Vol.31, No.2, 2018, pp.47-53.

상세보기
Folstad, A., & Brandtzaeg, P. B., "Chatbots and？the new world of HCI Interactions," Vol.24,？No.4, 2017, pp.38-42.
Cave, S., Dignum, V., & Meyer, J., "The ethics of？AI: Issues, developments, and？recommendations," In Proceedings of the 2018？AAAI/ACM Conference on AI, Ethics, and？Society, 2018, pp. 1-7.
김효정, "ChatGPT의 특성이 사용의도에 미치는？영향에 관한 연구: 교사의 디지털 기술 조절효과를 중심으로," (사)디지털산업정보학회 논문지, 제19권, 제2호, 2023, pp.135-145.
Devadas Menon, K Shilpa, "Chatting with？ChatGPT: Analyzing the factors influencing？users' intention to Use the Open AI's ChatGPT？using the UTAUT model," Heliyon, Vol.9,？No.11, 2023, pp.127-143.
Kreps S, George J, Lushenko P, Rao A.,？"Exploring the artificial intelligence Trust？paradox: Evidence from a survey experiment in？the United States," PLOS ONE, Vol.18, No.7,？2023

상세보기
Gillath, O., Ai, T., Branicky, M. S., Keshmiri, S.,？Davison, R. B., & Spaulding, R., "Attachment？and trust in artificial intelligence," Computers in？Human Behavior, Vol.115, 2021, p.106607.

상세보기
Czaja, S. J., Charness, N., Fisk, A. D., Hertzog,？C., Nair, S. N., Rogers, W. A., & Sharit, J.,？"Factors predicting the use of technology:？Findings from the Center for Research and？Education on Aging and Technology？Enhancement (CREATE)," Psychology and？Aging, Vol.21, No.2, 2006, pp.333-352.

상세보기
Shandilya, E., & Fan, M., "Understanding Older？Adults' Perceptions and Challenges in Using？AI-enabled Everyday Technologies," arXiv, 2022.
Gillespie, N., Lockey, S., Curtis, C., Pool, J., &？Akbari, A. "Trust in Artificial Intelligence: A？Global Study," The University of Queensland？and KPMG Australia, 2023
Hillesheim, A. J., Rusnock, C. F., Bindewald, J.？M., & Miller, M. E., "Relationships between？User Demographics and User Trust in an？Autonomous Agent. Proceedings of the Human？Factors and Ergonomics Society Annual？Meeting," Vol.61, No.1, 2017, pp.314-318.
Kizilcec, R. F. "How much information?: Effects？of transparency on trust in an algorithmic？interface," In Proceedings of the 2016 CHI？Conference on Human Factors in Computing？Systems, 2016, pp. 2390-2395.
Jackson, K. M., Sharp, W. H., & Shaw, T. H.,？"Modeling the Impacts of Positive Interaction？Frequency on Subjective Trust in an？Autonomous Agent: A Linear Mixed Model？Approach," Proceedings of the Human Factors？and Ergonomics Society Annual Meeting,？Vol.66, No.1, 2022, pp.798-801.
Bach, T. A., Khan, A., Hallock, H., Beltrao, G., &？Sousa, S. "A Systematic Literature Review of？User Trust in AI-Enabled Systems: An HCI？Perspective," International Journal of Human-Computer Interaction, 2022, pp.1-16.
Venkatesh, V., Morris, M. G., Davis, G. B., &？Davis, F. D., "User acceptance of information？technology: Toward a unified view," MIS？Quarterly, Vol.27, No.3, 2003, pp.425-478.
Sheridan, T. B., "Individual differences in？attributes of trust in automation: Measurement？and application to system design," Frontiers in？Psychology, Vol.10, 2019, p.1117.

상세보기
Muir, B. M., "Trust in automation: Part I.？Theoretical issues in the study of trust and？human intervention in automated systems,"？Ergonomics, Vol.37, No.11, 1994, pp.1905-1922.

상세보기
Parasuraman, R., Sheridan, T. B., & Wickens, C.？D., "Situation awareness, mental workload, and？trust in automation: Viable, empirically？supported cognitive engineering constructs,"？Journal of Cognitive Engineering and Decision？Making, Vol.2, No.2, 2008, pp.140-160.

상세보기
Castelfranchi, C., & Falcone, R., "Trust theory: A？socio-cognitive and computational model," John？Wiley & Sons, 2010, p.147-190.
Jian, J. Y., Bisantz, A. M., and Drury, C. G.,？"Foundations for an empirically determined？scale of trust in automated systems," Int. J.？Cogn. Ergon, Vol.4, 2000, pp.53-71.

상세보기
Meinald T. Thielsch & Gerrit Hirschfeld, "Facets？of Website Content, Human-Computer？Interaction," Vol.34, No.4, 2019, pp.279-327.
박상우, "인공지능 역량 함양을 위한 경험학습 기반 교육에 관한 고찰," (사)디지털산업정보학회논문지, 제19권, 제1호, 2023, pp.153-172.

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증