[논문]Neurocomputational mechanism of controllability inference under a multi-agent setting

Kim, Jaejoong; Lee, Sang Wan; Yoon, Seokho; Park, Haeorm; Jeong, Bumseok

doi:10.1371/journal.pcbi.1009549

[해외논문] Neurocomputational mechanism of controllability inference under a multi-agent setting 원문보기

PLoS computational biology, v.17 no.11, 2021년, pp.e1009549 -

Kim, Jaejoong (Graduate School of Medical Science and Engineering, Korea Advanced Institute for Science and Technology (KAIST), Daejeon, Republic of Korea) , Lee, Sang Wan (KAIST Institute for Health Science and Technology and KI for Artificial Intelligence, KAIST, Daejeon, Republic of Korea) , Yoon, Seokho (Graduate School of Medical Science and Engineering, Korea Advanced Institute for Science and Technology (KAIST), Daejeon, Republic of Korea) , Park, Haeorm (Graduate School of Medical Science and Engineering, Korea Advanced Institute for Science and Technology (KAIST), Daejeon, Republic of Korea) , Jeong, Bumseok (Graduate School of Medical Science and Engineering, Korea Advanced Institute for Science and Technology (KAIST), Daejeon, Republic of Korea)

Abstract ▼ AI-Helper

Controllability perception significantly influences motivated behavior and emotion and requires an estimation of one’s influence on an environment. Previous studies have shown that an agent can infer controllability by observing contingency between one’s own action and outcome if there are no other outcome-relevant agents in an environment. However, if there are multiple agents who can influence the outcome, estimation of one’s genuine controllability requires exclusion of other agents’ possible influence. Here, we first investigated a computational and neural mechanism of controllability inference in a multi-agent setting. Our novel multi-agent Bayesian controllability inference model showed that other people’s action-outcome contingency information is integrated with one’s own action-outcome contingency to infer controllability, which can be explained as a Bayesian inference. Model-based functional MRI analyses showed that multi-agent Bayesian controllability inference recruits the temporoparietal junction (TPJ) and striatum. Then, this inferred controllability information was leveraged to increase motivated behavior in the vmPFC. These results generalize the previously known role of the striatum and vmPFC in single-agent controllability to multi-agent controllability, and this generalized role requires the TPJ in addition to the striatum of single-agent controllability to integrate both self- and other-related information. Finally, we identified an innate positive bias toward the self during the multi-agent controllability inference, which facilitated behavioral adaptation under volatile controllability. Furthermore, low positive bias and high negative bias were associated with increased daily feelings of guilt. Our results provide a mechanism of how our sense of controllability fluctuates due to other people in our lives, which might be related to social learned helplessness and depression.Author summaryHow we perceive controllability over an outcome if there are multiple other agents who can simultaneously influence that outcome? Previous ‘single-agent’ studies showed that an agents’ inferred controllability depends on contingency between its own action and following outcome and this inference involves striatum. Here, we show that in a multi-agent setting, other people’s action-outcome contingency information is integrated with one’s own action-outcome contingency to infer controllability, which was explained as a biased Bayesian inference. Notably, bias in inference played an adaptive role under volatile controllability and was associated with a perception of guilt. Striatum and temporoparietal junction (TPJ) were involved in this multi-agent Bayesian controllability inference and this controllability information was leveraged to increase motivated behavior in the vmPFC. Our results first provide a neurocomputational mechanism of multi-agent controllability inference.

참고문헌 (41)

1 Ly V , Wang S , Bhanji JP , Delgado MR . A reward-based framework of perceived control . Frontiers in neuroscience . 2019 ; 13 : 65 . doi: 10.3389/fnins.2019.00065 30809112

상세보기
2 Stolz DS , Muller-Pinzler L , Krach S , Paulus FM . Internal control beliefs shape positive affect and associated neural dynamics during outcome valuation . Nature Communications . 2020 ; 11 ( 1 ): 1 ？ 13 . doi: 10.1038/s41467-019-13993-7 31911652

상세보기
3 Maier SF , Seligman ME . Learned helplessness at fifty: Insights from neuroscience. Psychological review . 2016 ; 123 ( 4 ): 349 . doi: 10.1037/rev0000033 27337390

상세보기
4 Moscarello JM , Hartley CA . Agency and the calibration of motivated behavior. Trends in cognitive sciences . 2017 ; 21 ( 10 ): 725 ？ 35 . doi: 10.1016/j.tics.2017.06.008 28693961

상세보기
5 Huys QJ , Dayan P . A Bayesian formulation of behavioral control. Cognition . 2009 ; 113 ( 3 ): 314 ？ 28 . doi: 10.1016/j.cognition.2009.01.008 19285311

상세보기
6 Chandrasekaran C. Computational principles and models of multisensory integration . Current opinion in neurobiology . 2017 ; 43 : 25 ？ 34 . doi: 10.1016/j.conb.2016.11.002 27918886

상세보기
7 Igelstrom KM , Graziano MS . The inferior parietal lobule and temporoparietal junction: a network perspective. Neuropsychologia . 2017 ; 105 : 70 ？ 83 . doi: 10.1016/j.neuropsychologia.2017.01.001 28057458

상세보기
8 Zhang L , Glascher J . A brain network supporting social influences in human decision-making . Science advances . 2020 ; 6 ( 34 ): eabb4159 . doi: 10.1126/sciadv.abb4159 32875112

상세보기
9 Dombrovski AY , Hallquist MN , Brown VM , Wilson J , Szanto K . Value-Based Choice, Contingency Learning, and Suicidal Behavior in Mid- and Late-Life Depression . Biol Psychiatry . 2019 ; 85 ( 6 ): 506 ？ 16 . Epub 2018/12/07. doi: 10.1016/j.biopsych.2018.10.006 ; PubMed Central PMCID: PMC6380943. 30502081

상세보기
10 Rescorla RA , Wagner AR . A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. Classical conditioning II: Current research and theory . 1972 ; 2 : 64 ？ 99 .
11 Rigoux L , Stephan KE , Friston KJ , Daunizeau J . Bayesian model selection for group studies―revisited. Neuroimage . 2014 ; 84 : 971 ？ 85 . doi: 10.1016/j.neuroimage.2013.08.065 24018303

상세보기
12 Woo C-W , Roy M , Buhle JT , Wager TD . Distinct brain systems mediate the effects of nociceptive input and self-regulation on pain . PLoS biology . 2015 ; 13 ( 1 ): e1002036 . doi: 10.1371/journal.pbio.1002036 25562688

상세보기
13 Dorfman HM , Bhui R , Hughes BL , Gershman SJ . Causal inference about good and bad outcomes . Psychological science . 2019 ; 30 ( 4 ): 516 ？ 25 . doi: 10.1177/0956797619828724 30759048

상세보기
14 Cui J , Liu Y , Nallanathan A . Multi-Agent Reinforcement Learning-Based Resource Allocation for UAV Networks . IEEE Transactions on Wireless Communications . 2019 ; 19 ( 2 ): 729 ？ 43 .
15 Beyer F , Sidarus N , Fleming S , Haggard P . Losing control in social situations: How the presence of others affects neural processes related to sense of agency. eneuro . 2018 ; 5 ( 1 ). doi: 10.1523/ENEURO.0336-17.2018 29527568

상세보기
16 Niv Y , Daniel R , Geana A , Gershman SJ , Leong YC , Radulescu A , et al . Reinforcement learning in multidimensional environments relies on attention mechanisms . Journal of Neuroscience . 2015 ; 35 ( 21 ): 8145 ？ 57 . doi: 10.1523/JNEUROSCI.2978-14.2015 26019331

상세보기
17 Amat J , Christianson JP , Aleksejev RM , Kim J , Richeson KR , Watkins LR , et al . Control over a stressor involves the posterior dorsal striatum and the act/outcome circuit . European Journal of Neuroscience . 2014 ; 40 ( 2 ): 2352 ？ 8 .

상세보기
18 Bault N , Joffily M , Rustichini A , Coricelli G . Medial prefrontal cortex and striatum mediate the influence of social comparison on the decision process . Proceedings of the national Academy of sciences . 2011 ; 108 ( 38 ): 16044 ？ 9 . doi: 10.1073/pnas.1100892108 21896760

상세보기
19 Wang KS , Delgado MR . Corticostriatal circuits encode the subjective value of perceived control . Cerebral Cortex . 2019 ; 29 ( 12 ): 5049 ？ 60 . doi: 10.1093/cercor/bhz045 30877791

상세보기
20 Samson D , Apperly IA , Chiavarino C , Humphreys GW . Left temporoparietal junction is necessary for representing someone else’s belief . Nature neuroscience . 2004 ; 7 ( 5 ): 499 ？ 500 . doi: 10.1038/nn1223 15077111

상세보기
21 Ciaramidaro A , Adenzato M , Enrici I , Erk S , Pia L , Bara BG , et al . The intentional network: how the brain reads varieties of intentions . Neuropsychologia . 2007 ; 45 ( 13 ): 3105 ？ 13 . doi: 10.1016/j.neuropsychologia.2007.05.011 17669444

상세보기
22 Seth AK , Barrett AB , Barnett L . Granger causality analysis in neuroscience and neuroimaging . Journal of Neuroscience . 2015 ; 35 ( 8 ): 3293 ？ 7 . doi: 10.1523/JNEUROSCI.4399-14.2015 25716830

상세보기
23 Friston KJ , Harrison L , Penny W . Dynamic causal modelling. Neuroimage . 2003 ; 19 ( 4 ): 1273 ？ 302 . doi: 10.1016/s1053-8119(03)00202-7 12948688

상세보기
24 Vollm BA , Taylor AN , Richardson P , Corcoran R , Stirling J , McKie S , et al . Neuronal correlates of theory of mind and empathy: a functional magnetic resonance imaging study in a nonverbal task . Neuroimage . 2006 ; 29 ( 1 ): 90 ？ 8 . doi: 10.1016/j.neuroimage.2005.07.022 16122944

상세보기
25 Shenhav A , Botvinick MM , Cohen JD . The expected value of control: an integrative theory of anterior cingulate cortex function . Neuron . 2013 ; 79 ( 2 ): 217 ？ 40 . doi: 10.1016/j.neuron.2013.07.007 23889930

상세보기
26 Li L , Liu Z , Niu H , Zheng L , Cheng X , Sun P , et al . Responsibility modulates the neural correlates of regret during the sequential risk-taking task . Experimental brain research . 2018 ; 236 ( 3 ): 679 ？ 89 . doi: 10.1007/s00221-017-5165-3 29299641

상세보기
27 Taylor SE , Brown JD . Illusion and well-being: a social psychological perspective on mental health. Psychological bulletin . 1988 ; 103 ( 2 ): 193 . 3283814

상세보기
28 Alloy LB , Clements CM . Illusion of control: Invulnerability to negative affect and depressive symptoms after laboratory and natural stressors . Journal of Abnormal Psychology . 1992 ; 101 ( 2 ): 234 . doi: 10.1037//0021-843x.101.2.234 1583214

상세보기
29 O’Connor LE , Berry JW , Weiss J , Gilbert P . Guilt, fear, submission, and empathy in depression . Journal of affective disorders . 2002 ; 71 ( 1？3 ): 19 ？ 27 . doi: 10.1016/s0165-0327(01)00408-6 12167497

상세보기
30 Branzei R , Dimitrov D , Tijs S . Models in cooperative game theory: Springer Science & Business Media ; 2008 .
31 Tangney JP . The test of self-conscious affect: George Mason Univ. ; 1989 .
32 Frassle S , Aponte EA , Bollmann S , Brodersen KH , Do CT , Harrison OK , et al . TAPAS: an open-source software package for Translational Neuromodeling and Computational Psychiatry. Frontiers in Psychiatry . 2021 ; 12 : 857 . doi: 10.3389/fpsyt.2021.680811 34149484

상세보기
33 Shanno DF . Conditioning of quasi-Newton methods for function minimization . Mathematics of computation . 1970 ; 24 ( 111 ): 647 ？ 56 .

상세보기
34 Mathys CD , Lomakina EI , Daunizeau J , Iglesias S , Brodersen KH , Friston KJ , et al . Uncertainty in perception and the Hierarchical Gaussian Filter. Frontiers in human neuroscience . 2014 ; 8 : 825 . doi: 10.3389/fnhum.2014.00825 25477800

상세보기
35 Daunizeau J , Adam V , Rigoux L . VBA: a probabilistic treatment of nonlinear models for neurobiological and behavioural data. PLoS computational biology . 2014 ; 10 ( 1 ): e1003441 . doi: 10.1371/journal.pcbi.1003441 24465198

상세보기
36 Stephan KE , Penny WD , Daunizeau J , Moran RJ , Friston KJ . Bayesian model selection for group studies. Neuroimage . 2009 ; 46 ( 4 ): 1004 ？ 17 . doi: 10.1016/j.neuroimage.2009.03.025 19306932

상세보기
37 Wilson RC , Collins AG . Ten simple rules for the computational modeling of behavioral data. eLife . 2019 ; 8 : e49547 . doi: 10.7554/eLife.49547 31769410

상세보기
38 Penny WD , Friston KJ , Ashburner JT , Kiebel SJ , Nichols TE . Statistical parametric mapping: the analysis of functional brain images : Academic press ; 2011 .
39 Kasper L , Bollmann S , Diaconescu AO , Hutton C , Heinzle J , Iglesias S , et al . The PhysIO toolbox for modeling physiological noise in fMRI data . Journal of neuroscience methods . 2017 ; 276 : 56 ？ 72 . doi: 10.1016/j.jneumeth.2016.10.019 27832957

상세보기
40 Mumford JA , Poline J-B , Poldrack RA . Orthogonalization of regressors in fMRI models. PloS one. 2015 ; 10 ( 4 ): e0126255 . doi: 10.1371/journal.pone.0126255 25919488

상세보기
41 Eklund A , Nichols TE , Knutsson H . Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates. Proceedings of the National Academy of Sciences . 2016 : 201602413 .

LOADING...

활용도 분석정보

상세보기

다운로드

내보내기

활용도 Top5 논문

해당 논문의 주제분야에서 활용도가 높은 상위 5개 콘텐츠를 보여줍니다.
더보기 버튼을 클릭하시면 더 많은 관련자료를 살펴볼 수 있습니다.

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증