초록 ▼

- 본 연구는 창조적 연구성과 창출 및 성과관리를 위한 성과평가 및 정보관리체계 구축 방안 마련을 위하여 창조적 연구 진흥을 유도할 수 있는 국가 R&D 및 기관성과평가 체계 구축과 제도 개선방안을 제안하고자 함
- 국가 연구개발사업 평가제도 개선방안(제1부)는 연구개발 성과평가 및 관리의 현황 조사를 통한 이해 증진, 연구개발 부문 질적 성과평가 해외사례를 통한 시사점 도출, QOIC 개념 세부내용 정립 및 사업유형별 질적 지표 제안, 질적 지표 설정 방안 도출 등 논의
- 출연기관 상위평가 개선방안(제2부)는 상위평가

- 본 연구는 창조적 연구성과 창출 및 성과관리를 위한 성과평가 및 정보관리체계 구축 방안 마련을 위하여 창조적 연구 진흥을 유도할 수 있는 국가 R&D 및 기관성과평가 체계 구축과 제도 개선방안을 제안하고자 함
- 국가 연구개발사업 평가제도 개선방안(제1부)는 연구개발 성과평가 및 관리의 현황 조사를 통한 이해 증진, 연구개발 부문 질적 성과평가 해외사례를 통한 시사점 도출, QOIC 개념 세부내용 정립 및 사업유형별 질적 지표 제안, 질적 지표 설정 방안 도출 등 논의
- 출연기관 상위평가 개선방안(제2부)는 상위평가 제도 변천 및 발전과정 조사를 통한 이해 증진, 상위평가 제도 현황 및 추진체계 분석을 통한 개선 이슈 도출, 기관평가 관련 사례 조사(국,내외)를 통한 시사점 도출, QOIC 체계를 적용한 상위평가 제도의 경쟁력 분석, 상위평가 개선전략 및 과제, 지표의 제안 등 제시
- 국가 연구개발사업 상위평가 관련 법제도 개선방안(제3부)는 창조경제 및 연구개발 성과평가의 방향성 조사, 국내 연구개발사업 성과평가 및 관리와 관련한 법령 구조 및 특징 및 한계 도출, 성과평가 관련 법령(정부업무평가법, 성과평가법 및 성과관리 관련 법령 등)의 개정제안, 주요국의 성과평가와 관리에 관한 법령 조사 및 사례 분석, 국내외 법령의 비교분석을 통한 개선 방향제시 및 제언 등 논의
- 국가 연구개발 성과정보 관리활용체계 개선방안(제4부)는 국가 R&D 성과정보 관리체계의 개선 필요성 및 배경 조사, 국가 R&D 성과정보 및 DB 현황 조사, 국가 R&D 성과정보 관리체계의 경쟁력 분석 및 효율화 방안 제시, NTIS 및 전문기관 보유 DB간 연계 활성화 및 실용적 활용 강화방안 제시 등 포함

Abstract ▼

Ⅳ. R&D results

Part I. Improvement of R&D project evaluation systems

- Enhancement of understanding through the survey on the current status of R&D performance evaluation and management systems
Proposal of major problems in the national R&D evaluation through the understanding of th

Ⅳ. R&D results

Part I. Improvement of R&D project evaluation systems

- Enhancement of understanding through the survey on the current status of R&D performance evaluation and management systems
Proposal of major problems in the national R&D evaluation through the understanding of the flow of national R&D project evaluation and the meta-evaluation as well as drawing of necessities to strengthen qualitative evaluation of R&D performance evaluation systems through the survey on the current status of performance index

- Drawing of suggestions through oversea' qualitative performance evaluation cases in R&D areas
Through the oversea qualitative R&D evaluation performance cases(NSF, NIH, OCED etc.),
1) utilization of various evaluation standards and index for performance evaluation of the researchers/ research projects and programs in oversea research sponsoring and management institutions, 2) classification and standardization of various qualitative performance evaluation standards to promote systematic performance evaluation for R&D activities and make efforts to improve the evaluation systems

- Establishment of QOIC concepts and the details, and proposal of qualitative index by project types

Core of 'Q(Quality)-O(Outcomes)-I(Impacts)-C(Competency) system'
1) Q evaluation is qualitative performance evaluation that created and utilized to achieve original goals of the R&D projects and institutions
2) O evaluation is the “behavioral additionality effect” that affect the behavioral changes of the doers who are directly and indirectly related to the R&D projects and institutions (e.g. governmental departments that operate R&D projects and the researchers, groups and institutions who are not belonged to the evaluated R&D projects but perform studies on the corresponding areas)
3) I evaluation is qualitative performance evaluation that directly and indirectly contributes to the operation of national policies and politics and solution of social issues
4) C evaluation is the evaluation on a series of activities and performance to enhance the research competencies of the R&D projects and institutions for themselves

- Concrete measures to realize QOIC systems
1) Evaluation of representative performance reflecting 'QOIC system'
2) Security of multivalence of the evaluation through 'QOIC system'
3) Security of core research competencies of government departments through 'QOIC system'
4) Evaluation of creative economy through 'QOIC system'

- Drawing of measures to set qualitative index
Set qualitative index utilizing logic model GI(goal-index) matric and proposed cases utilized GI index(basic research, mid- and long-term industrial technology development, public technology development, facilities and equipment)
Considered the stages of the projects and types of performances through the basic logic model and provide a diagram about the performance according to the period under it.

Part II. Improvement of meta-evaluation systems for government funded institutions

- Enhancement of understanding through the changes of mata-evaluation systems and development processes
The changes of institution evaluations are as follows

Performance-oriented evaluation system was introduced with the opening of Science and Technology Innovation headquarter(October 2004)
1) As the measures to improve national R&D project evaluation systems are decided in the NSTC, self-evaluation systems by the departments and the meta-evaluation and specialized evaluation systems by NSTC were organized and started to be operated in order to increase the autonomy of the departments and the efficiencies of NSTC.
2) According to the establishment of [Act on Performance Evaluation and Management of National Research And Development Projects etc.], performance-oriented research institution evaluation systems were reorganized and meta-evaluation systems are introduced
3) With reorganization of the governmental organizations and revision of [Act on Performance Evaluation and Management of National Research And Development Projects etc.], the meta-evaluation businesses were moved from the Ministry of Science and Technology to the Ministry of Strategy and Finance.
4) With revision of [Act on Performance Evaluation and Management of National Research And Development Projects etc.], the meta-evaluation businesses were moved from the Ministry of Strategy and Finance to the NSTC

- Drawing of suggestions through the (domestic and oversea) cases studies regarding evaluation of institutions (domestic cases)
1) Necessary to strengthen the index about plans to promote vision, strategies and major projects rather than short-term index
2) Necessary to strengthen consulting function for the institutions with inferior performance evaluation results
3) As drawing suitable evaluation index for each environment through strengthening original index for institutions, it is necessary to enhance the usefulness of the evaluation systems
4) Management evaluation and the institutional directors' evaluation are separately operated in the evaluation of public institutions. Although their evaluation purposes are clearly separated, the evaluation for the government funded institutions seem to integrate the institution directors' leadership within the index for the evaluation of the institution

- Drawing of suggestions through (domestic and oversea) case studies regarding evaluation of institutions
1) Oversea institutions are evaluated by external institutions based on concrete rules and laws regarding evaluation of the departments and governments.
2) Evaluation is promoted through the negotiation of the systems similar to performance based contracts, and the management and operations are separately evaluated based on separated index for each.
3) In case of meta-evaluation cases, the rational establishment level of performance evaluation system, policy issues to be raised and the improvement measures and adjustment measures for the institutions are provided through the evaluation process.
4) Regarding evaluation methods, diverse views are maintained by utilizing various methods including evaluation by experts, peer review, review by external institutions
5) Evaluation results are variously utilized for incentives, employment, expansion of contracts, establishment of budgets and complementation of evaluation systems etc.
However it is mainly functioned to be reflected to the budget for the following year.
6) Evaluations are promoted as effective evaluation systems to improve efficiencies of the research, management of the institutions and performance of policies rather than administrative meaning.

- Competencies analysis of the meta-evaluation system applying QOIC system
1) Q evaluates whether quality institution evaluation was held
2) O evaluates whether the institution evaluations directly and indirectly contributed to the achievement of the strategic targets for the corresponding institutions
3) I measures whether the institution evaluation led the institutions' contribution to the socio-economic impact and the achievement of the targets that could affect operation of the mid- and long-term operation of the government funded institutions
4) C evaluates whether the government funded institutions are operated to accumulate their competencies

- Strategies to improve meta-evaluation

Quality. Evaluation of the performance of quality institution evaluation
1) Adjustment of the index and weight to improve discrimination of the evaluation
2) Detailed classification of the evaluation index and concretization evaluation standards
3) Addition of index to evaluate the ability of the institutions' management to respond to the current issues
4) Strengthening of the expertise and recognition of the evaluation committee members
5) Strengthening the efficiencies of the institution evaluation performance considering the institutions to be evaluated
Outcome. Drawing of the results to lead short-term changes of the institutions
1) Evaluation of the consent between research performance contract and actual performance
2) Utilization of sufficient information to improve credibility of the evaluation and strengthening of the appeal process
3) Leading to draw substantial institution evaluation reports

Impact. Utilization of the results to lead change of the operation of the institutions to achieve their social and economic goals

1) Strengthening the index for I
2) Necessary to strengthen the qualitative and effective aspects of the meta-evaluation
3) Reflection of the indicated issues by evaluation committee members and relevant institutions, contribution to the national affairs and tasks, active reflection of the institution evaluation results to the institutions' strategies

Competency. Support to fulfill the QUIC system in the R&D area and accumulate the competencies

1) New establishment of the index for C
2) Necessary for active change of meta-evaluation functions which are only limited to the improvement of the systems and policy suggestions
3) Evaluation of the fact whether the qualitative evaluation about accumulation of competencies was reflected to the institution evaluation processes
4) Evaluation of the level of contribution to improve the competencies of the institutions through the results of the meta-evaluation

- Proposal of meta-evaluation index

Part III. Improvement of meta-evaluation related laws and systems

- Characteristics of creative economy
1) Consistent strengthening of creative and innovative competencies
2) Leading role for global open-innovation
3) Recognition of the value of creativity and its social acceptance etc.
◦ The direction of this to be applied to the science and technology area include consistent strengthening of created R&D, security of international position through expansion of global cooperative researches, and organization of creative research eco-system for creative activities and its self-development

- Necessities to survey on the laws and ordinances regarding domestic performance evaluation and review and improvement of the structures
◦ It is found that similar tasks are excessively separated and some tasks are duplicated
◦ Problems that it is difficult to respond to new issues or to set policy targets under current strict legal systems are exposed

- Implications after review laws and systems
◦ The purposes of the performance evaluation in the national research and development project oriented rules could be regarded as mainly administrative and managerial evaluations(regulative characteristics). Considerations on the achievement of policy goals through the performance evaluations are insufficiently considered. The items regarding performance evaluation under the [Act on the Government Funded Institutions] are also regulated in the respect of administrative and managerial purposes of the government funded institutions.
◦ (Response to the administrative demands) Although it is partly mentioned in the basic principles etc., the corresponding law does not purpose to evaluate the research and development projects in its qualitative aspects like activation or creativeness of the researches, it considers research and development projects in administrative aspect only.
◦ (Shortage of standard performance guide) According to the item 6, no.3, the minister of Ministry of Science, ICT and Future Planning should provide 'standard performance index'reflecting the research and development projects, and the item 8, no.3 indicates that 'standard performance evaluation guide' should be provided, there is no open and legally bind index or guide are found.

- Limitations and necessities to improve for the laws and ordinances regarding domestic performance evaluation and management
1) Structural rigidity and complexity
2) Diversification of the rules and regulations and duplication of the tasks
3) Processes and management-oriented performance evaluation rules
4) Shortage of standard evaluation guide
- Benchmarking and suggestions regarding performance evaluation and management laws and ordinances of major countries

It studied the Government Performance and Results Act(GPRA) and Program Assessment Rating Tools(PART) of USA and the outline of the R&D evaluation by Japan and induced the suggestions as follows.

1) Regarding the R&D evaluation systems
Although the legal base for the domestic R&D performance evaluation of too comprehensive, general, and diversified, those of oversea countries secure consistency in the evaluation and lessen the duplication of the evaluation even though the structures of laws and ordinances seem a little bit weak.
2) Evaluation of the contents of the R&D
Domestic R&D performance evaluation seems to be focused on its management aspect under the administrative purpose. However, oversea cases purposes qualitative improvement of R&D and put importance on the creativeness of the researchers, uniqueness and prospectus of the tasks and development of the researches, to be largely different from domestic case.

- Directions to improve the limitations in the current laws and ordinances
1) It is necessary to simplify the legal systems compared to the current status in order to avoid the structural rigidity and complexity, of regulations and duplication of the tasks.
2) To improve the problem that current systems is focused on the administrative purpose with process and management oriented performance evaluation rules, it is necessary to include qualitative improvement factors including promotion of creative research etc. under the goals and meaning of the evaluation.

- Suggestions to improve laws and ordinances
1) Our relevant laws and ordinances are highly structuralized and complex, but the standard evaluation guide to be commonly utilized, which is indicated under the laws and ordinances, are still short. This is an urgent problem to solve.
2) When improving the performance evaluation system for the future, it is necessary to avoid existing management and administrative aspects but stress on the motivation, public relations/promotion, and training and learning aspects which are weak under current systems.
3) It is necessary to actually unify the laws and ordinance systems which are currently separated as the R&D tasks( set by NTIS), R&D institutions(set by Act on the Government Funded Institutions), and R&D activities(Act on the Performance Evaluation Act by NTIS) by delegating the rest to the Performance Evaluation Act by NTIS.
4) The concrete performance evaluation measures for R&D projects should be defined at the level of enforcement regulations rather than current laws or enforcement ordinances in order to reflect new policy environmental changes.

Part IV. Improvement of national R&D performance information management systems

- Survey on the necessities and background for the improvement of national R&D performance information management systems
1) Current separated R&D targeting to secure technology and knowledge needs to be changed as eco system creative R&D that integrates a series of processes to create new industries.
2) It is necessary to reorganize the R&D planning, evaluation and management systems to develop creative performances
3) It is necessary to improve the efficiencies of national R&D projects
4) The needs to provide current general status and complex analysis data that can support strategic investment decisions and others are increased as national R&D projects became large and complex

- Problems in the performance-oriented evaluation systems
1) Since the systems were changed to the performance-oriented evaluation systems, the quantitative performances of R&D have been largely grown. However, their qualitative aspects are still weak compared to developed countries, so that it is necessary to continuously improve the systems.

- Analysis of the current status and problems in general view
1) Issues on policy making and planning stages : As the information management system for relevant data are weakly set, it is difficult to provide proper information timely for the issue but the contents and performances for the current issues should be restudied manually when current issues are incurred in the nation.
2) Issues on the performance management stage : Standard process and information connection systems are weak regarding post management area. As such, although this post management of the research management institution should be maintained for five years under the current legal system, there are problems for NTIS to utilize relevant information as the work process of the corresponding department regarding the post management of the performance and the information. Thus, NTIS has difficulties to trace the evaluation, so that their post utilization of the information and management or distribution of royalty for technology experiences problems. Also their post evaluation results are not connected to the planning of the tasks in the future.

- Proposal of major tasks to promote
1) Establishment and support for the big data analysis systems for science and technology in order to respond to current issues
2) Expand provision of original data regarding the performances and strengthen support for the post management

목차 Contents

• 표지 ... 1
• 제 출 문 ... 3
• 보고서 요약서 ... 5
• 요 약 문 ... 6
• Summary ... 13
• Contents ... 23
• 목차 ... 24
• 제1부. 국가연구개발사업 평가제도 개선방안 ... 29
• 제1장. 서론 ... 30
• 제1절. 연구의 배경 및 필요성 ... 30
• 제2절. 연구의 주요 내용 및 보고서의 구성 ... 35
• 제2장. 해외사례 분석: 연구개발의 질적 평가사례를 중심으로 ... 38
• 제1절. 연구개발의 질적 평가 수행 해외사례 ... 38
• 제2절. 소결론 (질적 성과평가 해외사례를 통한 시사점) ... 54
• 제3장. 국가연구개발사업 성과평가·성과관리체제의 현황 및 진단 ... 55
• 제1절. 국가연구개발사업 평가제도의 연혁 ... 55
• 제2절. 국가연구개발사업 자체·상위 평가의 현황 ... 63
• 제3절. 국가 R&D 평가의 현황 및 문제점 ... 72
• 제4절. 성과지표의 활용 현황 진단 ... 74
• 제4장. 창조경제 실현을 위한 국가연구개발사업 성과평가전략: QOIC ... 78
• 제1절. QOIC의 개념 ... 78
• 제2절. QOIC 평가의 주요 세부내용 ... 87
• 제5장. QOIC 기반의 유형별 지표 설정 및 활용 방안 ... 97
• 제1절. 연구유형별 QOIC 평가체제 적용 ... 97
• 제2절. 목표-지표 연계표(Gl Matrix )를 활용한 질적 지표의 설정 ... 99
• 제3절. 연구유형별 QOIC 평가지표 활용 방안 ... 109
• 제4절. 목표-지표 연계표(GI Matrix ) 활용 사례 ... 124
• 제6장. 결론 ... 129
• 제2부. 출연기관 상위평가 개선방안 ... 135
• 제1장. 서론 ... 136
• 제1절. 배경 및 필요성 ... 136
• 제2장. 상위평가 개요 및 변천과정 ... 139
• 제1절. 공공기관 기관평가 개요 및 변천과정 ... 139
• 제2절. 출연기관 상위평가 변천과정 ... 141
• 제3절.2006년 상위평가제도 사례조사 ... 142
• 제3장. 상위평가 제도의 현황 및 추진체계 ... 144
• 제1절. 상위평가 제도의 현황 ... 144
• 제2절. 상위평가 추진체계 ... 149
• 제3절. 상위평가 결과 ... 153
• 제4장. 출연기관 평가 관련 사례분석 ... 155
• 제1절. 국내사례 : 공공기관 경영평가 ... 155
• 제2절. 해외 사례 ... 161
• 제3절. 사례분석의 함의 ... 181
• 제5장. 상위평가 제도의 경쟁력 분석 ... 182
• 제1절. 이론적 고찰 ... 182
• 제2절. 경쟁력 분석 프레임워크의 활용 ... 184
• 제3절. QOIC 체계의 상위평가 적용방안 ... 187
• 제4절. 경쟁력 분석 결과 ... 188
• 제5절. 기타 상위평가 제도의 문제점 및 한계 ... 192
• 제6장. 자체평가 제도의 경쟁력 분석 및 개선방향 : 연구회 소관 연구기관을 중심으로 ... 194
• 제1절. 자체평가 개관 ... 194
• 제2절. 2014년 정부의 자체평가 실시계획 ... 200
• 제3절. 2014년 연구회의 자체평가 추진계획 ... 208
• 제4절. QOI 관점의 평가 및 개선방안 ... 216
• 제7장. 상위평가 제도의 개선 전략 및 과제 ... 224
• 제1절. 개선전략 ... 224
• 제2절. 개선과제 ... 225
• 제3부. 국가연구개발사업 상위평가관련 법제도 개선방안 ... 243
• 제1장 서론 ... 244
• 제1절. 연구의 배경 및 필요성 ... 244
• 제2절. 연구의 주요 내용 및 보고서의 구성 ... 246
• 제2장. 창조경제와 국가 R&D 성과평가 발전방향 ... 249
• 제1절. 창조경제의 개념 및 함의 : 국가 R&D 성과평가의 관점에서 ... 249
• 제2절. 창조경제 실천을 위한 국가 R&D 성과평가 발전 방향 ... 251
• 제3장. 국가 RSD 성과평가 및 관리와 관련한 법령 구조 및 특징 ... 257
• 제1절. 성과평가 및 관리 관련 법령 ... 257
• 제2절. 과학기슬관련 법령 중 성과평가 및 관리에 관한 사항 ... 260
• 제3절. 국가재정 및 공공기관 운영관련 법령 중 성과평가 및 관리에 관한 사항 ... 272
• 제4절. 국내 성과평가 및 관리 관련 법령의 특징 ... 274
• 제4장. 주요국 법·제도 사례 분석 ... 275
• 제1절. 미국의 성과평가체계 및 관련 법령 ... 275
• 제2절. 일본의 성과평가체계 및 관련 법령 ... 280
• 제5장. 국내외 법령의 비교 분석 및 평가 ... 284
• 제1절. 국내외 법제 비교분석 ... 284
• 제2절. 창조경제 관점에서 본 국내 법령의 문제점 및 한계 : QOIC 관점을 중심으로 ... 286
• 제6장. 국내 성과평가 관련 법령 개선방향 및 과제 ... 291
• 제1절. 창조경제 실천을 위한 개선 방향 ... 291
• 제2절. 향우 추진 과제 ... 292
• 제4부. 국가 R&D 성과정보 관리체계 개선방안 ... 357
• 제1장. 연구의 배경 및 필요성 ... 358
• 제1절. 국가R&D 성과정보 관리체계 개선의 필요성 및 배경 ... 358
• 제2절. 국가 R&D 성과정보 및 DB 현황 ... 359
• 제3절. 국가 R&D 성과정보 관리체계의 경쟁력 분석 ... 384
• 제4절. 국가 R&D 성과정보 관리체계의 효율화 방안 ... 387
• 부록 ... 410
• 참고문헌 ... 424
• 끝페이지 ... 426