초록 ▼

본 연구과제는 클라우드 환경에서 빅데이터를 기반으로 수집, 정제, 통계 분석, 시각화 작업을 처리할 수 있는 통합 솔루션 개발을 목표로 함.
ㅇ 연구개발 성과
- 기존 통계 분석 방식의 고비용과 복잡성을 해소함
- 빅데이터 분석 시스템의 개발 기간을 단축함
- 빅데이터 분석 소프트웨어의 국산화를 달성함
- 대외적 인증 성과를 통한 경쟁력 확보
• 상표 출원, 저작권 등록, 특허 출원 등
• 정보통신산업진흥원(NIPA) 지원 및 컨설팅, 한국정보통신진흥협회(TTA) 인증을 통한 품질 향상
<

본 연구과제는 클라우드 환경에서 빅데이터를 기반으로 수집, 정제, 통계 분석, 시각화 작업을 처리할 수 있는 통합 솔루션 개발을 목표로 함.
ㅇ 연구개발 성과
- 기존 통계 분석 방식의 고비용과 복잡성을 해소함
- 빅데이터 분석 시스템의 개발 기간을 단축함
- 빅데이터 분석 소프트웨어의 국산화를 달성함
- 대외적 인증 성과를 통한 경쟁력 확보
• 상표 출원, 저작권 등록, 특허 출원 등
• 정보통신산업진흥원(NIPA) 지원 및 컨설팅, 한국정보통신진흥협회(TTA) 인증을 통한 품질 향상

ㅇ 연구개발 내용
- 데이터 수집, 적재 기술 개발
- 데이터 저장소, 전송 제어 모듈 개발
- 정제 프로세스 디자이너 개발
- 통계 분석 도구 개발
- 시각화 도구 개발
- ETL 에이전트 개발
- 서버 분석 엔진 개발
- 통계 분석 알고리즘, 시각화 그래프 추천 시스템 개발
- 통합 메타데이터 관리 도구 개발
- 포털 구축

(출처 : 요약서 3p)

Abstract ▼

□ Purpose & Contents
1. Purpose
ㅇ Unified Data Analytics Suite(UDAS) provides the visualized results of data analysis and various analytic skills and techniques suited for big data environments by eliminating the high cost and complexity of existing methods and helps reduce development time in

□ Purpose & Contents
1. Purpose
ㅇ Unified Data Analytics Suite(UDAS) provides the visualized results of data analysis and various analytic skills and techniques suited for big data environments by eliminating the high cost and complexity of existing methods and helps reduce development time in big data analysis system. The purpose is to adapt the software to meet the requirements of the rapidly growing local bid data software industry market.
ㅇ Development of Unified Data Analytics Suites for analyzing and visualizing big data
- Developed the unified collection and load of data
- Developed the big data statistical analysis tool by language R
- Developed the visualization tools by D3.js, the open source visualization library
- Researched and analyzed the open source R and D3.js to collect metadata and exploit solutions to catch up the latest technology and increase the competitiveness for open source market.

2. Contents
ㅇ Development of data unified collection technology
- Multi data collection technology
• Collects various forms of structured / unstructured / semi-structured data in real time and batch forms. Collects receive / send mail data, RDB data, system data, and log data of users.
- Unified data repository
• Collects data backup and various environment setting and save log data for data stability. It saves and manages data of service management and process management and data related to monitoring.
- Unified data collection process management technology
• Manages the configuration, control, and monitors all processes from data collection to loading.
- Collection policy and rule repository
• Defines policies related to data collection exclusions, policy definitions related to personal information, and data conversion, compression files, and attachment data extraction.
- Data transfer control technology
• Stable and various types of data transmission technology.
ㅇ Development of data refining techniques
- Data refining process technology
• Visualize the entire process of statistical analysis and analyze model to each step of refining process.
- Refinement process design technology
• Process all the processes of data refinement from big data to ODS, DW, and DM to process step by step.
- Query design technology
• Provides schema information for the data source with reverse engineering, This information enables not only the ERD but also the entity, relationship information, and index to design the query.
- ERD design technology
• Provides tools to create accurate data models to reduce data duplication, improve productivity, and meet standards.
- Mapping design technology
• Big data refinement is difficult to manage and reuse. Mapping designers simplify the creation of data source queries and generate load information by mapping the queryresults to the target database structure in various ways.
- ETL Agent technology
• ETL is based on mapping metadata information, and query results are loaded into the target database based on metadata.
- Data consistency verification technology
• Identification / non-identification relation, distinguish and extract abnormal values according to numerical values, and abnormal values according to attributes
ㅇ Common modules
- Unified data analytics suite IDE
• Unified development environment that enables all processes of data refinement, analysis model design, statistical analysis execution and visualization.
- Unified metadata management technology
• IIndividual metadata generated in the analysis process is managed in the integrated metadata repository, and each metadata is reconstructed and integrated to create andmanage analysis templates of a structured structure.
ㅇ Development of statistical analysis technology
- Analysis model design technology
• Manages the analysis process by mapping the data to be analyzed, statistical analysis algorithms, and visualization graphs based on metadata.
- Statistical analysis algorithm recommendation and linkage technology
• Provides recommendation function using data schema of analysis model and statistical analysis algorithm metadata.
- Analysis engine serverization technology
• When execute analyzing, it can be processed through the analysis engine server service when the PC can not process it according to the configuration data size and analysis typeof analysis model.
- Statistical analysis execute and view distribution technology
• Analysts can run a specific analysis model or run an analysis of the entire model, where the data from the refiningprocess can be re-imported or the imported data can beused as is.
- Repetitive analysis technology using analysis template
• It provides the templates that the statistical analysis expert creates in the future and can be used by using or modifying the statistical analysis process. It is a technology for reusing and repeating analysis by changing only the data source.
ㅇ Development of data visualization technology
- Visualization graph recommendation and linkage technology
• Metadata providing recommendations use the data schema of analysis model and visualization graphs.
- Graph image reprocessing technology
• This technology is analyze and generate comparison data to result graph overlap processing.
- Visualization execute and result view distribution technology
• Analysis The visualization graph that is executed can be confirmed through the dedicated result view and can be configured together with the result data of R and shared tothe analysis portal.
ㅇ Substantiation case and evaluating
- Build cloud-based big data analytics environment (Apply virtual data of Lotte Data Communication Company)
- Build big data analysis substantiation case
- Solution validation and evaluation

□ Results
1. Technical results
A. Research and development results
ㅇ Real-time service and resource monitoring technology for data collect
- Monitoring system service status and system resource usage for data collect.
ㅇ Secured attachment and compression file disassembly techniques
- Extract attachments and extract compressed file data from the mail.
ㅇ Applied public cloud systems
- Development of cloud-based system.
ㅇ Developed of analysis process automation wizard
- Simplify the entire process of the analysis process through the step-by-step setup function, It is wizards for non-experts to do statistical analysis.
ㅇ Developed of statistical analysis template
- By providing pre-modeled templates, users can create statistical analysis models in a short time by changing only thedata source and modifying the statistical analysis or modeling.
ㅇ Development of server analysis system
- Process statistical analysis using high performance and highly efficient server resources without using user's local resources. The server manages the distributed analysis agents and distributes the work appropriately according to the priority queue.
B. External certification results
ㅇ Applied 5 trademark of UDAS (1 trademark + 4 services)
ㅇ Registered 6 copyright of UDAS
- UDASTransfer(Mail/Log/Database transfer)
- UDASCollector(Mail/Log/Database collector)
- UDASCollector(Mail collector)
- UDASProcessor(Mail parser/transfer)
- Development of big data-base unified data analytics suite in cloud environment
- Unified Data Analytics Suite (UDAS)
ㅇ Registered software escrow
- New: Development of big data-base unified data analytics suite in cloud environment (2016. 07. 08 ~ 2017. 07. 07)
- Update: Development of big data-base unified data analytics suite in cloud environment (2017. 07. 08 ~ 2018. 07. 07)
ㅇ Registered 2 entries in software asset bank
ㅇ Applied 5 patents
- System for retrieving, processing, converting, and saving data for use as big data
- Method for retrieving, processing, converting, and saving data for use as big data
- ANSI query statement generation system using graphical user interface and method thereof
- Statistical analysis function recommendation system based on table structure and data characteristics
- Expert knowledge learning and recommendation methods of component-based software analysis tools

2. Technical results – Securing source technology
ㅇ System and method for retrieving, processing, converting, and saving data for use as big data
- Implemented through one unified technology in various data type collection and load from various types of data collection
- Email attachment files decomposition and text data extraction technology
ㅇ ANSI query statement generation system using graphical user interface and method thereof
- An ANSI query statement generation system using a graphical user interface that can automatically generate corresponding SQL query statements by analyzing the join relationship betweentables represented by symbols
ㅇ Statistical analysis function recommendation system based on table structure and data characteristics
- Statistical analysis function recommendation system based on data structure and table structure which can automatically recommend statistical analysis function according to table structure and characteristics by extracting characteristic information for each column of table and learning characteristic information and statistical analysis function
ㅇ Expert knowledge learning and recommendation methods of component-based software analysis tools
- Expert knowledge and recommendation method of component-based software analysis tool that enables non-experts,such as experts, to easily use modularized (component) -based ITrelated tools by learning mechanics of expert's experience and knowledge

3. Economic results
ㅇ Achieved domestic market share of 0.05%.
ㅇ Increase sales due to UDAS (KRW 11.9 billion)
- UDAS use contract for statistical analysis and visualization necessary for“GS 칼텍스 경영계획 및 손익추정 업무시스템 구축”(KRW 1.5 billion)
- UDAS, and contracted to improve “삼성전기 TSVC MES 3.0” with refining, statistical analysis and visualization modules (KRW1.04 billion)
ㅇ Creation of jobs ( Supervise agency : 6, Part agency : 2 )
ㅇ Economic results of substantiation base
- Reduce data refining operations by 50 % or more
- Reduce data refining costs by 50 % or more

□ Expected Contribution
ㅇ Contributing to fostering data scientists through university free distribution
ㅇ UDAS statistical analysis module and analysis template expansion – VPL application
ㅇ Build a market for big data analytics systems that are low-cost and affordable
ㅇ Acquire additional cloud technology and join Advanced Analytics with technology partnership with Microsoft
ㅇ Artificial intelligence (machine learning) added to existing statistical analysis library recommendation algorithm.
ㅇ Contributing to statistical analysis popularization

(출처 : SUMMARY 13p)

목차 Contents

• 표지 ... 1
• 제 출 문 ... 2
• 보고서 요약서 ... 3
• 연구개발사업 주요 연구성과 ... 5
• 국문 요약문 ... 6
• Summary ... 13
• Contents ... 20
• 목차 ... 22
• 표목차 ... 24
• 그림목차 ... 25
• 제1장. 연구개발과제의 개요 ... 28
• 1. 연구개발 목적 ... 28
• 2. 연구개발의 필요성 ... 28
• 3. 연구개발 범위 ... 29
• 제2장. 국내외 기술 개발 현황 ... 32
• 1. 국내 기술 개발 현황 ... 32
• 2. 국외 기술 개발 현황 ... 33
• 3. 기술 개발 현황에서의 위치 ... 37
• 제3장. 연구 수행 내용 및 성과 ... 39
• 1. 연구 수행 내용 ... 39
• 2. 연구 수행 내용 - 실증 ... 70
• 3. 연구 수행 성과 ... 83
• 제4장. 목표 달성도 및 관련 분야 기여도 ... 99
• 1. 목표 달성도 ... 99
• 2. 관련 분야 기여도 ... 105
• 제5장. 연구개발성과의 활용계획 ... 108
• 제6장. 연구 과정에서 수집한 해외 과학기술 정보 ... 109
• 1. VPL (Visual Programming Language) ... 109
• 2. 통계 추천 알고리즘 – LDA(Latent Dirichlet Allocation) ... 112
• 3. 순환신경망 - RNN(Recurrent Neural Network) ... 114
• 4. 하둡 에코 시스템(Hadoop Echo System) ... 119
• 5. Azure … Microsoft Cloud ... 121
• 제7장. 연구개발성과의 보안등급 ... 123
• 제8장. 국가과학기술종합정보시스템에 등록한 연구시설·장비 현황 ... 124
• 제9장. 연구개발과제 수행에 따른 연구실 등의 안전 조치 이행 실적 ... 125
• 1. 연구실 안전 점검 ... 125
• 2. 교육 훈련 ... 125
• 제10장. 연구개발과제의 대표적 연구 실적 ... 126
• 제11장. 기타 사항 ... 127
• 제12장. 참고 문헌 ... 128
• 첨부 1. 자체 보안관리 진단표 ... 129
• 첨부 2. 연구실 안전조치 이행표 ... 130
• 첨부 3. 연구과제 성과(기술성) 증빙자료 ... 131
• 끝페이지 ... 133