연구과제
타임라인
▼

초록
▼

- 증강현실 합성을 위한 실시간 카메라 및 페이스 트래킹 기술 개발
- 이미지를 3차원 메쉬로 처리하는 기술
- 얼굴표정인식 기술 및 재현 기술
- 3차원 그래픽 실시간 시뮬레이션 인코딩 기술
- 모바일용 버전의...

- 증강현실 합성을 위한 실시간 카메라 및 페이스 트래킹 기술 개발
- 이미지를 3차원 메쉬로 처리하는 기술
- 얼굴표정인식 기술 및 재현 기술
- 3차원 그래픽 실시간 시뮬레이션 인코딩 기술
- 모바일용 버전의 알고리즘 기술

(출처 : 요약서 3p)

Abstract
▼

Purpose & Contents
The goal of this project is to develop a competitive digital contents platform and service (hereinafter refe...

Purpose & Contents
The goal of this project is to develop a competitive digital contents platform and service (hereinafter referred to as Honey Vue) that utilizes virtual human recognition technology and real-time enhancement synthesis technology in the 'live social media’ market which is increase gradually throughout inside/outside of the country.
In other words, the Honey Vue analyzes and registers signals (personal face of a broadcaster, gesture, etc.) inputted through a general webcam and a microphone by using various techniques described in the [content of development] section, thus, the user can freely synthesize / display the contents selected in the library in an environment in which broadcasting is performed live.
As a result, individual broadcasters can express various effects such as face change, expression of effect according to motion, expression of text, application of background effect, insertion of object, etc. at desired timing as desired in live social media environment.
In other words, it is the objective of this project is to develop a platform and service that allows individual broadcasts to self-apply various effects that have been expressed on the basis of 'edit video' can do.

Results
A. Development of foundation technology
(1) Developed high-performance face tracking technology for real-time broadcasting
(A) Optimizing calculation speed
- Face tracking calculation speed reached 16ms in FHD image
(B) Developed a robust algorithm for obstacles
① Stable tracking is possible even with long hair styles that partially covers face
② Possible to track eye related feature with wearing glasses
③ Possible to track when a hand or other obstacles cover faces temporarily
(C) Optimizing recognizable face angle
- Can recognize 121.8 degrees left and right and 82.8 degrees up and down Face angle
(2) Developed facial expression recognition and facial expression reproduction technology
(A) Generates blend shape type data automatically to reproduce user's facial expression
① Automatically generates blend shapes for arbitrary faces by retargeting the main facial shape of the predefined standard model to the 3D face
② Takes less than 1 second to generate blend shape automatically
③ Possible to reproduce natural facial expressions from 12 main expression shapes
(B) Analyzing facial expression of user's face and reproducing 3D character's expression
① Based on the analysis of facial expressions, it is possible to create a 3D face model expression corresponding to a user's facial expression by synthesizing a virtual human blend shape
② Similarity rate of user facial expression and reproduced facial expression reached 90%
③ Advanced racial expression recognition function and facial reproduction technique
- Improved facial expression extraction accuracy compared to existing method by tracking face through a local image enhancement that reacts dynamically to lighting and developing facial expression extraction module
(3) Developed matching technique for realistic image and 3D object
(A) Robust algorithm for camera motion noise
① Optimizing calculation speed
- Real time (60 times / second)
② Robust algorithm for camera motion noise
- Realized posture stabilization function for real-time camera
(4) Developed a real-time broadcasting tool for digital contents for live social media broadcasting
(A) Real-time encoding of camera images and digital content
(B) GUI/UI for enhancing the usability of digital contents
- Advanced interface and design to improve the usability of Honey Vue program
(C) Implementation of detailed function
- Preview function for pre-visualization
- Video recording / image capture function
(5) Developed digital contents element technology for live social media broadcasting
(A) Can implement prop, emoticons, subtitles, effects, background insert functions
① Possible to insert digital content elements linked with face position and angle
② Possible to synthesize image sequence and digital contents
(B) Implement the facial mask function
① Real-time 3D face motion generation and mask image mapping technology
② Synthesis technology of 3D face motion with mask mapped and real image
(C) Implement 3D character face replacement function
① Possible to create blend shape to automatically reproduce the facial expression in the broadcast image
② Realization of facial expression control technology for 3D characters
③ Facial tracking data can be linked with 3D characters
(D) 3D object insertion function
① 3D objects can be inserted from camera parameters acquired

B. Developed digital contents assets for built-in contents library
(1) Established criteria for digital content assets to be uploaded and downloaded to the online content library in connected with the development of the asset authoring tool of participating firm #2
(2) Created and designed digital contents asset which focus on one-person live social media users

C. Built a platform and service model
(1) Designed and built a complete platform and service model
- Synchronized all services available to the creator/user of Honey Vue platform services
- Established policies for service operation
- Applied/extended to mobile platform base
: Applied/extended of Honey Vue technology based on mobile platform with optimization/advancement and commercialization based on PC platform

D. Developed analytical and optimization algorithms for real-time transmission of broadcasting and linkage to on-line contents library (based on PC platform)
- Enhanced security for data communication with Honey Vue program by introducing security server
- Ensure that quality content can be served to users by reviewing the free /unassigned asset files registered by the creators
- Real-time update and marketing functions are possible for free digital content because the service provider can control the digital contents provided by Honey Vue program at all times.
- Improved user access to free item purchases

E. Developed output structure to send final content with real-time enhancement synthesis technology to streaming
(1) Connected to streaming server client
- Connected to desk top broadcasting program
- Connected to a viewer client

F. Developed contents library
(1) Built a contents library for live social media broadcasting
- Established classification system by contents type
- Established a service for Honey Vue's pilot service and commercialization
(2) Built a management system of contents library
- Managed the content registration created by the creator
- Built and managed content marts
: Completed contents library website business registration
- Established statistics system based on contents utilization
- Built a system which considers the convenience and scalability of operating system and operation maintenance
- Established optimal contents integration management system for live social media broadcasting and operation
(3) Expansion of related technology to cover mobile platform beyond PC platform
- Built a mobile version of the content library to extend useful PC version features that has good sales and revenue value to Mobile and view them in real time

G. Developed asset authoring tool
(1) Developed digital contents asset authoring tool for live social media broadcasting
- Optimized asset authoring tool for upload and download high quality 2D image and 3D objects freely
- Flexible contents interface for user convenience
(2) Developed a structure for uploading and downloading contents to a contents library
- Standardized of 2D image type asset production
- Standardized of 3D object type asset production
- Image data importer function
- 3D data importer function

Expected Contribution
1. B2C commercialization plan
(Honey Vue service management plan in the future)
A beta service of the Honey Vue launched October this year, and the Honey Vue service will be open in middle January next year. During a beta service, we will offer a variety of event benefits to our members and improve the inconveniences through feedback. Provide about 100 assets for free from the content library and update additional assets continuously to increase the number of content users. These assets will be produced in various styles with different tastes and characteristics according to the type of broadcasting and BJ so that they can be used universally.
The operating organization that manages the Honey Vue service is structured as shown below and will create employment by hiring new people to. Honey Vue service operation increases the number of users and members in the main target market and increases demands for services. One of the core parts of the Honey Vue service is 'Contents Library' which enables individual creators to upload freely and individual broadcasters to download freely.
Expecting to create new market within digital content market and to create new employment market.

2. Commercialization plan for B2B
A. Application and expansion to MCN business field
A new form of MCN relay platform is emerging to meet the needs of existing large MCN operators as well as small and medium enterprises wishing for social marketing. Build a service model that leverages Honey Vue for creator’s campaign who are the demand class for Honey Vue through these partnerships. Also, business model to provide custom contents (assets) for affiliate marketing with companies that are customers of MCN relay platform operators will be build.
UCONNEC jointed with strategic business agreement with host organization and it provides a platform service that connects companies and creators (broadcasting BJ), and provides a service to match the company's marketing / advertising requests so that appropriate creators can promote it. UCONNEC is a startup group with about 100 creators and plan to use it as activate Honey Vue’s solution and contents. Honey Vue will also be able to effectively advertise this service along with broadcast content with the power of SNS due to the business dismissal with MCN operators and to attract more users.
Provide Honey Vue digital contents which are optimized for corporate marketing/advertising, helping to promote brand promotion and marketing campaigns with the most appropriate creators and digital content and strengthen the brand's message further. Through this, BJ can gain synergy by increasing marketing and advertising effectiveness and make profits. By maximizing marketing and advertising effectiveness, companies can create synergies with content creators and companies positively impacting each other.
Honey Vue allows consumers to apply various effects freely expressed in the airwave broadcasting and others to meet consumer needs in the 'live social media market' that was not met by the weaknesses of existing technology. In addition, expand the market with applications related to profit generation model that plays a role as a marketing tool is planned.

B. Application and expansion to home shopping business field.
With the flexibility of Honey Vue and its maximized application, it can be easily applied to home shopping broadcasting service which is a similar industry. It is expected that this will serve as a driving force for expanding and developing not only the linked industry but also the 'digital contents market' and 'media market as a whole'.

Honey Vue plans to launch a Honey Vue solution to the fashion e-commerce platform with a strategic business agreement (MOU) with Chunil Networks, which is preparing for a new online live sales platform specialized in the Chinese online commerce market. Platform + from Cheonil Networks is preparing MCN business to export beauty and fashion themed contents to the biggest fashion platform in China such as Taobao and Honey Vue's real- time AR composite effect will be applied to provide fun elements to broadcasting contents for beauty and fashion and the goal is to increase satisfaction of viewers and increase sales.

The Honey Vue solution which is applied to home shopping broadcasts that provide on-line real-time one-person broadcasting service, is used as a marketing tool and helps to increase sales. Viewers can use the synthetic digital content elements that were not possible in real time and use the digital contents of Honey Vue in the situation that it is difficult to be real-time broadcasting, such as vacating for a while or changing clothes to increase viewer’s satisfaction.

Through Honey Vue’s broadcasting system that variety of designs and items can be applied in real time for internet broadcasting BJs to express their personality. Also reviewing to create templets in digital contents to promote and introduce products for company and sponsor event and advertising assets.

3. Overseas commercialization plan
Honey Vue wanted to export to Japan and Malaysia including China in the future and applied for international patent in Japan and Malaysia. As a localization stage of the business, expanding the service platform in local language is the first thing to do and need to participate related fairs and press releases to match and cooperate with business partners for commercialization.

Media Front, a company who organized Honey Vue platform service has an office in Japan to use local infrastructure and the advantage of being able to conduct local marketing and marketing optimized. Japan has a particularly high quality animation and video digital content market so if you use local infrastructure actively for Honey Vue services, it will provide satisfactory digital contents to the one-person live social media market that is developing worldwide.

Malaysia is also one of the countries where IT is rapidly developing and many global companies are investing as much as Silicon Valley and the government is reforming the system into an information-oriented society so the outlook for the IT industry is positive. Especially, it has high interest in e-learning and education infrastructure program so it has potential to commercialize it in order to connect industry and profit with application characteristics of Honey Vue. In addition, it is expected that there will be needs and demand for interesting communication tools based on the fact that locals use mobile communication service and SNS service is high, and environment where Honey Vue service is available.

(출처 : SUMMARY 19p)

목차
Contents
▼

• 표지 ... 1
• 제 출 문 ... 2
• 보고서 요약서 ... 3
• 연구개발사업 주요 연구성과 ... 4
• 국문 요약문 ... 11
• SUMMARY ... 19
• 목차 ... 28
• 제1장. 연구개발과제의 개요 ... 29
• 1. 연구개발 목적 ... 29
• 2. 연구개발의 필요성 ... 29
• 3. 연구개발 범위 ... 30
• 제2장. 국내외 기술 개발 현황 ... 35
• 제3장. 연구 수행 내용 및 성과 ... 42
• 1. 연구 개발 ... 42
• 2. 사업화 현황 ... 62
• 제4장. 목표 달성도 및 관련 분야 기여도 ... 64
• 1. 정량적 성과 목표 및 달성도 ... 64
• 2. 관련 분야 기여도 ... 71
• 제5장. 연구개발성과의 활용계획 ... 78
• 제6장. 연구 과정에서 수집한 해외 과학기술 정보 ... 90
• 제7장. 연구개발성과의 보안등급 ... 91
• 제8장. 국가과학기술종합정보시스템에 등록한 연구시설·장비 현황 ... 92
• 제9장. 연구개발과제 수행에 따른 연구실 등의 안전 조치 이행 실적 ... 93
• 제10장. 연구개발과제의 대표적 연구 실적 ... 94
• 제11장. 기타 사항 ... 95
• 제12장. 참고 문헌 ... 96
• 붙임 1. 자체보안관리 진단표 ... 97
• 붙임 2. 자체연구실 안전조치 이행표 ... 98
• 끝페이지 ... 99

참고문헌

• 전체(0)
• 논문(0)
• 특허(0)
• 보고서(0)