보고서 정보
주관연구기관 |
한국과학기술원 Korea Advanced Institute of Science and Technology |
연구책임자 |
장샤를바장
|
보고서유형 | 최종보고서 |
발행국가 | 대한민국 |
언어 |
대한민국
|
발행년월 | 2020-05 |
과제시작연도 |
2019 |
주관부처 |
과학기술정보통신부 Ministry of Science and ICT |
과제관리전문기관 |
한국연구재단 National Research Foundation of Korea |
등록번호 |
TRKO202100019893 |
과제고유번호 |
1711085918 |
사업명 |
개인기초연구(과기정통부)(R&D) |
DB 구축일자 |
2022-03-26
|
키워드 |
Drone.Artifical Intelligence (Deep learning).Computer Vision.Computer Graphics.Robotics.HCI.Video capture.Video editing.
|
초록
Abstract
▼
□ 연구개요
In this NRF project, we proposed and investigated to leverage the great potential of Artificial Intelligence (AI) to create new smart computational techniques in order to simplify and automatize drone video acquisition and editing for casual users.
Drones are getting more and more popul
□ 연구개요
In this NRF project, we proposed and investigated to leverage the great potential of Artificial Intelligence (AI) to create new smart computational techniques in order to simplify and automatize drone video acquisition and editing for casual users.
Drones are getting more and more popular for video acquisition: they are relatively cheap, portable, provide various views of a scene (from ground level to high altitude), and can operate in different kinds of environment (e.g. urban and countryside). They can also be used in different contexts and applications, for example during sport activities acquisition or to capture Hollywood-grade movies.
First, in the context of video capture, drones are still mainly manually controlled, which is time-consuming and reserved for skilled expert users. To solve these limitations, we first developed a novel computational technique for drone video capture. Our technique simplifies video acquisition by drones to automatically capture subjective shots traditionally captured by professional human camera operators.
Second, once the drone videos are captured, video editing is a common process. However it can be time-consuming and difficult, especially for casual users. Motivated by the great popularity and demand of 360-degree VR contents, we focused on the automatic stabilization and upright rectification of 360-degree VR videos captured by drones. Due to the vibrations and dynamic orientation of the drone, the captured videos are shaky and tilted. To solve these challenges, we developed a novel algorithm based on Artificial Intelligence (deep learning) that automatically returns a stabilized, upright version of the input video.
Third, beyond capture and editing, we proposed an AI pipeline that can automatically retrieve drone videos with specific properties, such as indoor and orbiting motion. For this, we need to train AI algorithms with numerous examples, and thus we prepared a large-scale, multi-purpose, annotated dataset of drone videos.
□ 연구 목표 대비 연구결과
Our project has been composed of five main tasks. In the following, we list them and write the outputs.
-Year 1
Task#1: Goal: drone platform set up, and initial experiments with automatic end-to-end systems for drone video capture and editing.
==> We successfully set up our drone platform with end-to-end control, and used it for preliminary experiments such as automatically following an actor with drone by tracking the face.
-Year 2
Task#2: 360-degree VR drone video stabilization and rectification.
==> We successfully developed the first AI deep learning-based approach to automatically return a stabilized upright version of a 360-degree VR video captured by a drone. We published a paper at the premier IEEE Virtual Reality 2019 international conference (acceptance rate around 20%). We prepared an extension of this work and are about to submit it to IEEE Transactions on Image Processing (TIP), one of the top journals in Computer Vision (impact factor of 6.79). We also filed one Korean patent and one US patent.
-Year 3
Task#3: drone imitation of cinematographic subjective shots.
==> We successfully proposed the first computational approach to capture subjective first-person view (FPV) videos by drones. This work has been conducted in close collaboration with ETH Zurich, Switzerland. We submitted our work to ACM Transactions on Graphics (TOG), the top journal in Computer Graphics (impact factor of 6.495). Our paper is currently under review.
Task#4: prepare a large-scale drone video dataset with annotation.
==> We successfully prepared the world’s largest, multi-purpose, annotated drone video clip dataset.
It is composed of more than 18,000 drone video clips that offer a great variety in terms of location, appearance, motion, scene category, shot type. In addition, our annotations include various levels of information such as social platform metadata, manual annotation on video editing (e.g. cut time, transition and logo presence), manual annotation on scene description (e.g. scene category and location). We prepared a manuscript based on this task and task#5.
Task#5: analysis of this dataset using machine learning techniques.
==> We experimented and evaluated several methods to analyze our dataset, such as automatic camera path by SLAM and 3D scene reconstruction. In addition, we leveraged our dataset to train some AI deep learning-based methods for image/video classification, such as indoor vs. outdoor, drone vs. non-drone, etc. Based on this analysis, we proposed an AI pipeline that can automatically retrieve drone videos with specific properties, such as indoor and orbiting motion. We prepared a manuscipt based on task#4 and #5, and are about to submit it to IEEE Transactions on Image Processing, one of the top journals in Computer Vision (impact factor of 6.79).
Conclusion: we have successfully completed the five tasks of the project.
□ 연구개발 결과의 중요성
Our research topic is challenging, highly creative, and valuable. It is clearly challenging, especially given the various concepts from different fields that need to be used and improved, and the physical constraints of drones. Automatic drone video capture and editing is a brand new research area that has been studied only a very little, therefore it is highly creative and offer great research opportunities. The drone market represents several billions of dollars per year, and there is a strong need of automatic video capture and editing from the market, therefore our topic is highly relevant and valuable, in both academia and industry. We published 1 international conference paper (IEEE VR), have 1 journal paper under review (TOG), and 2 journal papers are about to be submitted (TIP). The manuscripts are in the supplementary material. We also set up a close collaboration with ETH Zurich, one of the best universities of the world.
(출처 : 연구결과 요약문 2p)
목차 Contents
- COVER ... 1
- 연구결과 요약문 ... 2
- 목차 ... 3
- 1. 연구개발과제의 개요 ... 4
- A) Background, motivation and goal ... 4
- B) Related work ... 5
- 2. 연구수행내용 및 연구결과 ... 5
- 3. 연구개발결과의 중요성 ... 8
- 4. 참고문헌 ... 9
- Appendix ... 10
- 5. 연구성과 ... 11
- 대표적 연구실적 ... 14
- End of Page ... 20
※ AI-Helper는 부적절한 답변을 할 수 있습니다.