초록

Abstract ▼

연구개요
We introduce a neural framework and associated architectures for learning semantic part segmentation of 3D shape geometry, based solely on part referential language. We exploit the fact that linguistic descriptions of a shape can provide priors on the shape’s parts. A Transformer-based atten

연구개요
We introduce a neural framework and associated architectures for learning semantic part segmentation of 3D shape geometry, based solely on part referential language. We exploit the fact that linguistic descriptions of a shape can provide priors on the shape’s parts. A Transformer-based attention module that is carefully incorporated into our framework can precisely highlight the semantic part or parts described in natural language. We demonstrate that our method achieves 3D semantic segmentation results comparable to those achieved by supervised learning without any direct geometry annotations.

연구 목표대비 연구결과
Beyond studying the capability of neural networks to jointly understand language and shape, this work also suggests a new potential way to collect data for object part segmentation. Object or scene segmentation is a fundamental problem in many vision tasks, but the advance of learning-based segmentation techniques is gated by the availability of large-scale human segmentation annotations of 2D images or 3D models. Particularly for 3D, collecting manual annotations on 3D objects requires a huge amount of human effort and cost. In contrast to this, uttering a language description is a much more natural way for people to provide information about object structure and geometry.
In this work, we achieves 79.4% mIoU 3D semantic part segmentation result for Chair class in ShapeNet dataset, which is comparable to the result obtained by supervised learning, 89.6% mIoU of PointNet []. It is worth noting that our segmentation results are obtained without any direct geometry supervision.
Moreover, we demonstrate that 3D part-level geometry prior learned by natural language is zero-shot generalizable to out-of-distribution categories of shapes specifically, with training done on – Chairs, good segmentic masks can be extracted out of instances of Tables and Lamps.

연구개발성과의 활용 계획 및 기대효과(연구개발결과의 중요성)
Our framework, PartGlot, achieves 3D part segmentation results comparable to those of supervised learning methods solely from referential language of an object without any explicit supervision of its part segmentations. That is, our approach opens the possibility of learning 3D shape parts from natural language alone, which requires much less human labor when collecting data. Thus, we believe that PartGlot facilitates 3D part annotation acquisition.
Beyond achieving comparable 3D part segmentation results on a training category of 3D shapes, PartGlot has also demonstrated its zero-shot generalization to other shape categories. Reflecting this capability, PartGlot trained on a single category can be deployed for part annotation acquisition of other categories in a zero-shot manner.

(source : Summary 2p)

목차 Contents

• COVER ... 1
• 연구결과 요약문 ... 2
• 목차 ... 3
• 1. 연구개발과제의 개요 ... 4
• 2. 연구개발과제의 수행 과정 및 수행 내용 ... 5
• 2-1. Background and Overview ... 5
• 2-2. Part-Name-Agnostic (PN-Agnostic) Learning ... 6
• 2-3. Part-Name-Aware (PN-Aware) Learning ... 6
• 3. 연구개발과제의 수행 결과 및 목표 달성 수준 ... 7
• 1) 정성적 연구개발성과(연구개발결과) ... 7
• 2) 세부 정량적 연구개발성과 ... 13
• 3) 목표 달성 수준 ... 13
• 4. 연구개발성과의 관련 분야에 대한 기여 정도(연구개발결과의 중요성) ... 13
• 5. 연구개발성과의 관리 및 활용 계획 ... 13
• 6. 자체점검표 ... 14
• 7. 참고문헌 ... 15
• [붙임1] 세부 정량적 연구개발성과 ... 16
• [붙임2] 연구책임자 대표적 연구실적 및 증빙(요약문 및 사본) ... 19
• End of Page ... 22