[논문]Design of Image Generation System for DCGAN-Based Kids' Book Text

Cho, Jaehyeon; Moon, Nammee

doi:10.3745/jips.02.0149

[국내논문] Design of Image Generation System for DCGAN-Based Kids' Book Text 원문보기

Journal of information processing systems, v.16 no.6, 2020년, pp.1437 - 1446

Cho, Jaehyeon (Division of Computer Engineering, Hoseo University) , Moon, Nammee (Division of Computer and Information Engineering, Hoseo University)

Abstract ▼ AI-Helper

For the last few years, smart devices have begun to occupy an essential place in the life of children, by allowing them to access a variety of language activities and books. Various studies are being conducted on using smart devices for education. Our study extracts images and texts from kids' book with smart devices and matches the extracted images and texts to create new images that are not represented in these books. The proposed system will enable the use of smart devices as educational media for children. A deep convolutional generative adversarial network (DCGAN) is used for generating a new image. Three steps are involved in training DCGAN. Firstly, images with 11 titles and 1,164 images on ImageNet are learned. Secondly, Tesseract, an optical character recognition engine, is used to extract images and text from kids' book and classify the text using a morpheme analyzer. Thirdly, the classified word class is matched with the latent vector of the image. The learned DCGAN creates an image associated with the text.

주제어

표/그림 (13)

그림 Fig. 1. A kid is being educated using smart device.
그림 Fig. 2. Reading text from an image using OCR.
그림 Fig. 3. Display a parse tree.
그림 Fig. 4. Principle of learning process of GAN.
그림 Fig. 5. DCGAN generator structure.
그림 Fig. 6. Image generation using DCGAN.
그림 Fig. 7. System design flow chart.
그림 Fig. 8. ImageNet Bear dataset.
그림 Fig. 9. Basic DCGAN structure of this study.
그림 Fig. 10. Training feature map.
그림 Fig. 11. The kids' book, "But the bear stays up"
그림 Fig. 12. Text recognized in the corrected image.
그림 Fig. 13. Tagging result of character string using NLTK.

AI 본문요약
AI-Helper

* AI 자동 식별 결과로 적합하지 않은 문장이 있을 수 있으니, 이용에 유의하시기 바랍니다.

가설 설정

The study proposed text-image matching and image generation using DCGAN to create images not represented in kids’ book. By visualizing the text in kids’ book, the proposed system can help children use smart devices as educational media.

제안 방법

This study proposes the extraction of images and texts from kids’ book with the use of smart devices and creation of new images for text in kids’ book by matching images and text. Such a system will be of significant importance when using smart devices as an educational media for children.

대상 데이터

The generator noise was sampled at a 100-dimensional unit normal distribution. A minibatch size of 64 was used, and the training was conducted for 600 epochs. Fig.
In future studies, the data accuracy and object type will be enhanced with the addition of 8, 189 flower datasets and 6, 033 Caltech-UCSD Birds datasets from the Oxford 102 flowers dataset. With an expansion in scope, utilization would also expand, which can ultimately lead to the creation of images for school textbooks.
The kids’ book used in this study was “Bear Stays Up, ” which was released on October 5, 2004. As shown in Fig.
The image learning process proceeded to a dataset in ImageNet. The matching title and image using 1, 164 images were categorized into 11 titles. The image size was 64×64×3, basic learning rate was 0.

참고문헌 (18)

I. T. Kim and K. J. Yoo, "Effects of augmented reality picture book on the language expression and flow of young children's in picture book reading activities," The Journal of Korea Open Association for Early Childhood Education, vol. 23, no. 1, pp. 83-109, 2018.

상세보기
K. M. Ryu, H. J. Kim, H. J. Kim, E. J. Lee, and J. Y. Heo, "A development of interactive storybook with digital board and smart device," in Proceedings of the HCI Society of Korea, Pyeongchang, Korea, 2017, pp. 1179-1182.
Y. Kim and H. Park, "Study on the relation between young children's smart device immersion tendency and their playfulness," Early Childhood Education Research & Review, vol. 20, no. 4, pp. 337-353, 2016.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets," Advances in Neural Information Processing Systems, vol. 27, pp. 2672-2680, 2014.
R. Tachibana, T. Matsubara, and K. Uehara, "Semi-supervised learning using adversarial networks," in Proceedings of 2016 IEEE/ACIS 15th International Conference on Computer and Information Science (ICIS), Okayama, Japan, 2016, pp. 1-6.
M. S. Ko, H. K. Roh, and K. H. Lee, "GANMOOK: generative adversarial network to stylize images like ink wash painting," in Proceedings of the Korea Computer Congress, 2017, pp. 793-795.
L. C. Yang, S. Y. Chou, and Y. H. Yang, "MidiNet: a convolutional generative adversarial network for symbolic-domain music generation," in Proceedings of the 18th International Society of Music Information Retrieval Conference, Suzhou, China, 2017, pp. 324-331.
G. C. Lee and J. Yoo, "Development an Android based OCR application for Hangul food menu," Journal of the Korea Institute of Information and Communication Engineering, vol. 21, no. 5, pp. 951-959, 2017.

원문보기 상세보기
R. Smith, "An overview of the Tesseract OCR engine," in Proceedings of the 9th International Conference on Document Analysis and Recognition (ICDAR), 2007, Parana, Brazil, pp, 629-633.
A. C. Rodriguez, T. Kacprzak, A. Lucchi, A. Amara, R. Sgier, J. Fluri, T. Hofmann, and A. Refregier, "Fast cosmic web simulations with generative adversarial networks," Computational Astrophysics and Cosmology, vol. 5, article no. 4, 2018.

상세보기
R. Yamashita, M. Nishio, R. K. G. Do, and K. Togashi, "Convolutional neural networks: an overview and application in radiology," Insights into Imaging, vol. 9, no. 4, pp. 611-629, 2018.

상세보기
A. Radford, L. Metz, and S. Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks," 2015 [Online]. Available: https://arxiv.org/abs/1511.06434.
Y. Han and H. J. Kim, "Face morphing using generative adversarial networks," Journal of Digital Contents Society, vol. 19, no. 3, pp. 435-443, 2018.
S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee, "Generative adversarial text to image synthesis," in Proceedings of the 33nd International Conference on Machine Learning (ICML), New York, NY, 2016, pp. 1060-1069.
J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller, "Striving for simplicity: the all convolutional net," 2014 [Online]. Available: https://arxiv.org/abs/1412.6806.
D. Triantafyllidou and A. Tefas, "Face detection based on deep convolutional neural networks exploiting incremental facial part learning," in Proceeding of 2016 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico, 2016, pp, 3560-3565.
E. Learned-Miller, G. B. Huang, A. RoyChowdhury, H. Li, and G. Hua, "Labeled faces in the wild: a survey," Advances in Face Detection and Facial Image Analysis. Cham, Switzerland: Springer, 2016, pp. 189-248.
Y. Susanti, T. Tokunaga, H. Nishikawa, and H. Obari, "Automatic distractor generation for multiple-choice English vocabulary questions," Research and Practice in Technology Enhanced Learning, vol. 13, article no. 15, 2018.

상세보기

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증