Methods and systems for assigning pixels distance-cost values using a flood fill technique
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06K-009/34
G06T-007/00
G06K-009/00
G06T-007/40
G06K-009/46
G06K-009/62
출원번호
US-0716495
(2015-05-19)
등록번호
US-9563962
(2017-02-07)
발명자
/ 주소
Lin, Dennis
Nguyen, Quang
Dang, Gia
Zhang, Yi
Venshtain, Simion
Nguyen, Cong
출원인 / 주소
PERSONIFY, INC.
대리인 / 주소
Invention Mine LLC
인용정보
피인용 횟수 :
0인용 특허 :
102
초록▼
Disclosed herein are methods and systems for assigning pixels distance-cost values using a flood fill technique. One embodiment takes the form of a process that includes obtaining video data depicting a head of a user, obtaining depth data associated with the video data, and selecting seed pixels fo
Disclosed herein are methods and systems for assigning pixels distance-cost values using a flood fill technique. One embodiment takes the form of a process that includes obtaining video data depicting a head of a user, obtaining depth data associated with the video data, and selecting seed pixels for a flood fill at least in part by using the depth information. The process also includes performing the flood fill from the selected seed pixels. The flood fill assigns respective distance-cost values to pixels of the video data based on position-space cost values and color-space cost values. In some embodiments, the process also includes classifying pixels of the video data as foreground based at least in part on the assigned distance-cost values. In some other embodiments, the process also includes assigning pixels of the video data foreground-likelihood values based at least in part on the assigned distance-cost values.
대표청구항▼
1. A method comprising: obtaining video data depicting a head of a user;obtaining depth data associated with the video data;selecting seed pixels for a hair-identification flood fill for identifying pixels depicting hair of the head of the user, the seed pixels selected at least in part by using the
1. A method comprising: obtaining video data depicting a head of a user;obtaining depth data associated with the video data;selecting seed pixels for a hair-identification flood fill for identifying pixels depicting hair of the head of the user, the seed pixels selected at least in part by using the obtained depth data, wherein the selected seed pixels have intermittent depth values that are within a threshold tolerance of a depth value corresponding to the head of the user;performing the hair-identification flood fill from the selected seed pixels, the hair-identification flood fill assigning respective distance-cost values to pixels of the video data based on respective position-space-cost values and respective color-space-cost values; andidentifying a persona of the user from the video data based at least in part on the respective distance-cost values assigned by the hair-identification flood fill. 2. The method of claim 1, wherein selecting seed pixels for the hair-identification flood fill further comprises selecting seed pixels for the hair-identification flood fill at least in part by using the video data. 3. The method of claim 1, further comprising: obtaining a head contour that estimates an outline of the depicted head of the user, the head contour being based at least in part on the depth data associated with the video data,wherein the selected seed pixels are on an upper contour, wherein the upper contour is an upper portion of the head contour. 4. The method of claim 3, wherein the selected seed pixels are equally distributed along the upper contour. 5. The method of claim 3, wherein the selected seed pixels are of colors that are found in a user-hair-color model. 6. The method of claim 1, wherein selecting seed pixels for the hair-identification flood fill comprises: identifying pixels having noisy depth values over a series of frames; andselecting the identified noisy depth-pixels as seed pixels for the hair-identification flood fill. 7. The method of claim 6, wherein the selected seed pixels are located within an extended head box. 8. The method of claim 6, wherein the selected seed pixels are of colors that are found in a user-hair-color model. 9. The method of claim 1, wherein a first set of the selected seed pixels are on an upper contour and a second set of the selected seed pixels have noisy depth values over a series of frames, wherein the upper contour is an upper portion of a head contour that estimates an outline of the depicted head of the user, the method further comprising: initializing the distance-cost values of the seed pixels in the first set to be zero; andinitializing the distance-cost values of the seed pixels in the second set to be non-zero. 10. The method of claim 1, wherein a distance-cost value of a given pixel comprises (i) a position-space cost value from a seed pixel to the given pixel and (ii) a color-space cost value from the seed pixel to the given pixel. 11. The method of claim 1, wherein performing the hair-identification flood fill comprises: identifying a plurality of neighbor pixels of a current pixel;determining respective step-cost values from the current pixel to each pixel in the plurality of neighbor pixels; andassigning each pixel in the plurality of neighbor pixels a respective distance-cost value based on a distance-cost value of the current pixel and the respective step-cost values. 12. The method of claim 1, wherein performing the hair-identification flood fill comprises: determining a minimum distance-cost value from at least one of the selected seed pixels to a current pixel; andassigning the current pixel a distance-cost value that is the determined minimum distance-cost value. 13. The method of claim 12, wherein determining a minimum distance-cost value comprises: comparing a current distance-cost value corresponding with a current flood-fill path to a prior distance-cost value corresponding with a prior flood-fill path. 14. The method of claim 13, wherein the current flood-fill path and the prior flood-fill path originate from a common seed pixel. 15. The method of claim 13, wherein the current flood-fill path and the prior flood-fill path originate from different seed pixels. 16. The method of claim 1, wherein performing the hair-identification flood fill comprises performing the hair-identification flood fill along a plurality of flood-fill paths, the method further comprising: terminating the hair-identification flood fill along a current flood-fill path in response to at least one termination criteria, the termination criteria comprising: a current pixel not being a user-hair color according to a user-hair-color model;the current pixel being a background color according to a background-color model;a distance-cost value to the current pixel being greater than a distance-cost threshold; anda step-cost value to the current pixel being greater than a step-cost threshold. 17. The method of claim 1, wherein identifying the persona of the user from the video data based at least in part on the respective distance-cost values assigned by the hair-identification flood fill comprises classifying pixels of the video data as foreground based at least in part on the assigned distance-cost values. 18. The method of claim 1, wherein identifying the persona of the user from the video data based at least in part on the respective distance-cost values assigned by the hair-identification flood fill comprises assigning pixels of the video data foreground-likelihood values based at least in part on the assigned distance-cost values. 19. A system comprising: a communication interface;a processor; anddata storage containing instructions executable by the processor for causing the system to carry out a set of functions, the set of functions including: obtaining video data depicting a head of a user;obtaining depth data associated with the video data;selecting seed pixels for a hair-identification flood fill for identifying pixels depicting hair of the head of the user, the seed pixels selected at least in part by using the depth data, wherein the selected seed pixels have intermittent depth values that are within a threshold tolerance of a depth value corresponding to the head of the user;performing the hair-identification flood fill from the selected seed pixels, the hair-identification flood fill assigning respective distance-cost values to pixels of the video data based on respective position-space cost values and respective color-space cost values; andidentifying a persona of the user from the video based at least in part on the respective distance-cost values assigned by the hair-identification flood fill. 20. The system of claim 19, wherein selecting seed pixels for the hair-identification flood fill further comprises selecting seed pixels for the hair-identification flood fill at least in part by using the video data. 21. The system of claim 19, the set of functions further comprising: obtaining a head contour that estimates an outline of the depicted head of the user, the head contour being based at least in part on the depth data associated with the video data,wherein the selected seed pixels are on an upper contour, wherein the upper contour is an upper portion of the head contour. 22. The system of claim 21, wherein the selected seed pixels are equally distributed along the upper contour. 23. The system of claim 21, wherein the selected seed pixels are of colors that are found in a user-hair-color model. 24. The system of claim 19, wherein selecting seed pixels for the hair-identification flood fill comprises: identifying pixels having noisy depth values over a series of frames; andselecting the identified noisy depth-pixels as seed pixels for the hair-identification flood fill. 25. The system of claim 24, wherein the selected seed pixels are located within an extended head box. 26. The system of claim 24, wherein the selected seed pixels are of colors that are found in a user-hair-color model. 27. The system of claim 19, wherein a first set of the selected seed pixels are on an upper contour and a second set of the selected seed pixels have noisy depth values over a series of frames, wherein the upper contour is an upper portion of a head contour that estimates an outline of the depicted head of the user, the set of functions further comprising: initializing the distance-cost values of the seed pixels in the first set to be zero; andinitializing the distance-cost values of the seed pixels in the second set to be non-zero. 28. The system of claim 19, wherein a distance-cost value of a given pixel comprises (i) a position-space cost value from a seed pixel to the given pixel and (ii) a color-space cost value from the seed pixel to the given pixel. 29. The system of claim 19, wherein performing the hair-identification flood fill comprises: identifying a plurality of neighbor pixels of a current pixel;determining respective step-cost values from the current pixel to each pixel in the plurality of neighbor pixels; andassigning each pixel in the plurality of neighbor pixels a respective distance-cost value based on a distance-cost value of the current pixel and the respective step-cost values. 30. The system of claim 19, wherein performing the hair-identification flood fill comprises: determining a minimum distance-cost value from at least one of the selected seed pixels to a current pixel; andassigning the current pixel a distance-cost value that is the determined minimum distance-cost value. 31. The system of claim 30, wherein determining a minimum distance-cost value comprises: comparing a current distance-cost value corresponding with a current flood-fill path to a prior distance-cost value corresponding with a prior flood-fill path. 32. The system of claim 31, wherein the current flood-fill path and the prior flood-fill path originate from a common seed pixel. 33. The system of claim 31, wherein the current flood-fill path and the prior flood-fill path originate from different seed pixels. 34. The system of claim 19, wherein performing the hair-identification flood fill comprises performing the hair-identification flood fill along a plurality of flood-fill paths, the set of functions further comprising: terminating the hair-identification flood fill along a current flood-fill path in response to at least one termination criteria, the termination criteria comprising: a current pixel not being a user-hair color according to a user-hair-color model;the current pixel being a background color according to a background-color model;a distance-cost value to the current pixel being greater than a distance-cost threshold; anda step-cost value to the current pixel being greater than a step-cost threshold. 35. The system of claim 19, wherein identifying the persona of the user from the video data based at least in part on the respective distance-cost values assigned by the hair-identification flood fill comprises classifying pixels of the video data as foreground based at least in part on the assigned distance-cost values. 36. The system of claim 19, wherein identifying the persona of the user from the video data based at least in part on the respective distance-cost values assigned by the hair-identification flood fill comprises assigning pixels of the video data foreground-likelihood values based at least in part on the assigned distance-cost values.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (102)
Cipolla Roberto (Cambridge GBX) Okamoto Yasukazu (Chiba-ken JPX) Kuno Yoshinori (Osaka-fu JPX), 3D human interface apparatus using motion recognition based on dynamic image processing.
Panahpour Tehrani, Mehrdad; Ishikawa, Akio; Sakazawa, Shigeyuki, Apparatus, method and computer program for classifying pixels in a motion picture as foreground or background.
Clanton,Charles H.; Ventrella,Jeffrey J.; Paiz,Fernando J., Cinematic techniques in avatar-centric communication during a multi-user online simulation.
DeMenthon Daniel F. (Columbia MD), Computer vision system for position monitoring in three dimensions using non-coplanar light sources attached to a monito.
Tian, Dihong; Mauchly, J. William; Friel, Joseph T., Generating and rendering synthesized views with multiple video streams in telepresence video conference sessions.
Iwamoto, Masayuki; Fujimura, Koichi, Image processing apparatus, method for processing and image and computer-readable recording medium for causing a computer to process images.
Carter, James; Yaacob, Arik; Darrah, James F., Managing the layout of multiple video streams displayed on a destination display screen during a videoconference.
Bang, Gun; Um, Gi-Mun; Chang, Eun-Young; Kim, Taeone; Hur, Nam-Ho; Kim, Jin-Woong; Lee, Soo-In, Method and apparatus for improving quality of depth image.
Haskell, Barin Geoffry; Puri, Atul; Schmidt, Robert Lewis, Scene description nodes to support improved chroma-key shape representation of coded arbitrary images and video objects.
Mackie, David J.; Tian, Dihong; Weir, Andrew P.; Buttimer, Maurice; Friel, Joseph T.; Mauchly, J. William; Chen, Wen-Hsiung, System and method for providing enhanced video processing in a network environment.
Weiser, Reginald; McGravie, Richard; Diouskine, Roman; Teboul, Jeremy, Systems and methods for providing video conferencing services via an ethernet adapter.
Rudolph, Eric; Rui, Yong; Malvar, Henrique S; He, Li Wei; Cohen, Michael F; Tashev, Ivan, Systems and methods for real-time audio-visual communication and data collaboration in a network conference environment.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.