Systems and methods for embedding a foreground video into a background feed based on a control input
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
H04N-005/272
H04N-001/00
H04N-013/04
G06F-003/01
G02B-027/00
G06T-005/50
G06T-007/00
G06T-007/11
G06T-007/194
출원번호
US-0076264
(2011-03-30)
등록번호
US-9628722
(2017-04-18)
발명자
/ 주소
Do, Minh N.
Nguyen, Quang Il.
Lin, Dennis
Patel, Sanjay I.
출원인 / 주소
PERSONIFY, INC.
대리인 / 주소
Invention Mine LLC
인용정보
피인용 횟수 :
3인용 특허 :
108
초록▼
A color image and a depth image of a live video are received. Each of the color image and the depth image are processed to identify the foreground and the background of the live video. The background of the live video is removed in order to create a foreground video that comprises the foreground of
A color image and a depth image of a live video are received. Each of the color image and the depth image are processed to identify the foreground and the background of the live video. The background of the live video is removed in order to create a foreground video that comprises the foreground of the live video. A control input may be received to control the embedding of the foreground video into a second background from a background feed. The background feed may also comprise virtual objects such that the foreground video may interact with the virtual objects.
대표청구항▼
1. A method for selectively embedding a foreground video into a background feed, the method comprising: receiving, using a computer, depth and color information of a first video from at least one camera;creating a foreground video by identifying a foreground portion and a background portion of the f
1. A method for selectively embedding a foreground video into a background feed, the method comprising: receiving, using a computer, depth and color information of a first video from at least one camera;creating a foreground video by identifying a foreground portion and a background portion of the first video based on the depth and color information of the first video, the creation of the foreground video comprising an inclusion of the foreground portion of the first video and a removal of the background portion of the first video;receiving the background feed comprising a second background;detecting, in the foreground video, a control input from a user;determining, based on the control input, whether or not to embed the foreground video into the second background;in response to a determination to embed the foreground video, creating an output video by embedding the foreground video into the second background; andin response to a determination to not embed the foreground video, creating the output video by using the second background without embedding the foreground video. 2. The method as set forth in claim 1, wherein the control input comprises a gesture from the user, the gesture detected by using the depth information of the first video. 3. The method as set forth in claim 1, wherein the control input comprises a depth of the foreground portion of the first video, the foreground portion is embedded into the background feed if the depth of the foreground portion is below a threshold value, the foreground portion is not embedded into the background feed if the depth of the foreground portion is above the threshold value. 4. The method as set forth in claim 1, wherein the control input comprises body motion of the user, the body motion comprising at least one of the user's horizontal movement, depth movement, and entering of a hot spot, the depth movement comprising the user moving forwards or backwards, the hot spot comprising a two dimensional (2D) or a three dimensional (3D) placement of the user, the 2D placement comprising a defined horizontal positioning of the user, the 3D placement comprising a defined horizontal and depth placement of the user. 5. The method as set forth in claim 1, wherein the background feed comprises a virtual object for changing the background feed to a third background, a placement of a user hand from the foreground video on the virtual object in the background feed changes the background feed to the third background. 6. The method as set forth in claim 1, further comprising adding, from a virtual object authoring module, a virtual object to the second background, the virtual object added is assigned a required user action and a virtual object effect. 7. The method as set forth in claim 1, wherein the background feed comprises at least one virtual object, each virtual object comprises a required user action and a virtual object effect, the foreground portion comprises a depiction of the user such that if the user from the foreground portion performs the required user action, the virtual object effect is activated in the output video. 8. The method as set forth in claim 1, wherein the second background comprises at least one of a presentation slide, video, shared computer desktop, and virtual scene. 9. The method as set forth in claim 1, wherein the depth information comprises a depth image for providing a distance of each pixel from the at least one camera. 10. A system for selectively embedding a foreground video into a background feed, the system comprising: at least one computer; anda non-transitory computer readable medium having stored thereon one or more programs, which when executed by the at least one computer, cause the at least one computer to:receive depth and color information of a first video from at least one camera;create a foreground video by identifying a foreground portion and a background portion of the first video based on the depth and color information of the first, the creation of the foreground video comprising an inclusion of the foreground portion of the first video and a removal of the background portion of the first video;receive the background feed comprising a second background;detect, in the foreground video, a control input from a user;determine, based on the control input, whether or not to embed the foreground video into the second background;in response to a determination to embed the foreground video, create an output video by embedding the foreground video into the second background; andin response to a determination to not embed the foreground video create the output video by using the second background without embedding the foreground video. 11. The system as set forth in claim 10, wherein the control input comprises a gesture from the user, the gesture detected by using the depth information of the first video. 12. The system as set forth in claim 10, wherein the control input comprises a depth of the foreground portion of the first video, the foreground portion is embedded into the background feed if the depth of the foreground portion is below a threshold value, the foreground portion is not embedded into the background feed if the depth of the foreground portion is above the threshold value. 13. The system as set forth in claim 10, wherein the control input comprises body motion of the user, the body motion comprising at least one of the user's horizontal movement, depth movement, and entering of a hot spot, the depth movement comprising the user moving forwards or backwards, the hot spot comprising a two dimensional (2D) or a three dimensional (3D) placement of the user, the 2D placement comprising a defined horizontal positioning of the user, the 3D placement comprising a defined horizontal and depth placement of the user. 14. The system as set forth in claim 10, wherein the background feed comprises a virtual object for changing the background feed to a third background, a placement of a user hand from the foreground video on the virtual object in the background feed changes the background feed to the third background. 15. The system as set forth in claim 10, further comprising a virtual object authoring module for adding a virtual object to the second background, the virtual object added is assigned a required user action and a virtual object effect. 16. The system as set forth in claim 10, wherein the background feed comprises at least one virtual object, each virtual object comprises a required user action and a virtual object effect, the foreground portion comprises a depiction of the user such that if the user from the foreground portion performs the required user action, the virtual object effect is activated in the output video. 17. The system as set forth in claim 10, wherein the second background comprises at least one of a presentation slide, video, shared computer desktop, and virtual scene. 18. The system as set forth in claim 10, wherein the depth information comprises a depth image for providing a distance of each pixel from the at least one camera. 19. A non-transitory computer readable medium carrying one or more instructions for selectively embedding a foreground video into a background feed, wherein the one or more instructions, when executed by one or more processors, cause the one or more processors to perform the steps of: receiving depth and color information of a first video from at least one camera;creating a foreground video by identifying a foreground portion and a background portion of the first video based on the depth and color information of the first video, the creation of the foreground video comprising an inclusion of the foreground portion of the first video and a removal of the background portion of the first video;receiving the background feed comprising a second background;detecting, in the foreground video, a control input from a user;determining, based on the control input, whether or not to embed the foreground video into the second background;in response to a determination to embed the foreground video, creating an output video by embedding the foreground video into the second background; andin response to a determination to not embed the foreground video, creating the output video by using the second background without embedding the foreground video. 20. The non-transitory computer readable medium as set forth in claim 19, wherein the control input comprises a gesture from the user, the gesture detected by using the depth information of the first video. 21. The non-transitory computer readable medium as set forth in claim 19, wherein the control input comprises a depth of the foreground portion of the first video, the foreground portion is embedded into the background feed if the depth of the foreground portion is below a threshold value, the foreground portion is not embedded into the background feed if the depth of the foreground portion is above the threshold value. 22. The non-transitory computer readable medium as set forth in claim 19, wherein the control input comprises body motion of the user, the body motion comprising at least one of the user's horizontal movement, depth movement, and entering of a hot spot, the depth movement comprising the user moving forwards or backwards, the hot spot comprising a two dimensional (2D) or a three dimensional (3D) placement of the user, the 2D placement comprising a defined horizontal positioning of the user, the 3D placement comprising a defined horizontal and depth placement of the user. 23. The non-transitory computer readable medium as set forth in claim 19, wherein the background feed comprises a virtual object for changing the background feed to a third background, a placement of a user hand from the foreground video on the virtual object in the background feed changes the background feed to the third background. 24. The non-transitory computer readable medium as set forth in claim 19, further comprising adding, from a virtual object authoring module, a virtual object to the second background, the virtual object added is assigned a required user action and a virtual object effect. 25. The non-transitory computer readable medium as set forth in claim 19, wherein the background feed comprises at least one virtual object, each virtual object comprises a required user action and a virtual object effect, the foreground portion comprises a depiction of the user such that if the user from the foreground portion performs the required user action, the virtual object effect is activated in the output video. 26. The non-transitory computer readable medium as set forth in claim 19, wherein the second background comprises at least one of a presentation slide, video, shared computer desktop, and virtual scene.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (108)
Cipolla Roberto (Cambridge GBX) Okamoto Yasukazu (Chiba-ken JPX) Kuno Yoshinori (Osaka-fu JPX), 3D human interface apparatus using motion recognition based on dynamic image processing.
Panahpour Tehrani, Mehrdad; Ishikawa, Akio; Sakazawa, Shigeyuki, Apparatus, method and computer program for classifying pixels in a motion picture as foreground or background.
Clanton,Charles H.; Ventrella,Jeffrey J.; Paiz,Fernando J., Cinematic techniques in avatar-centric communication during a multi-user online simulation.
DeMenthon Daniel F. (Columbia MD), Computer vision system for position monitoring in three dimensions using non-coplanar light sources attached to a monito.
Tian, Dihong; Mauchly, J. William; Friel, Joseph T., Generating and rendering synthesized views with multiple video streams in telepresence video conference sessions.
Iwamoto, Masayuki; Fujimura, Koichi, Image processing apparatus, method for processing and image and computer-readable recording medium for causing a computer to process images.
Carter, James; Yaacob, Arik; Darrah, James F., Managing the layout of multiple video streams displayed on a destination display screen during a videoconference.
Bang, Gun; Um, Gi-Mun; Chang, Eun-Young; Kim, Taeone; Hur, Nam-Ho; Kim, Jin-Woong; Lee, Soo-In, Method and apparatus for improving quality of depth image.
Haskell, Barin Geoffry; Puri, Atul; Schmidt, Robert Lewis, Scene description nodes to support improved chroma-key shape representation of coded arbitrary images and video objects.
Mackie, David J.; Tian, Dihong; Weir, Andrew P.; Buttimer, Maurice; Friel, Joseph T.; Mauchly, J. William; Chen, Wen-Hsiung, System and method for providing enhanced video processing in a network environment.
Prahlad, Anand; Schwartz, Jeremy A.; Ngo, David; Brockway, Brian; Muller, Marcus S., Systems and methods for classifying and transferring information in a storage network.
Weiser, Reginald; McGravie, Richard; Diouskine, Roman; Teboul, Jeremy, Systems and methods for providing video conferencing services via an ethernet adapter.
Rudolph, Eric; Rui, Yong; Malvar, Henrique S; He, Li Wei; Cohen, Michael F; Tashev, Ivan, Systems and methods for real-time audio-visual communication and data collaboration in a network conference environment.
Sato, Hirochika, Communication support system, information processing apparatus, control method, and storage medium that display an output image obtained by superposing a reference image over a captured image.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.