Methods, systems, and computer program products for creating three-dimensional video sequences
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
H04N-013/02
H04N-013/00
G06T-007/00
출원번호
US-0288209
(2011-11-03)
등록번호
US-9185388
(2015-11-10)
발명자
/ 주소
McNamer, Michael
Markas, Tassos
출원인 / 주소
3DMedia Corporation
대리인 / 주소
Olive Law Group PLLC
인용정보
피인용 횟수 :
0인용 특허 :
124
초록▼
Methods and systems for creating three-dimensional video sequences of a scene are disclosed. An example method can include receiving multiple frames of a scene. The method may include selecting a target frame from among the multiple frames; selecting a first subset of frames, N, from among the multi
Methods and systems for creating three-dimensional video sequences of a scene are disclosed. An example method can include receiving multiple frames of a scene. The method may include selecting a target frame from among the multiple frames; selecting a first subset of frames, N, from among the multiple frames that are associated with the target frame that is representative of a large stereo baseline; and analyzing the first frame subset to identify two images for forming a stereoscopic pair of frames. Further, the method includes extracting depth data of static objects in the stereoscopic pair. The method includes selecting a second subset of frames that are associated with the target frame that is representative of a smaller stereo baseline than that represented by N; and utilizing the second frame subset to calculate depth of moving objects. The method includes generating a three-dimensional video frame based on the depth data.
대표청구항▼
1. A method for creating a three-dimensional video sequence of a scene, the method comprising: utilizing a two-dimensional video sequence of a scene, the two-dimensional video sequence including a plurality of frames;selecting a target frame from among the plurality of frames;selecting a first subse
1. A method for creating a three-dimensional video sequence of a scene, the method comprising: utilizing a two-dimensional video sequence of a scene, the two-dimensional video sequence including a plurality of frames;selecting a target frame from among the plurality of frames;selecting a first subset of frames, N, from among the plurality of frames that are associated with the target frame;identifying and classifying static and moving objects utilizing the first subset of frames;extracting depth data of static objects utilizing the first subset of frames;selecting a second subset of frames, n, from among the plurality of frames that are associated with the target frame, representative of a stereo displacement that is substantially different than the displacement represented by N;utilizing the second subset of frames to calculate depth of moving objects at the time of the target frame; andcombining the depth values of static and moving objects to generate a three-dimensional video frame. 2. The method of claim 1, further comprising one or more steps of identifying suitable frames, registration, stabilization, color correction, transformation, and depth adjustment. 3. The method of claim 1, further comprising generating one or more additional frames and frame viewpoints using one of existing raster data and depth information. 4. The method of claim 1, further comprising using a micro stereo base technique for generating image representations of close and moving objects of the scene. 5. The method of claim 1, further comprising using a macro stereo base technique for generating image representations of a background and non-moving objects of the scene. 6. The method of claim 1, further comprising using an image capture device for capturing the two-dimensional video sequence. 7. The method of claim 1, further comprising using the steps of claim 1 for generating a plurality of three-dimensional video frames. 8. The method of claim 1, further comprising dividing the captured two-dimensional video sequence into segments divided by utilizing one of a scene change detection and camera pose information. 9. The method of claim 8, further comprising creating a stereoscopic video sequence from each segment. 10. The method of claim 9, further comprising equalizing the depths and other three-dimensional parameters and combining the individual stereoscopic segments to form a single three-dimensional video stream. 11. The method of claim 1, further comprising analyzing captured frames to measure object displacements between two or more frames via motion vectors and identifying and classifying static and moving objects within the current target frame. 12. The method of claim 1, further comprising: analyzing captured frames; andidentifying a position of key static objects to estimate the position of the camera. 13. The method of claim 1, further comprising measuring the motion vectors of the moving objects and estimating their relative position on the x, y space. 14. The method of claim 1, further comprising: measuring sizes of moving objects; andestimating their relative position on the z-space based on the rate of increase or decrease of their sizes. 15. The method of claim 1, further comprising estimating the depth of moving objects by identifying the depth of key points of moving objects relative to the depth of neighboring static objects with known depths and extrapolating based on temporal or structural relationships. 16. The method of claim 1, further comprising estimating the depth of moving objects by determining whether they are moving behind or in front of static objects and utilizing the depth of static objects to estimate the depth of a moving object based on the moving object's trajectory in time. 17. The method of claim 1, further comprising: segmenting the static portions of the scene into static objects with known positions in three-dimensional space and utilizing prediction techniques to estimate the depth of new information entering into the captured video sequence associated with the same or related static objects. 18. The method of claim 1, further comprising: measuring sizes of static objects; andestimating changes in camera focal length and adjusting the depth of the scene based on the relative zoom factor of the camera based on their relative rate of increase or decrease over time. 19. A system for creating a three-dimensional video sequence of a scene, the system comprising: a memory having stored therein computer program code;a computer processor that executes the computer program code;a video generator configured to: receive a two-dimensional video sequence of a scene, the two-dimensional video sequence including a plurality of frames;select a target frame from among the plurality of frames;select a first subset of frames, N, from among the plurality of frames that are associated with the target frame;extract depth data of static objects utilizing the first subset of frames;select a target frame from among the plurality of frames;select a second subset of frames, n, from among the plurality of frames that are associated with the target frame, representative of a stereo displacement that is substantially different than the displacement represented by N;utilize the second subset of frames to calculate depth of moving objects at the time of the target frame; andcombine the depth values of static and moving objects to generate a three-dimensional video frame. 20. The system of claim 19, wherein the video generator is configured to implement one or more functions of identifying suitable frames, registration, stabilization, color correction, transformation, and depth adjustment. 21. The system of claim 19, wherein the video generator is configured to generate one or more additional frames and frame viewpoints using one of existing raster data and depth information. 22. The system of claim 19, wherein the video generator is configured to use a micro stereo base technique for generating image representations of close and moving objects of the scene. 23. The system of claim 19, wherein the video generator is configured to use a macro stereo base technique for generating image representations of a background and non-moving objects of the scene. 24. The system of claim 19, comprising an image sensor and a lens configured to capture the two-dimensional video sequence. 25. The system of claim 19, wherein the video generator is configured to generate a plurality of three-dimensional video frames using the functions implemented by the video generator recited in claim 9. 26. The system of claim 25, comprising a display configured to display the plurality of three-dimensional video frames in a sequence. 27. A computer program product for creating three-dimensional video sequence, the computer program product comprising: a non-transitory computer readable storage medium having computer readable program code embodied therewith, the non-transitory computer readable program code comprising: computer readable code configured to receive a two-dimensional video sequence of a scene, the two-dimensional video sequence including a plurality of frames;computer readable code configured to select a target frame from among the plurality of frames;computer readable code configured to select a first subset of frames, N, from among the plurality of frames that are associated with the target frame;computer readable code configured to identify and classify static and moving objects using the first subset of frames;computer readable code configured to extract depth data of static objects utilizing the first subset of frames;computer readable code configured to select a second subset of frames, n, from among the plurality of frames that are associated with the target frame, representative of a stereo displacement that is substantially different than the displacement represented by N;computer readable code configured to utilize the second subset of frames to calculate depth of moving objects at the time of the target frame; andcomputer readable code configured to combine the static and moving objects to generate a three-dimensional video frame. 28. The computer program product of claim 27, wherein the computer readable program code comprises computer readable code to implement one or more steps of identifying suitable frames, registration, stabilization, color correction, transformation, and depth adjustment. 29. The computer program product of claim 27, wherein the computer readable program code comprises computer readable code to generate one or more additional frames and frame viewpoints using one of existing raster data and depth information. 30. The computer program product of claim 27, wherein the computer readable program code comprises computer readable code to use a micro stereo base technique for generating image representations of close and moving objects of the scene. 31. The computer program product of claim 27, wherein the computer readable program code comprises computer readable code to use a macro stereo base technique for generating image representations of a background and non-moving objects of the scene. 32. The computer program product of claim 27, wherein the computer readable program code comprises computer readable code to use an image capture device for capturing the two-dimensional video sequence. 33. The computer program product of claim 27, wherein the computer readable program code comprises computer readable code to generate a plurality of three-dimensional video frames. 34. The computer program product of claim 33, wherein the computer readable program code comprises computer readable code to control a display to display the plurality of three-dimensional video frames in a sequence.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (124)
Sullivan, Steve; Trombla, Alan D.; Callari, Francesco G., 2D to 3D image conversion.
Kim Man-bae,KRX ; Song Mun-sup,KRX ; Kim Do-kyoon,KRX, Apparatus and method for converting two-dimensional image sequence into three-dimensional image using conversion of motion disparity into horizontal disparity and post-processing method during genera.
Bacs ; Jr. Aron (Burke VA) Mayhew Christopher A. (Oakton VA) Fernekes Leo M. (New York NY) Buchroeder Richard A. (Tucson AZ) Rublowsky Stefan J. (Brooklyn NY), Autostereoscopic imaging apparatus and method using a parallax scanning lens aperture.
Jayavant, Rajeev; Nuechterlein, David W., Circuitry and systems for performing two-dimensional motion compensation using a three-dimensional pipeline and methods of operating the same.
Haruhiko Murata JP; Yukio Mori JP; Shuugo Yamashita JP; Akihiro Maenaka JP; Seiji Okada JP; Kanji Ihara JP, Device and method for converting two-dimensional video into three-dimensional video.
Herman ; deceased Joshua Randy ; Bergen James Russell ; Peleg Shmuel,ILX ; Paragano Vincent ; Dixon Douglas F. ; Burt Peter J. ; Sawhney Harpreet ; Gendel Gary A. ; Kumar Rakesh ; Brill Michael H., Method and apparatus for mosaic image construction.
Azarbayejani Ali (Cambridge MA) Galyean Tinsley (Cambridge MA) Pentland Alex (Cambridge MA), Method and apparatus for three-dimensional, textured models from plural video images.
Thier Uri (West Hartford CT) Thier Oren (West Hartford CT) Woodbury William (Gainesville FL), Method for controlling a 3D patch-driven special effects system.
Wong, Earl Q.; Nakamura, Makibi; Kushida, Hidenori; Triteyaprasert, Soroj, Method of and apparatus for generating a depth map utilized in autofocusing.
Nakagawa Yasuo (Chigasaki PA JPX) Nayer Shree K. (Pittsburgh PA), Method of detecting solid shape of object with autofocusing and image detection at each focus level.
Kaye,Michael C.; Best,Charles J. L., Method of hidden surface reconstruction for creating accurate three-dimensional images converted from two-dimensional images.
Choquet Bruno (Rennes FRX) Pele Danielle (Rennes FRX) Chassaing Francoise (La Chapelle des Fougeretz FRX), Method of processing and transmitting over a “MAC”type channel a sequence of pairs of sterescopic television images.
Yukinori Matsumoto JP; Hajime Terasaki JP; Kazuhide Sugimoto JP; Masazumi Katayama JP; Tsutomu Arakawa JP; Osamu Suzuki JP, Methods for creating an image for a three-dimensional display, for calculating depth information and for image processing using the depth information.
Rubbert,R체dger; Weise,Thomas; Sporbert,Peer; Imgrund,Hans; Kouzian,Dimitrij, Methods for registration of three-dimensional frames to create three-dimensional virtual models of objects.
Matsumura, Koichi; Baumberg, Adam Michael; Lyons, Alexander Ralph; Nagasawa, Kenichi; Saito, Takashi, Photographing apparatus, device and method for obtaining images to be used for creating a three-dimensional model.
Routhier, Nicholas; Thibeault, Claude; Belzile, Jean; Malouin, Daniel; Carpentier, Pierre Paul; Dallaire, Martin, Process and system for encoding and playback of stereoscopic video sequences.
Routhier, Nicholas; Thibeault, Claude; Belzile, Jean; Malouin, Daniel; Carpentier, Pierre-Paul; Dallaire, Martin, Process and system for encoding and playback of stereoscopic video sequences.
Zhang, Zhengyou; Anandan, Padmanabhan; Shum, Heung-Yeung, System and method for determining structure and motion using multiples sets of images from different projection models for object modeling.
Wetzel, Arthur W.; Gilbertson, II, John R.; Beckstead, Jeffrey A.; Feineigle, Patricia A.; Hauser, Christopher R.; Palmieri, Jr., Frank A., System for creating microscopic digital montage images.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.