A device for processing data includes a first input port for receiving color image data from a first image sensor and a second input port for receiving depth-related image data from a second image sensor. Processing circuitry generates a depth map using the depth-related image data. At least one out
A device for processing data includes a first input port for receiving color image data from a first image sensor and a second input port for receiving depth-related image data from a second image sensor. Processing circuitry generates a depth map using the depth-related image data. At least one output port conveys the depth map and the color image data to a host computer.
대표청구항▼
1. A device for processing data, comprising: a first input port for receiving color image data, comprising a first array of color image pixels, from a first image sensor;a second input port for receiving depth-related image data from a second image sensor;processing circuitry, for generating a depth
1. A device for processing data, comprising: a first input port for receiving color image data, comprising a first array of color image pixels, from a first image sensor;a second input port for receiving depth-related image data from a second image sensor;processing circuitry, for generating a depth map using the depth-related image data, the depth map comprising a second array of depth pixels with respective depth values, and after generating the depth values, registering the depth map with the color image data by computing different, respective shifts for the depth pixels responsively to the depth values, and applying the respective shifts to the depth pixels in the second array in order to register the depth pixels with respective, associated color image pixels in the first array; andat least one output port for conveying the registered depth map and the color image data to a host computer. 2. The device according to claim 1, wherein the first and second input ports, the processing circuitry and the at least one output port are integrated circuit components, which are fabricated on a single semiconductor substrate. 3. The device according to claim 1, wherein the at least one output port comprises a single port and is coupled to convey the depth map and the color image data to the host computer as a single data stream in a multiplexed format. 4. The device according to claim 3, wherein the processing circuitry is configured to synchronize the depth map with the color image data in the single data stream. 5. The device according to claim 3, and comprising a third input port for receiving audio information, wherein the processing circuitry is configured to incorporate the audio information together with the depth map and the color image data into the single data stream for conveyance to the host computer via the single port. 6. The device according to claim 3, wherein the single port comprises a Universal Serial Bus (USB) port. 7. The device according to claim 1, wherein the processing circuitry is configured to compute the respective shifts incrementally, such that the shifts of the pixels within a line of the depth map are computed as a function of a preceding pixel shift. 8. The device according to claim 7, wherein the processing circuitry is configured to compute the respective shifts using an incremental fitting process. 9. The device according to claim 1, wherein the processing circuitry is configured to compute the respective shifts for the pixels on the fly, while receiving the depth values, without storing the entire depth map in the device. 10. The device according to claim 1, wherein the processing circuitry is configured to compress the depth map and the color image data for transmission to the host via the at least one output port. 11. The device according to claim 10, wherein the processing circuitry is configured to apply a process of lossless compression to the depth map and the color image data. 12. The device according to claim 1, wherein the depth-related image data comprise an image of a pattern that is projected onto an object, and wherein the processing circuitry is configured to generate the depth map by measuring shifts in the pattern relative to a reference image. 13. Apparatus for imaging, comprising: a first image sensor, which is configured to capture and output a color image of an object, comprising a first array of color image pixels;a second image sensor, which is configured to capture and output depth-related image data with respect to the object; anda processing device, which comprises: a first input port for receiving the color image from the first image sensor;a second input port for receiving the depth-related image data from the second image sensor;processing circuitry, for generating a depth map using the depth-related image data, the depth map comprising a second array of depth pixels with respective depth values, and after generating the depth values, registering the depth map with the color image data by computing different, respective shifts for the depth pixels responsively to the depth values, and applying the respective shifts to the depth pixels in the second array in order to register the depth pixels with respective, associated color image pixels in the first array; andat least one output port for conveying the registered depth map and the color image data to a host computer. 14. The apparatus according to claim 13, and comprising an audio sensor, wherein the processing device comprises a third input port for receiving audio information from the audio sensor, and wherein the processing circuitry is configured to convey the audio information together with the depth map and the color image data to the host computer via the at least one output port. 15. The apparatus according to claim 13, wherein the at least one output port comprises a single port and is coupled to convey the depth map and the color image data to the host computer as a single data stream in a multiplexed format. 16. The apparatus according to claim 13, and comprising an illumination subassembly, which is configured to project a pattern onto the object, wherein the depth-related image data comprise an image of the pattern on the object, and wherein the processing circuitry is configured to generate the depth map by measuring shifts in the pattern relative to a reference image. 17. A method for processing data, comprising: receiving color image data, comprising a first array of color image pixels, from a first image sensor via a first input port;receiving depth-related image data from a second image sensor via a second input port;generating a depth map using the depth-related image data, the depth map comprising a second array of depth pixels with respective depth values;after generating the depth values, registering the depth map with the color image data by computing different, respective shifts for the depth pixels responsively to the depth values, and applying the respective shifts to the depth pixels in the second array in order to register the depth pixels with respective, associated color image pixels in the first array; andconveying the registered depth map and the color image data via an output port to a host computer. 18. The method according to claim 17, wherein the depth map is generated by processing circuitry, and wherein the first and second input ports, the processing circuitry and the at least one output port are integrated circuit components, which are fabricated on a single semiconductor substrate. 19. The method according to claim 17, wherein the at least one output port comprises a single port and is coupled to convey the depth map and the color image data to the host computer as a single data stream in a multiplexed format. 20. The method according to claim 19, wherein conveying the depth map comprises synchronizing the depth map with the color image data in the single data stream. 21. The method according to claim 19, and comprising receiving audio information, and incorporating the audio information together with the depth map and the color image data into the single data stream for conveyance to the host computer via the single port. 22. The method according to claim 19, wherein the single port comprises a Universal Serial Bus (USB) port. 23. The method according to claim 17, wherein computing the respective shifts comprises processing the respective shifts incrementally, such that the shifts of the pixels within a line of the depth map are computed as a function of a preceding pixel shift. 24. The method according to claim 23, wherein processing the respective shifts comprises computing the respective shifts using an incremental fitting process. 25. The method according to claim 17, wherein computing the respective shifts comprises calculating the respective shifts for the pixels on the fly, while receiving the depth values, without storing the entire depth map while calculating the respective shifts. 26. The method according to claim 17, wherein conveying the depth map and the color image data comprises compressing the depth map and the color image data for transmission to the host via the at least one output port. 27. The method according to claim 26, wherein compressing the depth map and the color image data comprises applying a process of lossless compression to the depth map and the color image data. 28. The method according to claim 17, wherein the depth-related image data comprise an image of a pattern that is projected onto an object, and wherein generating the depth map comprises measuring shifts in the pattern relative to a reference image.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (62)
Matsumoto,Yukinori; Fujimura,Kota; Sugimoto,Kazuhide; Oue,Yasuhiro; Kitamura,Toru; Ota,Osamu, 3-D model providing device.
Binns Bruce W. ; Price Rodney M. ; Palm Charles S. ; Weaver Suzanne E., Apparatus for interactive image correlation for three dimensional image production.
Brown, Daniel M.; Erbach, Peter Scott; Pezzaniti, Joseph Larry; Minamitani, Takahisa, Autostereoscopic display with rotated microlens and method of displaying multidimensional images, especially color images.
Knighton, Mark S.; Agabra, David S.; McKinley, William D.; Zheng, John Z.; Drobnis, David D.; Logan, J. Douglas; Bahhour, Basel F.; Haynie, Jill E.; Vuong, Kevin H.; Tandon, Amit; Sidney, Kent E.; Diaconescu, Peter L., Digitizer using plural capture methods to image features of 3-D objects.
Quarendon Peter,GBX, Image processing system and method for generating data representing a number of points in a three-dimensional space from a plurality of two-dimensional images of the space.
Harada Toshiaki,JPX ; Iwaki Tetsuo,JPX ; Yamada Eiji,JPX ; Okuda Tohru,JPX, Imaging apparatus having a spatial filter and image shifting mechanism controller based on an image mode.
Link Hans-Jorg,DEX, Measuring unit for determining dimensions of test pieces, preferably of hollow bodies, in particular, of bores of workpieces, and method for measuring such dimensions.
Greivenkamp ; Jr. John E. (Rochester NY) Palum Russell J. (Rochester NY) Sullivan Kevin G. (Ft. Myers FL), Method and apparatus for absolute Moire distance measurements using a grating printed on or attached to a surface.
Rushmeier, Holly E.; Bernardini, Fausto, Method and apparatus for acquiring a set of consistent image maps to represent the color of the surface of an object.
Michniewicz Mark A. ; Frazer Matthew P., Optical method and system for measuring three-dimensional surface topography of an object having a surface contour.
Berenz, John J.; McIver, George W.; Niesen, Joseph W.; Dunbridge, Barry; Shreve, Gregory A., Optimized human presence detection through elimination of background interference.
Krueger Myron W. (55 Edith Rd. Vernon CT 06066) Hinrichsen Katrin (81 Willington Oaks Storrs CT 06268) Gionfriddo Thomas S. (81 Willington Oaks Storrs CT 06268), Real time perception of and response to the actions of an unencumbered participant/user.
Bass, Leland J.; Quick, Jr., Roy F.; Shah, Ashwin V.; Wickwire, Ralph O., System for electronically displaying portions of several different images on a CRT screen through respective prioritized viewports.
Bieman Leonard H. (Farmington Hills MI) Michniewicz Mark A. (Milford MI), System for optically measuring the surface contour of a part using more fringe techniques.
Albertson, Jacob C.; Arnold, Kenneth C.; Goldman, Steven D.; Paolini, Michael A.; Sessa, Anthony J., Tracking a range of body movement based on 3D captured image streams of a user.
Venkataraman, Kartik; Gallagher, Paul; Jain, Ankit K.; Nisenzon, Semyon; Lelescu, Dan; Ciurea, Florian; Molina, Gabriel, Autofocus system for a conventional camera that uses depth information from an array camera.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions captured by camera arrays.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions focused on an image sensor by a lens stack array.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions focused on an image sensor by a lens stack array.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images using camera array incorperating Bayer cameras having different fields of view.
Srikanth, Manohar; Ramamoorthi, Ravi; Venkataraman, Kartik; Chatterjee, Priyam, System and methods for depth regularization and semiautomatic interactive matting using RGB-D images.
Nayar, Shree; Venkataraman, Kartik; Pain, Bedabrata; Lelescu, Dan, Systems and methods for controlling aliasing in images captured by an array camera for use in super resolution processing using pixel apertures.
Lelescu, Dan; Venkataraman, Kartik, Systems and methods for controlling aliasing in images captured by an array camera for use in super-resolution processing.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan; Venkataraman, Kartik; Molina, Gabriel, Systems and methods for detecting defective camera arrays and optic arrays.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints.
Venkataraman, Kartik; Lelescu, Dan; Molina, Gabriel, Systems and methods for generating compressed light field representation data using captured light fields, array geometry, and parallax information.
Venkataraman, Kartik; Lelescu, Dan; Molina, Gabriel, Systems and methods for generating compressed light field representation data using captured light fields, array geometry, and parallax information.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using a camera arrays incorporating monochrome and color cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using a camera arrays incorporating monochrome and color cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using images captured by camera arrays incorporating cameras having different fields of view.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using light focused on an image sensor by a lens element array.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan, Systems and methods for manufacturing camera modules using active alignment of lens stack arrays and sensors.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan, Systems and methods for manufacturing camera modules using active alignment of lens stack arrays and sensors.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for measuring depth using an array of independently controllable cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for measuring depth using images captured by a camera array including cameras surrounding a central camera.
Venkataraman, Kartik; Huang, Yusong; Jain, Ankit K.; Chatterjee, Priyam, Systems and methods for performing high speed video capture and depth estimation using array cameras.
Lelescu, Dan; Duong, Thang, Systems and methods for synthesizing high resolution images using image deconvolution based on motion and depth information.
Lelescu, Dan; Molina, Gabriel; Venkataraman, Kartik, Systems and methods for synthesizing high resolution images using images captured by an array of independently controllable imagers.
Venkataraman, Kartik; Nisenzon, Semyon; Chatterjee, Priyam; Molina, Gabriel, Systems and methods for synthesizing images from image data captured by an array camera using restricted depth of field depth maps in which depth estimation precision varies.
Venkataraman, Kartik; Nisenzon, Semyon; Chatterjee, Priyam; Molina, Gabriel, Systems and methods for synthesizing images from image data captured by an array camera using restricted depth of field depth maps in which depth estimation precision varies.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.