Frequency domain processing techniques for plenoptic images
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06T-005/10
H04N-013/02
H04N-005/232
H04N-005/225
G06T-005/50
출원번호
US-0585380
(2014-12-30)
등록번호
US-9538075
(2017-01-03)
발명자
/ 주소
Lumsdaine, Andrew
Willcock, Jeremiah
Zhou, Yuduo
Lin, Lili
출원인 / 주소
Indiana University Research and Technology Corporation
대리인 / 주소
Faegre Baker Daniels LLP
인용정보
피인용 횟수 :
0인용 특허 :
51
초록▼
One embodiment is a method of operating a plenoptic image processing system. The method includes storing plenoptic image data in a non-transitory computer readable memory of the system. The plenoptic image data includes data of a plurality of microimages. The method includes applying a frequency dom
One embodiment is a method of operating a plenoptic image processing system. The method includes storing plenoptic image data in a non-transitory computer readable memory of the system. The plenoptic image data includes data of a plurality of microimages. The method includes applying a frequency domain transform to the stored plenoptic image data to provide a frequency domain plenoptic image data structure stored in the memory, and processing the data structure using an extended slice technique to select a focal plane from the data structure. The extended slice technique comprises taking multiple slices of the frequency domain transformed plenoptic image data structure and configuring the multiple slices to provide extended slice data. The method further includes applying an inverse frequency domain transform to the extended slice data to provide inverse transform extended slice data, and providing an image output based upon the inverse transformed extended slice data.
대표청구항▼
1. A method of operating a plenoptic image processing system, the method comprising: storing plenoptic image data in a non-transitory computer readable memory of the system, the plenoptic image data comprising data of a plurality of microimages;applying a frequency domain transform to the stored ple
1. A method of operating a plenoptic image processing system, the method comprising: storing plenoptic image data in a non-transitory computer readable memory of the system, the plenoptic image data comprising data of a plurality of microimages;applying a frequency domain transform to the stored plenoptic image data to provide a frequency domain plenoptic image data structure stored in the non-transitory computer readable memory of the system;processing the frequency domain transformed plenoptic image data structure using an extended slice technique to select a focal plane from the data structure, wherein the extended slice technique comprises taking multiple slices of the frequency domain transformed plenoptic image data structure and configuring the multiple slices to provide extended slice data;applying an inverse frequency domain transform to the extended slice data to provide inverse transform extended slice data; andproviding an image output based upon the inverse transformed extended slice data. 2. The method of claim 1 further comprising selecting, prior to applying the frequency domain transform, respective center portions of at least a subset of the plurality of microimages, and forming, prior to applying the frequency domain transform, a four-dimensional array from the respective center portions of at least a subset of the plurality of microimages, the four-dimensional array comprising x and y positions of the array and x and y positions within each microimage of the array. 3. The method of claim 1 wherein the frequency domain transformed plenoptic image data structure is a multi-dimensional array the method further comprising normalizing the array to compensate for different numbers of the microimages overlapping at different points. 4. The method of claim 1 further comprising applying, prior to applying the frequency domain transform, a window function to effective reduce the influence of the outer portions of the microimages on the image output. 5. The method of claim 1 further comprising, prior to applying the frequency domain transform, padding the microimages effective to increase accuracy of an output of the frequency domain transform. 6. The method of claim 1 wherein the frequency domain transformed plenoptic image data structure is a four-dimensional array and the extended slice technique comprises taking a two-dimensional slice of the four-dimensional array. 7. A method according to claim 6 wherein the inverse frequency domain transform comprises a 2D inverse FFT. 8. The method of claim 1 wherein providing an image output based upon the inverse transformed extended slice data comprises displaying the image output on a display of the system. 9. The method of claim 1 wherein the plenoptic image data is captured by a focused plenoptic camera, wherein the focused plenoptic camera includes a main lens, a microlens array including plurality of microlenses, and a photosensor, each microlens in the microlens array having a focal distance ƒ, the microlens being positioned a distance a behind a focus of the main lens and a distance b in front of the photosensor, wherein a, b, and f together satisfy the equation: 1/a+1/b=1/ƒ. 10. The method of claim 1 wherein the image processing system is structured as one of a personal computer, a digital camera, a smartphone, and a tablet computer. 11. The method of claim 1 further comprising performing, prior to applying the frequency domain transform, the following operations: selecting respective center portion of at least a subset of the plurality of microimages;forming a four-dimensional array from at least a subset of the respective center portions, the four-dimensional array comprising x and y positions of the array and x and y positions within each microimage of the array;normalizing the array to compensate for different numbers of microimages overlapping at different points;applying a window function to reduce the effect of the outer portions of each microimage; andpadding each microimage. 12. The method of claim 1 wherein the processing comprises: applying a 4D Fast Fourier Transform (“FFT”) to the array;creating a 2D slice from the array; andapplying a 2D inverse FFT to the slice. 13. The method of claim 1 wherein the acts of applying a frequency domain transform and processing comprise: generating a 4D array of windowed portions of the microimages;applying 1D FFTs in the x and y directions;for each microimage, optionally padding the microimage, applying 1D Fast Fourier Transforms (“FFTs”) in the p and q directions, and interpolating all necessary points from a microimage to place them into the correct positions in the output 2D arrays;for each output image, applying 1D inverse FFTs in the x and y directions, dividing the other color channels of the image by the normalization channel, and outputting the image into a file. 14. An image processing system comprising: a non-transitory computer readable memory configured to store plenoptic image data;a processor structured to: apply a frequency domain transform to the stored plenoptic image data to provide a frequency domain transformed plenoptic image data structure stored in the non-transitory computer readable memory,take an extended slice of the frequency domain transformed plenoptic image data structure by taking multiple slices of the frequency domain transformed plenoptic image data structure and combining the multiple slices to provide extended slice data, andapply an inverse frequency domain transform to the extended slice data to provide output image data; andan output device configured to receive the output image data. 15. The system of claim 14 wherein the processor is configured to perform the following operations prior to applying the frequency domain transform: select respective center portion of at least a subset of the plurality of microimages;form a four-dimensional array from at least a subset of the respective center portions, the four-dimensional array comprising x and y positions of the array and x and y positions within each microimage of the array;normalize the array to compensate for different numbers of microimages overlapping at different points;apply a window function to reduce the effect of the outer portions of each microimage; andpad each microimage. 16. The system of claim 14 wherein the plenoptic image data is structured as an array and the processor is configured to: apply a four-dimensional Fast Fourier Transform (“FFT”) to the array;create a two-dimensional slice from the array; andapply a two-dimensional inverse FFT to the slice. 17. The system of claim 14 wherein the processor is configured to: generate a 4D array of windowed portions of the microimages;apply 1D FFTs in the x and y directions;for each microimage, optionally pad the microimage, apply 1D FFTs in the p and q directions, and interpolate all necessary points from a microimage and place them into the correct positions in the output 2D arrays;for each output image, apply 1D inverse FFTs in the x and y directions, divide the other color channels of the image by the normalization channel, output the image into a file. 18. The system of claim 14 wherein the output device comprises one of a non-transitory memory device configured to store the image data as a file, and a display operatively coupled with the processor and configured to display the image data. 19. An image processing system according to claim 14 wherein the plenoptic image data is captured by a focused plenoptic camera, wherein the focused plenoptic camera includes a main lens, a microlens array including plurality of microlenses, and a photosensor, each microlens in the microlens array having a focal distance ƒ, the microlens being positioned a distance a behind a focus of the main lens and a distance b in front of the photosensor, wherein a, b, and ƒ together satisfy the equation: 1/a+1/b=1/ƒ. 20. The image processing system according to claim 14, wherein the extended slice comprises a central slice and slice extensions, wherein the extended slice is formed by appending the slice extensions to respective ends of the central slice.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (51)
Georgiev Todor, 3D graphics based on images and morphing.
de Montebello Roger L. (New York NY) Globus Ronald P. (New York NY) Buck Howard S. (New York NY), Integral photography apparatus and method of forming same.
Georgiev, Todor G.; Chunev, Georgi N., Methods and apparatus for rendering output images with simulated artistic effects from focused plenoptic camera data.
Georgiev, Todor G.; Lumsdaine, Andrew, Methods, apparatus, and computer-readable storage media for depth-based rendering of focused plenoptic camera data.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.