Moving object super-resolution systems and methods
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06K-009/32
G06T-007/40
출원번호
US-0901459
(2010-10-08)
등록번호
US-8995793
(2015-03-31)
발명자
/ 주소
Laflen, John Brandon
Brooksby, Glen William
Greco, Christopher
출원인 / 주소
Lockheed Martin Corporation
대리인 / 주소
McDermott Will & Emery LLP
인용정보
피인용 횟수 :
3인용 특허 :
8
초록▼
In some approaches, super-resolution of static and moving objects can be performed. Results of moving object super-resolution may be improved by means of performing image co-registration. The quality of images of moving objects in an automated form may be improved. A sequence of images may be proces
In some approaches, super-resolution of static and moving objects can be performed. Results of moving object super-resolution may be improved by means of performing image co-registration. The quality of images of moving objects in an automated form may be improved. A sequence of images may be processed wherein objects can be detected and tracked in succeeding frames. A small region around a tracked object may be extracted in each frame. These regions may be co-registered to each other using frequency domain techniques. A set of co-registered images may be used to perform super-resolution of the tracked object. Also described are image processing systems and articles of manufacture having a machine readable storage medium and executable program instructions.
대표청구항▼
1. A method of performing moving object super-resolution on a plurality of images including a moving object, the method comprising: acquiring a target at an initial frame;tracking the target through subsequent N−1 frames, wherein tracking the target through subsequent N−1 frames includes (i) proceed
1. A method of performing moving object super-resolution on a plurality of images including a moving object, the method comprising: acquiring a target at an initial frame;tracking the target through subsequent N−1 frames, wherein tracking the target through subsequent N−1 frames includes (i) proceeding across the N image frames in pairs of adjacent frames, (ii) for each pair of adjacent frames, performing pair-wise 2D similarity registration, and (iii) aggregating pair-wise motion estimates across N frames, to estimate the motion between each frame and the initial frame;calculating sub-pixel motion estimates of the target across all N frames, wherein the sub-pixel motion estimates indicate how a given pixel in each image frame maps back onto the initial image frame;extracting an image from each of the N frames, wherein each extracted image is roughly centered on the target and is large enough to completely surround the target;computing the Fourier spectrum for the extracted images;calculating a magnitude for spectra resulting from computing the Fourier spectrum, based on the equation: m(u,v)=sqrt(Re(s(u,v))2+Im(s(u,v))2), wherein s(u,v) is a spectral point and m(u,v) is a magnitude for the spectral point;transforming the calculated magnitude for the spectra to log-polar images based on the equation: LP(logr,t)=m(U(logr,t),V(logr,t)), wherein U(logr,t)=R*exp(logr)*cos(t*dt), V(logr,t)=R*exp(logr)*sin(t*dt), with a radial operating point R, angular delta dt, wherein logr and t are integral indices into a corresponding LP matrix, and wherein if a coordinate (U,V) does not coincide with a known point in m(u,v), then (U,V) is calculated by interpolation; andperforming super-resolution using (i) the extracted images following the transforming, and (ii) the sub-pixel motion estimates, to produce a super-resolved image. 2. The method of claim 1, wherein performing 2D registration further comprises using sub-pixel translation estimation. 3. The method of claim 2, further comprising performing correction for scale and rotation changes between the two images. 4. The method of claim 1, further comprising windowing and coarsely registering the two images. 5. The method of claim 1, wherein computing the Fourier spectrum comprises using a fast Fourier transform (FFT). 6. The method of claim 1, further comprising performing sub-pixel translation estimation between the two log-polar images. 7. The method of claim 6, further comprising estimating the rotation and scale change between the two images. 8. The method of claim 7, further comprising transforming one of the original two images to remove rotation and scale change. 9. The method of claim 8, wherein transforming one of the original two images comprises interpolation. 10. The method of claim 1, wherein windowing the two images comprises using a Hamming window. 11. The method of claim 1, wherein windowing the two images comprises using a Hanning window. 12. The method of claim 1, wherein windowing the two images comprises using a Gaussian window. 13. The method of claim 1, further comprising computing the Fourier Phase Correlation between the two images. 14. The method of claim 13, further comprising estimating sub-pixel translation from the Fourier Phase Correlation. 15. The method of claim 2, further comprising estimating a rigid translation between a probe image and reference images. 16. An article of manufacture comprising: a non-transitory machine readable storage medium; andexecutable program instructions embodied in the machine readable storage medium that when executed by a processor of a programmable computing device configures the programmable computing device to control an image processing system receiving a plurality of images, to perform functions for image resolution, including functions to:acquire a target at an initial frame;track a target through subsequent N−1 frames, wherein tracking the target through subsequent N−1 frames includes (i) proceeding across the N image frames in pairs of adjacent frames, (ii) for each pair of adjacent frames, performing pair-wise 2D similarity registration, and (iii) aggregating pair-wise motion estimates across N frames, to estimate the motion between each frame and the initial frame;calculate sub-pixel motion estimates of the target across all N frames, wherein the sub-pixel motion estimates indicate how a given pixel in each image frame maps back onto the initial image frame;extract an image from each of the N frames, wherein each extracted image is roughly centered on the target and is large enough to completely surround the target;compute the Fourier spectrum for the extracted images;calculated a magnitude for spectra resulting from computing the Fourier spectrum, based on the equation: m(u,v)=sqrt(Re(s(u,v))2+Im(s(u,v))2), wherein s(u,v) is a spectral point and m(u,v) is a magnitude for the spectral point; andtransform the calculated magnitude for the spectra to log-polar images based on the equation: LP(logr,t)=m(U(logr,t),V(logr,t)), wherein U(logr,t)=R*exp(logr)*cos(t*dt), V(logr,t)=R*exp(logr)*sin(t*dt), with a radial operating point R, angular delta dt, wherein logr and t are integral indices into a corresponding LP matrix, and wherein if a coordinate (U,V) does not coincide with a known point in m(u,v), then (U,V) is calculated by interpolation; andperform super-resolution using (i) the images extracted from the N frames following the transforming, and (ii) the sub-pixel motion estimates, to produce a super-resolved image. 17. The article of manufacture of claim 16, wherein performing 2D registration further comprises using sub-pixel translation estimation. 18. The article of manufacture of claim 17, further comprising instructions for performing estimation and correction for scale and rotation changes between the two images. 19. The article of manufacture of claim 16, further comprising instructions for windowing and coarsely registering the two images. 20. The article of manufacture of claim 16, wherein computing the Fourier spectrum comprises using a fast Fourier transform (FFT). 21. The article of manufacture of claim 16, further comprising instructions for performing sub-pixel translation estimation between the two log-polar images. 22. The article of manufacture of claim 21, further comprising instructions for estimating the rotation and scale change between the two images. 23. The article of manufacture of claim 22, further comprising instructions for transforming one of the original two images to remove rotation and scale change. 24. The article of manufacture of claim 23, wherein transforming one of the original two images comprises interpolation. 25. An image processing system configured to receive a plurality of input images and produce an image with higher resolution than the input images, the system comprising: a memory; anda processor connected to the memory and configured to: track a target through subsequent N−1 frames, wherein tracking the target through subsequent N−1 frames includes (i) proceeding across the N image frames in pairs of adjacent frames, (ii) for each pair of adjacent frames, performing pair-wise 2D similarity registration, and (iii) aggregating pair-wise motion estimates across N frames, to estimate the motion between each frame and the initial frame;calculate sub-pixel motion estimates of the target across all N frames, wherein the sub-pixel motion estimates indicate how a given pixel in each image frame maps back onto the initial image frame;extract an image from each of the N frames, wherein each extracted image is roughly centered on the target and is large enough to completely surround the target;compute the Fourier spectrum for the extracted images, andcalculate a magnitude for spectra resulting from computing the Fourier spectrum, based on the equation: m(u,v)=sqrt(Re(s(u,v))2+Im(s(u,v))2), wherein s(u,v) is a spectral point and m(u,v) is a magnitude for the spectral point;transform the calculated magnitude for the spectra to log-polar images based on the equation: LP(logr,t)=m(U(logr,t),V(logr,t)), wherein U(logr,t)=R*exp(logr)*cos(t*dt), V(logr,t)=R*exp(logr)*sin(t*dt), with a radial operating point R, angular delta dt, wherein logr and t are integral indices into a corresponding LP matrix, and wherein if a coordinate (U,V) does not coincide with a known point in m(u,v), then (U,V) is calculated by interpolation; andperform super-resolution using (i) the images extracted from the N frames following the transforming, and (ii) the sub-pixel motion estimates, to produce a super-resolved image. 26. The image processing system of claim 25, wherein performing 2D registration further comprises using sub-pixel translation estimation.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (8)
Gribble, Andrew Paul; Gordon-Ingram, Iain Robert, Absolute encoder setup indication.
Riley, Ronald Alan; Bakir, Tariq; Peter, Adrian M.; Akbari, Morteza, Structured smoothing for superresolution of multispectral imagery based on registered panchromatic image.
Gupta, Akash; Vatnani, Rinku; Modi, Krupal Chandresh; Ladak, Akbar Abdulmalik, System and method for mapping object coordinates from a video to real world coordinates using perspective transformation.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.