초록 ▼

Techniques for aligning images are disclosed. The frames might have been captured by a video camera on a hand held device, as one example. Collaboratively aligning related frames of image data is taught. Collaborative alignment determines a correspondence between pixels in pairs of the frames of ima

Techniques for aligning images are disclosed. The frames might have been captured by a video camera on a hand held device, as one example. Collaboratively aligning related frames of image data is taught. Collaborative alignment determines a correspondence between pixels in pairs of the frames of image data, as well as a confidence in that correspondence. A coordinate system (or transformation) is assigned to each of the frames that is consistent with the correspondences between each of the frame pairs. The confidence in the respective correspondences may be used to provide a weighting to a correspondence when assigning the coordinate systems. Outlying frames may be discarded, and the process repeated for a robust solution.

대표청구항 ▼

1. A method, comprising: accessing a plurality of frames of images, the frames having pixels;selecting pairs of the frames;for each of the selected pairs, finding a correspondence between the pixels or regions in the frames in the pair, including estimating a set of difference transformation paramet

1. A method, comprising: accessing a plurality of frames of images, the frames having pixels;selecting pairs of the frames;for each of the selected pairs, finding a correspondence between the pixels or regions in the frames in the pair, including estimating a set of difference transformation parameters for each of the selected pairs of frames, a difference transformation parameter in the set for a given pair being the difference between a first set of unknown transformation parameters for a first member of the pair and a second set of unknown transformation parameters for a second member of the pair;associating a confidence with the difference transformation parameters in the set for each of the selected pairs of frames; andassigning a coordinate system to each of the plurality of frames that is consistent with the correspondence for each of the selected pairs of frames, including weighting the difference transformation parameters in the set for each of the selected pairs of frames based on the associated confidence. 2. The method of claim 1, wherein the assigning a coordinate system to each of the plurality of frames based on the correspondence for each of the selected pairs of frames includes: finding a set of transformation parameters for each of the plurality of frames that is consistent with the set of difference transformation parameters for each of the selected pairs of frames. 3. The method of claim 1, further comprising: after assigning the coordinate system to each of the plurality of frames, removing from the plurality of frames a frame that is an outlier; andrepeating the selecting, the finding a correspondence, the associating a confidence, and the assigning after the outlying frame has been removed from the plurality of frames. 4. The method of claim 1, wherein the assigning a coordinate system to each of the plurality of frames that is consistent with the correspondence for each of the selected pairs of frames includes: determining a solution to a set of equations in which the coordinate system for each of the plurality of frames are unknown and the correspondence for each of the frames of pairs are known. 5. The method of claim 1, further comprising: aligning the plurality of frames based on the coordinate system assigned to each of the plurality of frames; anddetecting a location of a screen in the plurality of frames after the alignment. 6. The method of claim 5, wherein the detecting a location of a screen in the plurality of frames includes: forming an image based on one of more of the plurality of frames;identifying a set of lines in the plurality of frames that are candidate for boundaries of the screen;integrating the image on each side of each line in the set of lines to form integration results;forming a set of screen hypothesis based on the integration results;scoring screens in the screen hypothesis according to a criterion; andselecting a screen from the screens in set of screen hypothesis based on the scoring. 7. An apparatus comprising: a processor configured to: access a plurality of frames of images, the frames having pixels;select pairs of the frames;for each of the selected pairs, find a correspondence between the pixels or regions in the frames in the pair and associate a confidence with the correspondence between the pixels or regions in the frames in the pair; andassign a coordinate system to each of the plurality of frames that is consistent with the correspondence and associated confidence for each of the selected pairs of frames. 8. The apparatus of claim 7, wherein the processor configured to find a correspondence between the pixels in the frames in the pair and associate a confidence with the correspondence between the pixels or regions in the frames in the pair comprises the processor configured to: estimate a set of difference transformation parameters for each of the selected pairs of frames, a difference transformation parameter in the set for a given pair being the difference between a first set of unknown transformation parameters for a first member of the pair and a second set of unknown transformation parameters for a second member of the pair, wherein the processor configured to assign a coordinate system to each of the plurality of frames based on the correspondence and associated confidence for each of the selected pairs of frames includes the processor configured to find a set of transformation parameters for each of the plurality of frames that is consistent with the set of difference transformation parameters for each of the selected pairs of frames. 9. The apparatus of claim 8, wherein the processor configured to associate a confidence with the correspondence between the pixels or regions in the frames in the pair comprises the processor being configured to: associate a confidence with the difference transformation parameters in the set for each of the selected pairs of frames, wherein the processor configured to assign a coordinate system to each of the plurality of frames include the processor configured to weight the difference transformation parameters in the set for each of the selected pairs of frames based on the associated confidence with the difference transformation parameters. 10. The apparatus of claim 7, wherein the processor is further configured to: remove, from the plurality of frames, a frame that is an outlier after the processor assigns the coordinate system to each of the plurality of frames;select new pairs of frames after the outlying frame has been removed from the plurality of frames;for each of the new selected pairs, find a correspondence between the pixels or regions in the frames in the pair; andassign a coordinate system to each of the plurality of frames that is consistent with the correspondence. 11. The apparatus of claim 7, further comprising: a video camera configured to capture the plurality of frames of images, wherein the processor is further configured to align the plurality of frames based on the coordinate system assigned to each of the plurality of frames. 12. The apparatus of claim 7, wherein the processor is configured to associate a confidence with the correspondence between the pixels in the frames in the pair. 13. The apparatus of claim 7, wherein the processor is configured to associate a confidence with the correspondence between the regions in the frames in the pair. 14. A computer-readable storage device having computer-readable instructions embodied thereon for use by a processor, the computer-readable instructions cause the processor to: access a plurality of frames of images, the frames having pixels;select pairs of the frames;for each of the selected pairs, estimate a set of difference transformation parameters for each of the selected pairs of frames; andassign a set of transformation parameters to each of the plurality of frames that is consistent with the sets of difference transformation parameters for each of the selected pairs of frames, wherein the computer-readable instructions that cause the processor to assign a set of transformation parameters to each of the plurality of frames further cause the processor to determine a least squares solution to a set of equations in which the set of transformation parameters for each of the plurality of frames are unknown and the set of difference transformation parameters for each of the frames of pairs are known. 15. The computer-readable storage device of claim 14, wherein the computer-readable instructions further cause the processor to: remove, from the plurality of frames, a frame that is an outlier after assigning the set of transformation parameters to each of the plurality of frames; andrepeat the selecting, the estimating the sets of difference transformation parameters, and the assigning after the outlying frame has been removed from the plurality of frames. 16. The computer-readable storage device of claim 14, wherein the computer-readable instructions further cause the processor to: align the plurality of frames based on the set of transformation parameters assigned to each of the plurality of frames; anddetect a location of a screen in the plurality of frames after the alignment. 17. The computer-readable storage device of claim 16, wherein the computer-readable instructions that cause the processor to detect a location of a screen in the plurality of frames further cause the processor to: form an image based on one of more of the plurality of frames;identify a set of lines in the plurality of frames that are candidate for boundaries of the screen;integrate the image on each side of each line in the set of lines to form integration results;form a set of screen candidates based on the integration results;score screens in the screen candidates according to a criterion; andselect a first of the screens in the set of screen candidates based on the scoring. 18. The computer-readable storage device of claim 14, wherein the computer-readable instructions further cause the processor to: associate a confidence in each of the difference transformation parameters in the set, wherein the instructions cause the processor to assign the set of transformation parameters to each of the plurality of frames that is consistent with the sets of difference transformation parameters and the associated confidences in each of the difference transformation parameters in the set for each of the selected pairs of frames. 19. An apparatus comprising: a processor configured to: access a plurality of frames of images, the frames having pixels and having regions;select pairs of the plurality of frames;for each of the selected pairs, find a correspondence between the pixels or the regions in the frames in the pair;assign a coordinate system to each of the plurality of frames that is consistent with the correspondence;align the plurality of frames based on the coordinate system assigned to each of the plurality of frames; anddetect a location of a screen in the plurality of frames after the alignment, wherein to detect the location of the screen the processor is configured to: a) form an image based on one of more of the plurality of frames;b) identify a set of lines in the plurality of frames that are candidates for boundaries of the screen;c) integrate the image on each side of each line in the set of lines to form integration results;d) form a set of screen hypothesis based on the integration results;e) score screens in the screen hypothesis according to a criterion; andf) select a screen from the screens in set of screen hypothesis based on the scoring. 20. An apparatus comprising: a processor configured to: access a plurality of frames of images, the frames having pixels;select pairs of the frames;for each of the selected pairs, find a correspondence between the pixels or regions in the frames in the pair;assign a coordinate system to each of the plurality of frames that is consistent with the correspondence for each of the selected pairs of frames;after the coordinate system is assigned to each of the plurality of frames, remove from the plurality of frames a frame that is an outlier;select new pairs of frames after the outlying frame has been removed from the plurality of frames;for each of the new selected pairs, find a correspondence between the pixels or regions in the frames in the pair; andassign a coordinate system to each of the plurality of frames that is consistent with the correspondence.