초록 ▼

The invention is related to methods and apparatus that use a visual sensor and dead reckoning sensors to process Simultaneous Localization and Mapping (SLAM). These techniques can be used in robot navigation. Advantageously, such visual techniques can be used to autonomously generate and update a ma

The invention is related to methods and apparatus that use a visual sensor and dead reckoning sensors to process Simultaneous Localization and Mapping (SLAM). These techniques can be used in robot navigation. Advantageously, such visual techniques can be used to autonomously generate and update a map. Unlike with laser rangefinders, the visual techniques are economically practical in a wide range of applications and can be used in relatively dynamic environments, such as environments in which people move. One embodiment further advantageously uses multiple particles to maintain multiple hypotheses with respect to localization and mapping. Further advantageously, one embodiment maintains the particles in a relatively computationally-efficient manner, thereby permitting the SLAM processes to be performed in software using relatively inexpensive microprocessor-based computer systems.

대표청구항 ▼

What is claimed is: 1. A method of computing a relative pose for autonomous localization for a mobile device, the method comprising: identifying one visual landmark of a plurality of stored visual landmarks that matches an image provided by a visual sensor coupled to the mobile device; identifying

What is claimed is: 1. A method of computing a relative pose for autonomous localization for a mobile device, the method comprising: identifying one visual landmark of a plurality of stored visual landmarks that matches an image provided by a visual sensor coupled to the mobile device; identifying corresponding features shown in both the image and the matching visual landmark; determining 2-dimensional coordinates within the image for the corresponding features of the image; retrieving 3-dimensional data consisting essentially of 3-dimensional coordinates of a plurality of the corresponding features of the matching visual landmark; computing a hypothetical device pose by: projecting the 3-dimensional coordinates of the corresponding features of the matching visual landmark onto new 2-dimensional coordinates of a hypothetical image, where the hypothetical image corresponds to an image that would be observed by the visual sensor if the device were to be re-posed according to the hypothetical device pose; generating a projection error by comparing the new 2-dimensional coordinates to the 2-dimensional coordinates for the corresponding features of the image; and solving for the hypothetical device pose that corresponds to a low projection error; and using the hypothetical device pose as the computed relative device pose. 2. The method as defined in claim 1, wherein the low projection error is lower than an initial projection error, where the initial projection error is calculated by comparing the 2-dimensional coordinates for the corresponding features of the image to the 2-dimensional coordinates that would be obtained if the device were at an origin of a landmark reference frame and were to have zero heading relative to the landmark reference frame. 3. The method as defined in claim 1, wherein the low projection error corresponds to a minimum root mean square (RMS) projection error. 4. The method as defined in claim 1, wherein the corresponding features correspond to scale-invariant feature transform (SIFT) features. 5. The method as defined in claim 1, wherein the 3-dimensional coordinates relate to displacements from a visual sensor coupled to the mobile device to the corresponding features at the time when the landmark was created. 6. The method as defined in claim 5, wherein the visual sensor corresponds to one or more cameras, and further comprising transforming the relative pose from a camera reference frame to a device reference frame. 7. The method as defined in claim 1, wherein the 3-dimensional coordinates for the corresponding features of the matching visual landmark are retrieved from a data store. 8. The method as defined in claim 1, wherein the 2-dimensional coordinates correspond to pixel locations. 9. The method as defined in claim 1, further comprising the step of acquiring the image with a single camera. 10. The method as defined in claim 1, wherein the one visual landmark that matches the image is identified using scale-invariant feature transform (SIFT) features. 11. A circuit for a mobile device that is configured to compute a relative pose for autonomous localization of the mobile device, the circuit comprising: a means for identifying one visual landmark of a plurality of stored visual landmarks that matches an image provided by a visual sensor coupled to the mobile device; a means for identifying corresponding features shown in both the image and the matching visual landmark: a means for determining 2-dimensional coordinates within the image for the corresponding features of the image; a means for retrieving 3-dimensional data consisting essentially of 3-dimensional coordinates of a plurality of the corresponding features of the matching visual landmark; a means for computing a hypothetical device pose further comprising: a means for projecting the 3-dimensional coordinates of the corresponding features of the matching visual landmark onto new 2-dimensional coordinates of a hypothetical image, where the hypothetical image corresponds to an image that would be observed by the visual sensor if the device were to be re-posed according to the hypothetical device pose; a means for generating a projection error by comparing the new 2-dimensional coordinates to the 2-dimensional coordinates for the corresponding features of the image; and a means for solving for the hypothetical device pose that corresponds to a low projection error, and a means for using the hypothetical device pose as the computed relative device pose. 12. The circuit as defined in claim 11, wherein the corresponding features correspond to scale-invariant feature transform (SIFT) features. 13. The circuit as defined in claim 11, wherein the mobile device comprises a mobile robot. 14. The circuit as defined in claim 11, wherein the circuit is configured to receive the image from a single camera. 15. A computer program embodied in a tangible medium for computing a relative pose for autonomous localization for a mobile device, the computer program comprising: a module with instructions for identifying one visual landmark of a plurality of stored visual landmarks that matches an image provided by a visual sensor coupled to the mobile device; a module with instructions for identifying corresponding features shown in both the image and the matching visual landmark; a module with instructions configured to determine 2-dimensional coordinates within the image for the corresponding features of the image; a module with instructions configured to retrieve 3-dimensional data consisting essentially of the 3-dimensional coordinates of a plurality of the corresponding features of the matching visual landmark; a module with instructions configured to compute a hypothetical device pose further comprises: instructions configured to project the 3-dimensional coordinates of the corresponding features of the matching visual landmark onto new 2-dimensional coordinates of a hypothetical image, where the hypothetical image corresponds to an image that would be observed by the visual sensor if the device were to be re-posed according to the hypothetical device pose; instructions configured to generate a projection error by comparing the new 2-dimensional coordinates to the 2-dimensional coordinates for the corresponding features of the image; and instructions configured to solve for the hypothetical device pose that corresponds to a low projection error; and a module with instructions configured to use the hypothetical device pose as the computed relative device pose. 16. The computer program as defined in claim 15, wherein the corresponding features correspond to scale-invariant feature transform (SIFT) features. 17. The computer program as defined in claim 15, wherein the computer program is configured to receive the image from a single camera. 18. A circuit in a mobile device for computing a relative pose for autonomous localization for the mobile device, the circuit comprising: a circuit configured to identifying one visual landmark of a plurality of stored visual landmarks that matches an image provided by a visual sensor coupled to the mobile device; a circuit configured to identifying corresponding features shown in both the image and the matching visual landmark; a circuit configured to determine 2-dimensional coordinates within the image for the corresponding features of the image; a circuit configured to retrieve 3-dimensional data consisting essentially of the 3-dimensional coordinates of a plurality of the corresponding features of the matching visual landmark; a circuit configured to compute a hypothetical device pose, further comprising; a circuit configured to project the 3-dimensional coordinates of the corresponding features of the matching visual landmark onto new 2-dimensional coordinates of a hypothetical image, where the hypothetical image corresponds to an image that would be observed by the visual sensor if the device were to be re-posed according to the hypothetical device pose; a circuit configured to generate a projection error by comparing the new 2-dimensional coordinates to the 2-dimensional coordinates for the corresponding features of the image; and a circuit configured to solve for the hypothetical device pose that corresponds to a low projection error; and where the circuit is configured to use the hypothetical device pose as the computed relative device pose. 19. The circuit as defined in claim 18, wherein the low projection error corresponds to a minimum root mean square (RMS) projection error. 20. The circuit as defined in claim 18, wherein the corresponding features correspond to scale-invariant transform (SIFT) features. 21. The circuit as defined in claim 18, wherein the circuit is embodied in a robot for navigation of the robot.