IPC분류정보
국가/구분 |
United States(US) Patent
공개
|
국제특허분류(IPC7판) |
|
출원번호 |
17895891
(2022-08-25)
|
공개번호 |
20230064401
(2023-03-02)
|
우선권정보 |
EP-21193138.1 (2021-08-25) |
발명자
/ 주소 |
- HEINZLE, Lukas
- GOHL, Pascal
- GERSTER, Andreas
- STRUPLER, Pascal
|
출원인 / 주소 |
- HEXAGON GEOSYSTEMS SERVICES AG
|
인용정보 |
피인용 횟수 :
0 인용 특허 :
0 |
초록
▼
A system for 3D surveying of an environment by an unmanned ground vehicle (UGV) and an unmanned aerial vehicle (UAV) has two lidar devices. A reference unit has a first and a second marker in a spatially fixed arrangement. An automatic detection of the first marker is carried out for a coordinative
A system for 3D surveying of an environment by an unmanned ground vehicle (UGV) and an unmanned aerial vehicle (UAV) has two lidar devices. A reference unit has a first and a second marker in a spatially fixed arrangement. An automatic detection of the first marker is carried out for a coordinative measurement by the first lidar device to determine relative position data for providing relative position information of the first marker with respect to the first lidar device. The relative position data and spatial 3D information is used for an automatic detection and a coordinative measurement of the second marker by the second lidar device. The coordinative measurements are used for a referencing of lidar data of the UGV lidar device and lidar data of the UAV lidar device with respect to a common coordinate system.
대표청구항
▼
1. A system for providing 3D surveying of an environment, wherein the system comprises a first and a second lidar device, wherein: one of the first and the second lidar device, is an unmanned ground vehicle (UGV) lidar device configured to be mounted on an unmanned ground vehicle and configured to g
1. A system for providing 3D surveying of an environment, wherein the system comprises a first and a second lidar device, wherein: one of the first and the second lidar device, is an unmanned ground vehicle (UGV) lidar device configured to be mounted on an unmanned ground vehicle and configured to generate UGV lidar data to provide a coordinative scan of the environment relative to the UGV lidar device, the other of the first and the second lidar device is an unmanned aerial vehicle (UAV) lidar device configured to be mounted on an unmanned aerial vehicle and configured to generate UAV lidar data to provide a coordinative scan of the environment relative to the UAV lidar device, and the system is configured to provide a referencing of the UGV lidar data and the UAV lidar data with respect to a common coordinate system for determining a 3D survey point cloud of the environment, wherein: the system comprises a reference unit comprising a first and a second marker, wherein the first and the second marker are in a spatially fixed arrangement with respect to each other and each of the first and the second marker is configured as target for a coordinative measurement of the respective marker by a lidar device, wherein the system is configured: to carry out an automatic detection of the first marker and to carry out a coordinative measurement of the first marker by the first lidar device to determine relative position data providing relative position information of the first marker with respect to the first lidar device,to take into account the relative position data and a spatial 3D information on the spatially fixed arrangement of the first and the second marker with respect to each other to carry out an automatic detection of the second marker and to carry out a coordinative measurement of the second marker by the second lidar device, andto take into account the coordinative measurement of the first marker and the coordinative measurement of the second marker to provide the referencing of the UGV lidar data and the UAV lidar data with respect to the common coordinate system. 2. The system according to claim 1, wherein: one of the first marker and the second marker is an UGV marker that is configured that in a nominal setup of the reference unit it is spatially arranged in such a way that the UGV lidar device can carry out a coordinative measurement of the UGV marker, wherein the coordinative measurement of the UGV marker is carried out from a sideways looking field-of-view associated with the montage of the UGV lidar device on the UGV,the other of the first and the second marker is an UAV marker that is configured that in the nominal setup of the reference unit, it is spatially arranged in such a way that the UAV lidar device can carry out a coordinative measurement of the UAV marker, wherein the coordinative measurement of the UAV marker is carried out from a downward looking field-of-view associated with the montage of the UAV lidar device on the UAV. 3. The system according to claim 1, wherein: the system is configured to access assignment data, which provide the spatial 3D information on the spatially fixed arrangement of the first and the second marker with respect to each other, and/orat least one of the first marker and the second marker comprises a visible code, optionally a barcode, or optionally a matrix barcode, which provides the spatial 3D information on the spatially fixed arrangement of the first and the second marker with respect to each other, wherein the system is configured to determine the spatial 3D information on the spatially fixed arrangement of the first and the second marker with respect to each other by using a visual pick-up device. 4. The system according to claim 1, wherein: the coordinative scan of the environment by the UGV lidar device is provided according to a UGV scan pattern that is provided locally by the UGV lidar device, wherein the UGV scan pattern has multiple scanning directions relative to the UGV lidar device,the coordinative scan of the environment by the UAV lidar device is provided according to a UAV scan pattern that is provided locally by the UAV lidar device, wherein the UAV scan pattern has multiple scanning directions relative to the UAV lidar device, andthe UGV scan pattern provides the same local angular distribution of the multiple scanning directions, the same angular point resolution of its individual scanning directions, and the same distance resolution as the UAV scan pattern. 5. The system according to claim 1, wherein: the UGV lidar device and the UAV lidar device are in each case embodied as a laser scanner, which is configured to generate lidar data by means of a rotation of a laser beam about two rotation axes, wherein: the laser scanner comprises a rotating body configured to rotate about one of the two rotation axes and to provide for a variable deflection of an outgoing and a returning part of the laser beam, thereby providing a rotation of the laser beam about the one of the two rotation axes, fast axis, the rotating body is rotated about the fast axis with at least 50 Hz, the laser beam is rotated about the other of the two rotation axes, slow axis, with at least 0.5 Hz,the laser beam is emitted as pulsed laser beam, particularly wherein the pulsed laser beam comprises 1.5 million pulses per second, providing for a point acquisition rate of the lidar data of at least 300,000 points per second, andfor the rotation of the laser beam about the two axes the field-of-view about the fast axis is 130 degrees and about the slow axis 360 degrees. 6. The system according to claim 1, wherein: the first and the second marker are arranged on a common component such that the relative spatial arrangement of the first and the second marker is mechanically fixed,optionally, wherein the common component comprises an alignment indicator providing for a visual determination of an alignment of the common component with respect to an outer coordinate system or with respect to a cardinal direction to establish the nominal setup. 7. The system according to claim 1, wherein: at least one of the first and the second marker comprises a visually detectable pattern, optionally provided by areas of different reflectivity, different gray scales and/or different colors,the system is configured to determine a 3D orientation of the pattern by: determining geometric features in an intensity image of the pattern, wherein the intensity image of the pattern is acquired by a scanning of the pattern with a lidar measurement beam of the UGV lidar device or the UAV lidar device and a detection of an intensity of a returning lidar measurement beam, andcarrying out a plane fit algorithm in order to determine an orientation of a pattern plane, by analyzing an appearance of the geometric features in the intensity image of the pattern, andthe system is configured to take into account the 3D orientation of the pattern for providing the referencing of the UGV lidar data and the UAV lidar data with respect to the common coordinate system. 8. The system according to claim 7, wherein: the pattern comprises a circular feature,the system is configured to identify an image of the circular feature within the intensity image of the pattern, andthe plane fit algorithm is configured to fit an ellipse to the image of the circular feature and, based thereof, to determine the orientation of the pattern plane, particularly wherein the center of the ellipse is determined and aiming information for aiming with the lidar measurement beam to the center of the ellipse are derived, optionally, wherein the pattern comprises inner geometric features, particularly comprising rectangular features, which are enclosed by the circular feature. 9. The system according to claim 1, wherein the first and the second marker each comprise a visible indication of the directions of at least two of the three main axes which span the common coordinate system, wherein the system is configured to determine the directions of the three main axes by using the UGV lidar device and the UAV lidar device, and to take into account the directions of the three main axes for providing the referencing of the UGV lidar data and the UAV lidar data with respect to the common coordinate system. 10. The system according to claim 1, wherein the system is configured that the coordinative measurement of the first marker is carried out by the UGV lidar device and the coordinative measurement of the second marker is carried out by the UAV lidar device, wherein the automatic detection of the second marker and the coordinative measurement of the second marker by the UAV lidar device is carried out at each take-off and landing of the unmanned aerial vehicle, optionally, wherein the system is configured that the relative position data is continuously updated so that the relative position information provides continuously updated spatial information about the arrangement between the first marker and the UGV lidar device. 11. The system according to claim 1, wherein: the system comprises a further marker, in addition to the first and the second marker, which is configured to be mounted on the unmanned ground vehicle, the UAV lidar device is configured to automatically carry out a coordinative measurement of the further marker, andthe system is configured to take into account the coordinative measurement of the further marker to provide the referencing of the UGV lidar data and the UAV lidar data with respect to the common coordinate system. 12. The system according to claim 1, wherein: the system comprises a visual pick-up device configured to be arranged on the unmanned ground vehicle or the unmanned aerial vehicle, optionally, wherein the visual pick-up device is a camera or one of the UGV lidar device or the UAV lidar device,the system is configured to generate a sparse map using the visual pick-up device and to carry out a localization of the UGV lidar data or the UAV lidar data in the sparse map. 13. The system according to claim 12, wherein: the sparse map is generated by photogrammetric triangulation and the localization comprises a first referencing between the UGV lidar data and the UAV lidar data, andafter the first referencing, a second referencing between the UGV lidar data and the UAV lidar data is carried out based on point-cloud matching between the UGV lidar data and the UAV lidar data,wherein the sparse map is referenced with respect to a known digital model of the environment. 14. The system according to claim 1, wherein the system comprises a UGV simultaneous localization and mapping unit (UGV SLAM unit) and a UAV simultaneous localization and mapping unit, UAV SLAM unit, wherein: the UGV SLAM unit is configured for reception of the UGV lidar data as UGV perception data providing a representation of the surroundings of the UGV lidar device at a current position, use of the UGV perception data to generate a UGV map of an environment, and determination of a trajectory of a path that the UGV lidar device has passed within the UGV map of the environment, andthe UAV SLAM unit is configured for reception of the UAV lidar data as UAV perception data providing a representation of the surroundings of the UAV lidar device at a current position, use of the UAV perception data to generate a UAV map of an environment, and determination of a trajectory of a path that the UAV lidar device has passed within the UAV map of the environment. 15. The system according to claim 1, wherein the system is configured for carrying out a system processing, which comprises carrying out a SLAM process associated with the unmanned ground vehicle and/or the unmanned aerial vehicle, providing the referencing of the UGV lidar data and/or the UAV lidar data to the common coordinate system, and carrying out a path planning to determine a further trajectory to be followed by the unmanned ground vehicle and/or the unmanned aerial vehicle, wherein the system comprises: a UGV computing unit configured to be located on the unmanned ground vehicle and configured to carry out at least part of the system processing, a UAV computing unit configured to be located on the unmanned aerial vehicle and configured to carry out at least part of the system processing, an external computing unit configured to carry out at least part of the system processing,a communication unit configured: to provide a mutual communication between the UGV computing unit, the UAV computing unit, and the external computing unit by using a cellular communication connection, particularly 4G or 5G, andto provide a mutual communication between the UGV computing unit and the UAV computing unit by using a local communication connection, particularly WLAN or Bluetooth,a workload selection module configured to monitor an available bandwidth for the cellular communication connection and for the local communication connection to carry out a dynamic change of an assignment of at least part of the system processing to the UGV computing unit, the UAV computing unit, and the external computing unit, wherein the dynamic change of the assignment depends: on the available bandwidth for the cellular communication connection and for the local communication connection, anda prioritization rule to minimize the available processing load of the UAV computing unit before minimizing the available processing load of the UGV computing unit and to minimize the available processing load of the UGV computing unit before minimizing the available processing load of the external computing unit.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.