System and method for managing mixed fleet worksites using video and audio analytics
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G07C-005/00
G06Q-010/06
G07C-005/08
출원번호
US-0676243
(2015-04-01)
등록번호
US-9685009
(2017-06-20)
발명자
/ 주소
Sprock, Christopher
Baumann, Ryan
Talmaki, Sanat
Halepatali, Praveen
출원인 / 주소
Caterpillar Inc.
대리인 / 주소
Baker Hostetler
인용정보
피인용 횟수 :
1인용 특허 :
3
초록▼
Systems and methods for managing and optimizing mixed fleet worksite operations based on video and or audio data are disclosed. One method includes receiving one or more models relating to a fleet of machines at the worksite, wherein the fleet of machines comprises an in-network machine and an out-o
Systems and methods for managing and optimizing mixed fleet worksite operations based on video and or audio data are disclosed. One method includes receiving one or more models relating to a fleet of machines at the worksite, wherein the fleet of machines comprises an in-network machine and an out-of-network machine, receiving first sensor data associated with the out-of-network machine at the worksite, receiving second sensor data associated with the in-network machine at the worksite, determining a machine state of each of the in-network machine and the out-of-network machine based at least on the first sensor data and the second sensor data, comparing the determined machine states to a modeled machine state represented by the received one or more models to classify site operations and/or detect an irregularity in site operations or an inefficiency in site operations, and generating a response based at least on the detected irregularity or inefficiency.
대표청구항▼
1. A method for managing mixed-fleet worksites, comprising: receiving, by one or more processors of a central station, one or more models relating to a worksite or a fleet of machines at the worksite, the fleet of machines including an in-network machine configured to communicate with the central st
1. A method for managing mixed-fleet worksites, comprising: receiving, by one or more processors of a central station, one or more models relating to a worksite or a fleet of machines at the worksite, the fleet of machines including an in-network machine configured to communicate with the central station, and an out-of-network machine that is not configured to communicate with the central station;receiving, by the one or more processors of the central station, first sensor data associated with the out-of-network machine at the worksite, the first sensor data comprising one or more of image data and audio data;receiving, by the one or more processors of the central station, second sensor data associated with the in-network machine at the worksite;determining, by the one or more processors, a machine state of each of the in-network machine and the out-of-network machine, the machine state of the out-of-network machine based at least on a comparison of a feature of an object detected in the image data of the first sensor data to a signature that represents a known or learned machine state, the machine state of the in-network machine based at least on the second sensor data, the machine state one of a full load, an empty load, a payload material type, a payload to air ratio, a payload placement, a payload compaction, a payload water content, a material amount moved, a drop placement, an excavator position, an idle state, a swing state, a dump position, a deformation of machine, an operator characteristic and a ground crew location relative to machine;comparing the determined machine states to a modeled machine state represented by the received one or more models to detect a non-failure irregularity in site operations or an inefficiency in site operations;generating a response based at least on the detected irregularity or inefficiency, the response including a warning or a remote reconfiguration of operational parameters of the in-network machine; andtransmitting the response to an operator, a display at the worksite, or to the in-network machine,wherein the one or more models include a machine model, a payload model or a worksite modelwherein the worksite includes a mining site or a construction site,wherein the out-of-network machine is an out-of-network digging machine, an out-of-network loading machine or an out-of-network hauling machine,wherein the in-network machine is an in-network digging machine, an in-network loading machine or an in-network hauling machine,wherein the out-of-network hauling machine and in-network hauling machine are each configured to carry excavated materials between different locations at the worksite. 2. A computer program product comprising a non-transitory storage medium having a computer readable program code embodied therein, the computer readable program code adapted to be executed to implement a method for managing a mixed-fleet worksite, the method comprising: receiving, by the one or more processors, one or more models relating to a worksite or a fleet of machines at the worksite, the fleet of machines including an in-network machine configured to communicate with a central station and an out-of-network machine that is not configured to communicate with the central station;receiving, by the one or more processors, first sensor data associated with the out-of-network machine at the worksite, the first sensor data comprising one or more of image data and audio data;receiving, by the one or more processors, second sensor data associated with the in-network machine at the worksite;determining, by the one or more processors, a machine state of each of the in-network machine and the out-of-network machine, the machine state of the out-of-network machine based at least on a comparison of a feature of an object detected in the image data of the first sensor data to a signature that represents a known or learned machine state, the machine state of the in-network machine based at least on the second sensor data, the machine state one of a full load, an empty load, a payload material type, a payload to air ratio, a payload placement, a payload compaction, a payload water content, a material amount moved, a drop placement, an excavator position, an idle state, a swing state, a dump position, a deformation of machine, an operator characteristic and a ground crew location relative to machine;comparing the determined machine states to a modeled machine state represented by the received one or more models to detect a non-failure irregularity in site operations or an inefficiency in site operations;generating a response based at least on the detected irregularity or inefficiency, the response including a warning or a remote reconfiguration of operational parameters of the in-network machine; andtransmitting the response to an operator, a display at the worksite, or to the in-network machine,wherein the one or more models include a machine model, a payload model or a worksite modelwherein the worksite includes a mining site or a construction site,wherein the out-of-network machine is an out-of-network digging machine, an out-of-network loading machine or an out-of-network hauling machine,wherein the in-network machine is an in-network digging machine, an in-network loading machine or an in-network hauling machine,wherein the out-of-network hauling machine and in-network hauling machine are each configured to carry excavated materials between different locations on the worksite. 3. The method of claim 1, wherein the one or more models comprises a worksite model and the method further comprises updating the worksite model based at least on the generated response. 4. The method of claim 1, wherein the first sensor data is received via an onsite sensor and the second sensor data is received via an offsite sensor. 5. The method of claim 1, wherein the first sensor data is received via an offsite sensor and the second sensor data is received via an onsite sensor. 6. The method of claim 1, wherein one or more of the determined machine state and the modeled machine state comprise a characteristic of a material load. 7. The method of claim 1, wherein the determining the machine state comprises generating a current signature representing the machine state. 8. The method of claim 1, wherein the response comprises one or more of an audible indicator and a visual indicator. 9. A system comprising: a processor configured to receive one or more models relating to a worksite or a fleet of machines at the worksite, the fleet of machines including an in-network machine configured to communicate with a central station and an out-of-network machine that is not configured to communicate with the central station;receive first sensor data associated with the out-of-network machine at the worksite, the first sensor data comprising one or more of image data and audio data;receive second sensor data associated with the in-network machine at the worksite;determine a machine state of each of the in-network machine and the out-of-network machine, the machine state of the out-of-network machine based at least on a comparison of a feature of an object detected in the image data of the first sensor data to a signature that represents a known or learned machine state, the machine state of the in-network machine based at least on the second sensor data, the machine state one of a full load, an empty load, a payload material type, a payload to air ratio, a payload placement, a payload compaction, a payload water content, a material amount moved, a drop placement, an excavator position, an idle state, a swing state, a dump position, a deformation of machine, an operator characteristic and a ground crew location relative to machine; andcompare the determined machine states to a modeled machine state represented by the received one or more models to classify one or more site operations,wherein the one or more models include a machine model, a payload model or a worksite model,wherein the worksite includes a mining site or a construction site,wherein the out-of-network machine is an out-of-network digging machine, an out-of-network loading machine or an out-of-network hauling machine,wherein the in-network machine is an in-network digging machine, an in-network loading machine or an in-network hauling machine,wherein the out-of-network hauling machine and in-network hauling machine are each configured to carry excavated materials between different locations at the worksite. 10. The system of claim 9, wherein the one or more models comprises a worksite model and the processor is further configured to update the worksite model based at least on the classified one or more site operations. 11. The system of claim 9, wherein the first sensor data is received via an onsite sensor and the second sensor data is received via an offsite sensor. 12. The system of claim 9, wherein the first sensor data is received via an offsite sensor and the second sensor data is received via an onsite sensor. 13. The system of claim 9, wherein one or more of the determined machine state and the modeled machine state comprise a characteristic of a material load. 14. The system of claim 9, wherein the determining the machine state comprises generating a current signature representing the machine state. 15. The computer readable program product of claim 2, wherein one or more of the determined machine state and the modeled machine state comprise a characteristic of a material load. 16. The computer readable program product of claim 2, wherein the first sensor data is received via an onsite sensor and the second sensor data is received via an offsite sensor. 17. The computer readable program product of claim 2, wherein the first sensor data is received via an offsite sensor and the second sensor data is received via an onsite sensor.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (3)
Lemelson Jerome H. ; Pedersen Robert D., GPS vehicle collision avoidance warning and control system and method.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.