[미국특허]
Systems and methods for categorizing motion event candidates
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06K-009/00
G08B-013/196
H04N-007/18
G06K-009/62
G06T-007/20
H04N-005/14
출원번호
US-0738034
(2015-06-12)
등록번호
US-9449229
(2016-09-20)
발명자
/ 주소
Laska, Jason N.
Hua, Wei
Chaudhry, Rizwan Ahmed
Varadharajan, Srivatsan
Heitz, III, George Alban
출원인 / 주소
GOOGLE INC.
대리인 / 주소
Morgan, Lewis & Bockius LLP
인용정보
피인용 횟수 :
16인용 특허 :
56
초록▼
The various embodiments described herein include methods, devices, and systems for categorizing motion event candidates. In one aspect, a method includes receiving and processing video frames that include a motion event candidate. The processing includes: (a) obtaining background factors correspondi
The various embodiments described herein include methods, devices, and systems for categorizing motion event candidates. In one aspect, a method includes receiving and processing video frames that include a motion event candidate. The processing includes: (a) obtaining background factors corresponding to a background in at least a subset of the video frames; (b) utilizing the background factors to identify one or more motion entities; (c) for each motion entity, obtaining one or more representative motion vectors based on a motion track of the respective motion entity; (d) identifying one or more features in at least a subset of the video frames; and (e) aggregating the background factors, the representative motion vectors, and the features to generate motion features. The method further includes sending the motion features to an event categorizer, where the event categorizer assigns a motion event category to the motion event candidate based on the received motion features.
대표청구항▼
1. A method comprising: at a computing system having one or more processors and memory: receiving a plurality of video frames, the plurality of video frames including a motion event candidate;processing the plurality of video frames, the processing comprising: obtaining one or more background factor
1. A method comprising: at a computing system having one or more processors and memory: receiving a plurality of video frames, the plurality of video frames including a motion event candidate;processing the plurality of video frames, the processing comprising: obtaining one or more background factors corresponding to a background in at least a subset of the plurality of video frames;utilizing the obtained background factors to identify one or more motion entities in at least a subset of the plurality of video frames;for each identified motion entity, obtaining one or more representative motion vectors based on a motion track of the respective motion entity;identifying one or more scene features in at least a subset of the plurality of video frames; andaggregating the obtained background factors, the obtained representative motion vectors, and the identified scene features to generate a plurality of motion features; andsending the plurality of motion features to an event categorizer;wherein the event categorizer assigns a motion event category to the motion event candidate based on the received motion features; andwherein the motion event category assigned to the motion event candidate is selected from a group consisting of: one or more known event types;one or more unknown event types; anda non-event type. 2. The method of claim 1, further comprising: performing object recognition on each identified motion entity; andclassifying each of at least a subset of the one or more motion entities in accordance with the performed object recognition; andwherein the motion event category is further based on the classified objects. 3. The method of claim 1, wherein the plurality of video frames correspond to a scene; the method further comprises obtaining distance information for the scene;wherein the aggregating includes aggregating the obtained distance information. 4. The method of claim 1, further comprising: training the event categorizer, the training comprising: obtaining a plurality of video clips, each video clip in the plurality of video clips including a respective motion event candidate;designating a motion event category for each respective motion event candidate;assigning, via the event categorizer, a motion event category to each respective motion event candidate; andadjusting the event categorizer based on differences between the assigned motion event categories and the designated motion event categories. 5. The method of claim 1, wherein the plurality of video frames are associated with a first user; the method further comprises obtaining user information corresponding to the first user; andwherein the motion event category is further based on the obtained user information. 6. The method of claim 5, wherein the user information comprises user feedback corresponding to one or more prior motion event candidates. 7. The method of claim 1, wherein the plurality of video frames correspond to a scene; the method further comprises obtaining environmental information corresponding to the scene; andwherein the motion event category is further based on the obtained environmental information. 8. The method of claim 7, wherein the environmental information comprises information regarding whether the scene is within a structure. 9. The method of claim 1, wherein the plurality of video frames correspond to one or more cameras; the method further comprises obtaining camera information corresponding to the one or more cameras; andwherein the motion event category is further based on the obtained camera information. 10. The method of claim 9, wherein the camera information comprises information regarding a relationship between each camera in the one or more cameras and the ground. 11. The method of claim 9, wherein the camera information comprises camera mode information corresponding to each camera in the one or more cameras. 12. The method of claim 11, wherein the camera mode information comprises information regarding whether a respective camera is in a low-light mode. 13. The method of claim 1, further comprising: receiving a second plurality of video frames, the second plurality of video frames including a second motion event candidate;processing the second plurality of video frames to generate a second plurality of motion features; andsending the second plurality of motion features to the event categorizer, wherein the event categorizer assigns a second motion event category to the second motion event candidate based on the second plurality of motion features. 14. The method of claim 13, wherein processing the second plurality of video frames comprises utilizing information corresponding to the processing of the plurality of video frames. 15. The method of claim 13, further comprising: prior to receiving the second plurality of video frames, creating a log entry corresponding to the motion event candidate; andupdating the log entry based on the second motion event category. 16. The method of claim 13, further comprising utilizing both the plurality of motion features and the second plurality of motion features to assign a motion event category to a third motion event candidate, the third motion event candidate corresponding to a combination of the first motion event candidate and the second motion event candidate. 17. The method of claim 1, further comprising generating a confidence score for the motion event candidate. 18. A server system comprising: one or more processors; andmemory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:receiving a plurality of video frames, the plurality of video frames including a motion event candidate;processing the plurality of video frames, the processing comprising: obtaining one or more background factors corresponding to a background in at least a subset of the plurality of video frames;utilizing the obtained background factors to identify one or more motion entities in at least a subset of the plurality of video frames;for each identified motion entity, obtaining one or more representative motion vectors based on a motion track of the respective motion entity;identifying one or more scene features in at least a subset of the plurality of video frames; andaggregating the obtained background factors, the obtained representative motion vectors, and the identified scene features to generate a plurality of motion features; andsending the plurality of motion features to an event categorizer;wherein the event categorizer assigns a motion event category to the motion event candidate based on the received motion features; andwherein the motion event category assigned to the motion event candidate is selected from a group consisting of: one or more known event types;one or more unknown event types; anda non-event type. 19. The server system of claim 18, wherein the plurality of video frames correspond to a scene; wherein the one or more programs further include instructions for obtaining distance information for the scene; andwherein the aggregating includes aggregating the obtained distance information. 20. The server system of claim 18, wherein the plurality of video frames are associated with a first user; wherein the one or more programs further include instructions for obtaining user information corresponding to the first user; andwherein the motion event category is further based on the obtained user information. 21. The server system of claim 18, wherein the plurality of video frames correspond to a scene; wherein the one or more programs further include instructions for obtaining environmental information corresponding to the scene; andwherein the motion event category is further based on the obtained environmental information. 22. The server system of claim 18, wherein the plurality of video frames correspond to one or more cameras; wherein the one or more programs further include instructions for obtaining camera information corresponding to the one or more cameras; andwherein the motion event category is further based on the obtained camera information. 23. The server system of claim 18, wherein the one or more programs further include instructions for: receiving a second plurality of video frames, the second plurality of video frames including a second motion event candidate;processing the second plurality of video frames to generate a second plurality of motion features; andsending the second plurality of motion features to the event categorizer, wherein the event categorizer assigns a second motion event category to the second motion event candidate based on the second plurality of motion features. 24. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing system, cause the system to: receive a plurality of video frames, the plurality of video frames including a motion event candidate;process the plurality of video frames, the processing comprising: obtaining one or more background factors corresponding to a background in at least a subset of the plurality of video frames;utilizing the obtained background factors to identify one or more motion entities in at least a subset of the plurality of video frames;for each identified motion entity, obtaining one or more representative motion vectors based on a motion track of the respective motion entity;identifying one or more scene features in at least a subset of the plurality of video frames; andaggregating the obtained background factors, the obtained representative motion vectors, and the identified scene features to generate a plurality of motion features; andsend the plurality of motion features to an event categorizer;wherein the event categorizer assigns a motion event category to the motion event candidate based on the received motion features; andwherein the motion event category assigned to the motion event candidate is selected from a group consisting of: one or more known event types;one or more unknown event types; anda non-event type. 25. The non-transitory computer-readable storage medium of claim 24, wherein the one or more programs further comprise instructions, which when executed by the computing system, cause the system to: perform object recognition on each identified motion entity; andclassify each of at least a subset of the one or more motion entities in accordance with the performed object recognition; andwherein the motion event category is further based on the classified objects. 26. The non-transitory computer-readable storage medium of claim 24, wherein the plurality of video frames correspond to a scene; wherein the one or more programs further comprise instructions, which when executed by the computing system, cause the system to obtain distance information for the scene; andwherein the aggregating includes aggregating the obtained distance information. 27. The non-transitory computer-readable storage medium of claim 24, wherein the plurality of video frames are associated with a first user; wherein the one or more programs further comprise instructions, which when executed by the computing system, cause the system to obtain user information corresponding to the first user; andwherein the motion event category is further based on the obtained user information. 28. The non-transitory computer-readable storage medium of claim 24, wherein the plurality of video frames correspond to a scene; wherein the one or more programs further comprise instructions, which when executed by the computing system, cause the system to obtain environmental information corresponding to the scene; andwherein the motion event category is further based on the obtained environmental information. 29. The non-transitory computer-readable storage medium of claim 24, wherein the plurality of video frames correspond to one or more cameras; wherein the one or more programs further comprise instructions, which when executed by the computing system, cause the system to obtain camera information corresponding to the one or more cameras; andwherein the motion event category is further based on the obtained camera information. 30. The non-transitory computer-readable storage medium of claim 24, wherein the one or more programs further comprise instructions, which when executed by the computing system, cause the system to: receive a second plurality of video frames, the second plurality of video frames including a second motion event candidate;process the second plurality of video frames to generate a second plurality of motion features; andsend the second plurality of motion features to the event categorizer, wherein the event categorizer assigns a second motion event category to the second motion event candidate based on the second plurality of motion features. 31. The non-transitory computer-readable storage medium of claim 24, wherein the one or more programs further comprise instructions, which when executed by the computing system, cause the system to: perform object recognition on each identified motion entity; andclassify each of at least a subset of the one or more motion entities in accordance with the performed object recognition; andwherein the motion event category is further based on the classified objects.
Laska, Jason N.; Nelson, Gregory R.; Duffy, Greg; Mitsuji, Hiro; Hill, Cameron; Davidsson, Martin; Montalbo, Michael D.; Wan, Tung Yuen, Method and system for cluster-based video monitoring and event categorization.
Sharma, Rajeev; Mummareddy, Satish; Hershey, Jeff; Jung, Namsoon, Method and system for segmenting people in a physical space based on automatic behavior analysis.
Schonfeld,Dan; Hariharakrishnan,Karthik; Raffy,Philippe; Yassa,Fathy, Occlusion/disocclusion detection using K-means clustering near object boundary with comparison of average motion of clusters to object and background motions.
Borzycki, Andrew; Deva, Mallikharjuna Reddy; Gajendar, Uday Nandigam; Roychoudhry, Anil, Single sign-on access in an orchestration framework for connected devices.
Lane, Corey A.; Buck, Heidi L.; Li, Joshua S.; Bagnall, Bryan D.; Stastny, John C.; Hallenborg, Eric C., System for tracking maritime domain targets from full motion video.
Watts, Tim J.; Offerdahl, Alex; Atwood, Joe; Driscoll, Jim; Loar, Steve; Stabnow, Jeff, Systems, computer-implemented methods, and computer medium to determine premiums and indemnities for supplemental crop insurance.
Wilson Charles Park ; Pedersen ; Jr. Chris Harvey ; Auyeung Alex Kamlun ; MacCormack David Ross, Video data capture and formatting in intelligent video information management system.
Laska, Jason N.; Nelson, Greg R.; Duffy, Greg; Mitsuji, Hiro; Neal, Lawrence W.; Hill, Cameron, Method and system for generating a smart time-lapse video clip.
Laska, Jason N.; Nelson, Gregory R.; Duffy, Greg; Mitsuji, Hiro; Hill, Cameron; Davidsson, Martin; Montalbo, Michael D.; Wan, Tung Yuen, Method and system for non-causal zone search in video monitoring.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.