System and method for automated testing of user interface software for visual responsiveness
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06F-003/0481
G06F-003/0484
G06F-009/44
G06F-011/36
G06F-008/38
G06T-007/40
출원번호
US-0249974
(2016-08-29)
등록번호
US-10114733
(2018-10-30)
발명자
/ 주소
Varghese, David
Saxena, Mohit
Sharma, Anshul
Ginetti, Arnold Jean-Marie Gustave
출원인 / 주소
Cadence Design Systems, Inc.
대리인 / 주소
Rosenberg, Klein & Lee
인용정보
피인용 횟수 :
0인용 특허 :
4
초록▼
A benchmark test system captures and records root, or input, behavior from a user input device as one or more time-displaced samples of input. The system also separately captures and records the canvas, or visual, behavior of a user interface in response to the captured input as a series of time-dis
A benchmark test system captures and records root, or input, behavior from a user input device as one or more time-displaced samples of input. The system also separately captures and records the canvas, or visual, behavior of a user interface in response to the captured input as a series of time-displaced image frames. The image frames are analyzed for visual prompts occurring responsive to the input, and parameters of the image frames are determined. A parametric difference between corresponding ones of the root events and canvas responses is thereby computed, in order to determine a degree of visual responsiveness for the user interface software respective to the root input.
대표청구항▼
1. A system for testing user interface software for time lag in actuating a visual prompt responsive to user manipulation of a user input device, the system comprising: a display unit defining a canvas for displaying image frames;a root capturing unit executable to capture user actuation of the user
1. A system for testing user interface software for time lag in actuating a visual prompt responsive to user manipulation of a user input device, the system comprising: a display unit defining a canvas for displaying image frames;a root capturing unit executable to capture user actuation of the user input device as at least one root event at a root software level, an operating system operating at the root software level, each root event being captured as a series of time-displaced samples of input device actuation;a canvas capturing unit executable to capture processing of the root event by the user interface software as a canvas response at a canvas software level, the user interface software operating at the canvas software level for user interaction with an application, the canvas response being captured as a series of time-displaced image frames; and,a test analysis unit coupled to said root and canvas capturing units, said test analysis unit executable to determine a parametric difference between corresponding ones of the root events and canvas responses, and to determine a degree of visual responsiveness for the user interface software based thereon, said test analysis unit thereby discriminating portions of the time lag introduced at the canvas software level from portions of the time lag introduced at the root software level. 2. The system of claim 1, wherein the parametric difference corresponds to a response time difference between a start time of the input device actuation and a visual start time of the canvas response. 3. The system of claim 2, wherein: each of the canvas image frames includes a time parameter,the test analysis unit is further executable to determine a pixel intensity of an image frame, andthe test analysis unit determines the visual start time of the canvas response to be the time of a changed canvas image frame having a difference in pixel intensity from that of an initial canvas image frame exceeding a predefined intensity difference threshold, the initial canvas image frame having a time corresponding in time to the start time of the input device actuation. 4. The system of claim 2, wherein: each of the canvas image frames includes a time parameter,the test analysis unit is further executable to detect a predefined response image template within an image frame, andthe test analysis unit determines the visual start time of the canvas response to be the time of a responding canvas image frame containing the predefined response image template. 5. The system of claim 2, wherein: each of the canvas image frames includes a time parameter,the canvas capturing unit captures the processing of the root event for the duration of a predefined time threshold; andthe degree of visual responsiveness is determined to be below a benchmark responsiveness if the visual start time of the canvas response is determined not to be a time of any image frame of the series of time-displaced image frames. 6. The system of claim 1, wherein: each of the root event samples includes a time parameter and a coordinate location parameter,each of the canvas image frames includes a time parameter, thereby corresponding in time with one of the root event samples, andthe parametric difference corresponds to at least one response distance between the coordinate location of a root event sample and a coordinate location of a visual response in a canvas image frame corresponding in time to the root event sample. 7. The system of claim 6, wherein: the test analysis unit is further executable to detect a coordinate location of a predefined response image template within an image frame, andthe test analysis unit determines the coordinate location of the visual response in the image frame to be the image template coordinate location. 8. The system of claim 6, wherein: the test analysis unit includes a composite image generator executable to generate composite image frames each displaying one of the canvas image frames and a root sample signifier, the root sample signifier displayed at the coordinate location of the root event sample corresponding in time with the canvas image frame, the composite image frame having a time parameter corresponding with the time parameter of the canvas image frame,the test analysis unit is further executable to detect a predefined response image template within an image frame, and to detect the root sample signifier within an image frame, andthe test analysis unit determines the response distance of a root event sample as a pixel distance, in the composite image frame corresponding in time with the root event sample, between the predefined response image template and the root sample signifier. 9. The system of claim 6, wherein the degree of visual responsiveness is determined to be below a benchmark responsiveness if the response distance exceeds a predefined threshold distance for at least a predefined threshold number of image frames. 10. The system of claim 6, wherein the degree of visual responsiveness is determined to be below a benchmark responsiveness if the response distance exceeds a predefined threshold distance in each canvas image frame of a consecutive subseries of the series of canvas image frames where the consecutive subseries is longer than a predefined threshold number. 11. The system of claim 1, wherein: each of the root event samples includes a time parameter and a coordinate location parameter,each of the canvas image frames includes a time parameter, thereby corresponding in time with one of the root event samples, andthe parametric difference corresponds to a number or ratio of canvas image frames for which an expected visual response of a root event sample was not present in a canvas image frame corresponding in time to the root event sample, for at least a subset of the captured root event samples. 12. A benchmark test system for testing user interface software for time lag in actuating a visual prompt responsive to user manipulation of a user input device, the system comprising: a display unit defining a canvas for displaying image frames;an input event recorder executable to record user actuation of the user input device as at least one input event at a root software level, an operating system operating at the root software level, each input event being recorded as a series of time-displaced samples of input device actuation;an interface response recorder executable to record a processing of the root input event by the user interface software as an interface response at a canvas software level, the user interface software operating at the canvas software level for user interaction with an application, the interface response being recorded as a series of time-displaced image frames; and,a test analysis unit coupled to said input event and interface response recorders, said test analysis unit executable to compute at least one of a response time difference and a response distance between corresponding ones of the input event samples and interface response image frames, and to determine a degree of visual responsiveness for the user interface software based thereon, said test analysis unit thereby discriminating portions of the time lag introduced at the canvas software level from portions of the time lag introduced at the root software level. 13. A method of testing user interface software for time lag in actuating a visual prompt responsive to user manipulation of a user input device, the method comprising: recording user actuation of the user input device as at least one root event at a root software level, an operating system operating at the root software level, each root event being recorded as a series of time-displaced samples of input device actuation;recording a processing of the root event by the user interface software as a canvas response at a canvas software level, the user interface software operating at the canvas software level for user interaction with an application, the canvas response being recorded as a series of time-displaced image frames;computing a parametric difference between corresponding ones of the root events and canvas responses; anddetermining a degree of visual responsiveness for the user interface software based upon the parametric difference, thereby discriminating portions of the time lag introduced at the canvas software level from portions of the time lag introduced at the root software level. 14. The method of claim 1, wherein: each of the canvas image frames includes a time parameter,the parametric difference corresponds to a response time difference between a start time of the input device actuation and a visual start time of the canvas response, andthe parametric difference computation includes: computing a pixel intensity of an initial canvas image frame and at least one other of the canvas response image frames, the initial canvas image frame having a time corresponding in time to the start time of the input device actuation, anddetermining the visual start time to be the time of the other image frame if a difference in pixel intensity between the initial canvas image frame and the other image frame exceeds a predefined intensity difference threshold. 15. The method of claim 13, wherein: each of the canvas image frames includes a time parameter,the parametric difference corresponds to a response time difference between a start time of the input device actuation and a visual start time of the canvas response, andthe parametric difference computation includes: detecting a predefined response image template within at least one of the canvas image frames, anddetermining the visual start time to be the time of the canvas image frame containing the predefined response image template. 16. The method of claim 13, wherein: each of the canvas image frames includes a time parameter,the parametric difference corresponds to a response time difference between a start time of the input device actuation and a visual start time of the canvas response,the processing of the root event is recorded for the duration of a predefined time threshold, andthe visual responsiveness determination includes determining the degree of visual responsiveness to be below a benchmark responsiveness if the visual start time of the canvas response is not determined to be a time of any image frame of the series of time-displaced image frames. 17. The method of claim 13, wherein: each of the root event samples includes a time parameter and a coordinate location parameter,each of the canvas image frames includes a time parameter, thereby corresponding in time with one of the root event samples, andthe parametric difference corresponds to at least one response distance between the coordinate location of a root event sample and a coordinate location of a visual response in a canvas image frame corresponding in time with the root event sample. 18. The method of claim 17, wherein the parametric difference computation includes detecting a coordinate location of a predefined response image template within at least one canvas image frame, the coordinate location of the visual response in the canvas image frame being the coordinate location of the predefined response image template. 19. The method of claim 17, further comprising generating, for at least one canvas image frame, a composite image frame displaying the canvas image frame and a root sample signifier, the root sample signifier displayed at the coordinate location of the root event sample corresponding with time to the canvas image frame, the composite image frame having a time parameter corresponding with the time parameter of the canvas image frame, wherein the parametric difference computation includes: detecting a predefined response image template within at least one composite image frame,detecting the root sample signifier within the at least one composite image frame, andcomputing the response distance of a root event sample corresponding in time with the at least one composite image frame as a pixel distance between the predefined response image template and the root sample signifier in the composite image frame. 20. The method of claim 13, wherein: each of the root event samples includes a time parameter and a coordinate location parameter,each of the canvas image frames includes a time parameter, thereby corresponding in time with one of the root event samples,the parametric difference computation includes, for at least a subset of the recorded root event samples: determining an expected visual response for a root event sample, anddetermining if the expected visual response is contained within a canvas image frame corresponding in time to the root event sample, andthe parametric difference corresponds to a number or ratio of canvas image frames for which the expected visual response of a root event sample was not present in a canvas image frame corresponding in time to the root event sample.
Hemmerlein Rodney J. ; West Stephen, Apparatus and method for diagnosing erratic pressure sensor operation in a fuel system of an internal combustion engine.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.