초록 ▼

The techniques described herein provide software testing of a candidate software system. In some examples, a testing service compares at least one candidate response to at least a first control response to obtain one or more candidate test differences. The testing service may compare at least a seco

The techniques described herein provide software testing of a candidate software system. In some examples, a testing service compares at least one candidate response to at least a first control response to obtain one or more candidate test differences. The testing service may compare at least a second control response of the plurality of control responses to at least one of the first control response of the plurality of control responses or a third control response of the plurality of control responses to obtain one or more control test differences. The testing service may then analyze the one or more candidate test differences based on the one or more control test differences to generate an evaluation of whether one or more of the candidate test differences are due to differences between the candidate software system and the control software system that generated the first control response.

대표청구항 ▼

1. A computer-implemented method comprising: receiving, based at least in part on a candidate request, an intermediate candidate communication that is issued by a candidate software system, the intermediate candidate communication generated at least in part to validate the candidate software system;

1. A computer-implemented method comprising: receiving, based at least in part on a candidate request, an intermediate candidate communication that is issued by a candidate software system, the intermediate candidate communication generated at least in part to validate the candidate software system;receiving a plurality of intermediate control communications that are issued by a control software system based at least in part on at least one control request, wherein the control software system is used for validating the candidate software system;comparing the intermediate candidate communication to at least some of the plurality of intermediate control communications to obtain validation data indicating validation data differences between the intermediate candidate communication and the at least some of the plurality of intermediate control communications;identifying variation data included in the plurality of intermediate control communications;analyzing the variation data and the validation data based at least in part on the at least some of the plurality of intermediate control communications to determine whether the differences are due to variations between the candidate software system and the control software system; andexcluding variation data differences between the candidate software system and the control software system corresponding to the variation data, the variation data differences corresponding to the variation data. 2. The computer-implemented method as recited in claim 1, further comprising outputting at least some of the differences via a dashboard service that facilitates interaction with the candidate software system. 3. The computer-implemented method as recited in claim 1, further comprising intercepting client requests for processing by the candidate software system to generate the intermediate candidate communication. 4. The computer-implemented method as recited in claim 3, further comprising intercepting client requests based at least in part on one of a percentage of the client requests or a type of client request. 5. The computer-implemented method as recited in claim 1, further comprising determining a score representing the differences based at least in part on the validation data. 6. The computer-implemented method as recited in claim 1, wherein the plurality of intermediate control communications are issued by the control software system to at least one service stub. 7. A system comprising: one or more processors; andmemory storing computer-implemented instructions that, when executed, cause the one or more processors to perform acts comprising:receiving, based at least in part on a candidate request, a candidate communication that is issued by a candidate software system, the candidate communication generated at least in part to validate the candidate software system;receiving a first control communication and a second control communication that are issued by a control software system based at least in part on at least one control request, wherein the control software system is used for validating the candidate software system;comparing the candidate communication to at least one of the first control communication or the second control communication to obtain validation data indicating differences between the candidate communication and at least one of the first control communication or the second control communication;determining that the first control communication is different than the second control communication; anddetermining that the differences between the candidate communication and the at least one of the first control communication or the second control communication are due to the first control communication being different than the second control communication. 8. The system as recited in claim 7, wherein the acts further comprise outputting at least some of the differences to a dashboard service that facilitates user interaction with at least the candidate software system. 9. The system as recited in claim 7, wherein the acts further comprise receiving the candidate communication as an intermediate candidate communication that is received subsequent to an initial candidate communication. 10. The system as recited in claim 7, wherein the acts further comprise receiving a third control communication that is issued by the control software system based at least in part on the at least one control request, wherein the third control communication is the same as the first control communication or the second control communication. 11. The system as recited in claim 7, wherein the acts further comprise intercepting client requests for processing by the candidate software system to generate an intermediate candidate communication. 12. The system as recited in claim 11, wherein the acts further comprise employing sampling rules to cause the intercepting of the client requests, the client requests being a subset of possible client requests available to be intercepted. 13. The system as recited in claim 7, wherein the control software system mimics operation of a production software system, wherein the production software system modifies customer data while the control software system refrains from modifying the customer data. 14. The system as recited in claim 7, wherein the first control communication and the second control communication are issued by the control software system based on a same control request. 15. The system as recited in claim 7, further comprising applying a score to the differences based at least in part on the first control communication being different than the second control communication. 16. A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform operations comprising: receiving, based at least in part on a candidate request, a candidate communication that is issued by a candidate software system, the candidate communication generated at least in part to validate the candidate software system;receiving a first control communication and a second control communication that are issued by a control software system based at least in part on at least one control request, wherein the control software system is used for validating the candidate software system;comparing the candidate communication to at least one of the first control communication or the second control communication to obtain validation data indicating differences between the candidate communication and at least one of the first control communication or the second control communication;determining that the first control communication is different than the second control communication; anddetermining that the differences between the candidate communication and the at least one of the first control communication or the second control communication are due to the first control communication being different than the second control communication. 17. The non-transitory computer-readable medium of claim 16, the operations further comprising extracting metadata from at least one of the candidate communication, the first control communication, or the second control communication, the metadata comprising performance data. 18. The non-transitory computer-readable medium of claim 16, wherein the comparing the candidate communication to at least one of the first control communication or the second control communication comprises comparing logical portions and performance data of the candidate communication to at least one the first control communication or the second control communication. 19. The non-transitory computer-readable medium of claim 16, the operations further comprising outputting at least some of the differences to a dashboard service that facilitates user interaction with at least the candidate software system. 20. The non-transitory computer-readable medium of claim 16, the operations further comprising storing at least a portion of at least one of the candidate communication, the first control communication, or the second control communication.