Colts Neck, NJ, USA | 25 October 2010 – Testcover.com released a summary of the company’s research activities during the last five years. The report is reprinted below for Testcover.com NEWS.
Introduction
The performance of a pairwise test case generator depends on two essential components: the number of test cases produced and the response time for generating them. Practical software engineering tools require an optimal balance between the two. A quick response with many test cases may present the test team with unnecessary work. On the other hand, a fast-paced development organization needs tools that can keep up. For this reason, Testcover.com’s research program involves ongoing improvements to both components, and the examples on our performance page report both.
Testcover.com’s research results are of interest to software engineering practitioners as well as researchers. Although some work is proprietary, we have shared many results in peer-reviewed publications and in conference presentations. This summary reports on Testcover.com’s research activities over the last five years and the impact of this work on service performance–number of test cases and response time.
Proprietary research
WSDL interface. In early 2010 Testcover.com introduced its WSDL interface to enable integration with other test tools. Although performance improvement was not the main objective, this WSDL implementation offers even faster response times compared with the HTML interface. When coupled with a test automation tool, the service has the potential to save even more time in the test design and execution process.
Test case search improvement. Unpublished research conducted this year has found significant response time improvements from an optimization of our search algorithm. Response time is reduced in small search spaces as well as large ones, with bigger reductions in the larger search spaces. Current plans call for introduction of the improvement in 2011.
Test case reduction. Testcover.com runs an ongoing research program to reduce the number of test cases generated by the service. For example, at STAREAST 2005 we demonstrated our results generating test cases for an HTML form. Test values were selected for 21 test factors, and 63 test cases were generated. Since this example was first shown, its results have been improved to 59 test cases (a reduction of more than 6%). The service has had thousands of such improvements in the last five years.
Academic papers
Optimal and near-optimal mixed covering arrays by column expansion. GB Sherwood. Discrete Mathematics, 308(24):6022-6035, 2008. This paper described new constructions for strength two mixed arrays having minimal numbers of test cases. It also showed how to maximize the numbers of factors in the arrays. The best constructions for various array parameters were reported in several tables.
Products of mixed covering arrays of strength two. CJ Colbourn, et al. Journal of Combinatorial Designs, 14(2):124-138, 2006. Strength two mixed covering arrays are the templates for pairwise test cases. This paper brought six authors together to report on several different construction methods for these arrays. It also discussed how the methods can be combined effectively.
Covering arrays of higher strength from permutation vectors. GB Sherwood, et al. Journal of Combinatorial Designs, 14(3):202-213, 2006. Higher-strength covering arrays are templates for testing values beyond pairwise: Three-at-a-time, four-at-a-time, etc. This paper introduced a new technique for constructing arrays having fewer test cases than previously known arrays. Subsequently, academic researchers have extended the technique to find additional higher strength arrays.
Public presentations
Pairwise Testing Comes of Age. This talk at the STAREAST 2008 reviewed the development of pairwise testing and provided examples to illustrate its use. The talk also discussed research topics and technical advances related to performance improvement.
A Column Expansion Construction for Optimal and Near-Optimal Mixed Covering Arrays. This presentation in May 2006 described research in progress for the Workshop on Covering Arrays at Carleton University in Ottawa, Canada. This work in its completed form was published in Discrete Mathematics (above).
Improving Test Design with a Novel Test Case Generator. This talk at STAREAST 2005 introduced Testcover.com’s new technology and its continuous improvement through ongoing research. The HTML form example (above) was first demonstrated at this conference.
About Testcover.com
Testcover.com offers a pairwise test case generator that helps testers get the results they need to develop optimal products and services in a fraction of the time, saving time and money on every project. Testcover.com is a software as a service (SaaS) independent test design solution and its Web Services Description Language (WSDL) interface enables ready integration with existing test tools. For more information visit http://testcover.com, dial +1 732 299 4852 or email mgmt@testcover.com.
