his.sePublications
Change search
Link to record
Permanent link

Direct link
BETA
Márki, András
Publications (4 of 4) Show all publications
Lindström, B. & Márki, A. (2019). On strong mutation and the theory of subsuming logic‐based mutants. Software testing, verification & reliability, 29(1-2 Special Issue: SI), 1-23, Article ID e1667.
Open this publication in new window or tab >>On strong mutation and the theory of subsuming logic‐based mutants
2019 (English)In: Software testing, verification & reliability, ISSN 0960-0833, E-ISSN 1099-1689, Vol. 29, no 1-2 Special Issue: SI, p. 1-23, article id e1667Article in journal (Refereed) Published
Abstract [en]

Redundant mutants might cause problems when benchmarking since testing techniques can get high scores without detecting any nonredundant mutants. However, removing nonredundant mutants might cause similar problems. Subsumed mutants are per definition also redundant since no additional tests are required to detect them once all other mutants are detected. We focus on relational operator replacement (ROR) and conditional operator replacement mutants. Subsumption relations between ROR mutants are defined by fault hierarchies. The fault hierarchies are proven for weak mutation but have since they were published been used with strong mutation. We prove that ROR fault hierarchies do not hold for strong mutation and show why. We also show that the probability for a random test to experience the problem can be more than 30% and that 50% of the mutants might be affected in a real software system. Finally, we show that there is a similar problem with the theory on sufficient conditional operator replacement.

Place, publisher, year, edition, pages
John Wiley & Sons, 2019
Keywords
error propagation, mutant subsumption, mutation testing, redundant mutants
National Category
Computer Sciences
Research subject
Distributed Real-Time Systems
Identifiers
urn:nbn:se:his:diva-15090 (URN)10.1002/stvr.1667 (DOI)000458911000003 ()2-s2.0-85045856317 (Scopus ID)
Projects
TOCSYC
Funder
Knowledge Foundation, 20130085
Available from: 2018-04-21 Created: 2018-04-21 Last updated: 2019-03-05Bibliographically approved
Márki, A. (2019). Towards minimal mutation analysis: Using the approximated dominator set of mutants.
Open this publication in new window or tab >>Towards minimal mutation analysis: Using the approximated dominator set of mutants
2019 (English)Report (Other academic)
Abstract [en]

In mutation testing, variants (i.e., mutants) of the software under test are created. Themutants are then used to design tests that can detect the difference between the mutantsand the original software under test. Empirical studies have shown that test suites thatare effective in detecting mutants are also effective in detecting real faults. Mutationanalysis is therefore often used to benchmark effectiveness of other testing techniques.The main drawback of mutation testing is that it is computationally expensive becauseof the large number of mutants to analyze. It is well known that many of these mutantsare redundant and recent studies have shown that the redundancy among the mutantscan be up to 99%. However, identifying which mutants that are redundant is challengingsince this depends on the software under test as well as the specific mutations.

This work aims to combine techniques from areas, such as static analysis and machinelearning, in a process for cost-effective mutation analysis. Such techniques are expectedto provide partial solutions to the problem of avoiding creation of the redundant mutants.The outcome of this research is two-fold: (i) an evaluation of techniques that canbe used to minimize the set of non-redundant mutants that needs to be created, and (ii)a process for mutation analysis combining such minimization techniques. A frameworkwill also be developed to evaluate the minimization techniques and the entire process.

Publisher
p. 48
Keywords
mutation analysis, dominator mutants, redundant mutants, mutation testing, software testing, mutant minimization
National Category
Computer and Information Sciences
Research subject
Distributed Real-Time Systems
Identifiers
urn:nbn:se:his:diva-16771 (URN)
Note

Research proposal, PhD programme, University of Skövde

Available from: 2019-04-12 Created: 2019-04-12 Last updated: 2019-04-12Bibliographically approved
Márki, A. & Lindström, B. (2017). Mutation tools for Java. In: SAC '17 Proceedings of the Symposium on Applied Computing: . Paper presented at 32nd ACM SIGAPP Symposium On Applied Computing, Marrakech, Morocco, April 4-6, 2017 (pp. 1364-1371). Association for Computing Machinery (ACM)
Open this publication in new window or tab >>Mutation tools for Java
2017 (English)In: SAC '17 Proceedings of the Symposium on Applied Computing, Association for Computing Machinery (ACM), 2017, p. 1364-1371Conference paper, Published paper (Refereed)
Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2017
National Category
Computer Sciences
Research subject
Distributed Real-Time Systems
Identifiers
urn:nbn:se:his:diva-13390 (URN)10.1145/3019612.3019825 (DOI)2-s2.0-85020881320 (Scopus ID)978-1-4503-4486-9 (ISBN)
Conference
32nd ACM SIGAPP Symposium On Applied Computing, Marrakech, Morocco, April 4-6, 2017
Available from: 2017-02-14 Created: 2017-02-14 Last updated: 2018-01-13Bibliographically approved
Lindström, B. & Márki, A. (2016). On Strong Mutation and Subsuming Mutants. In: Proceedings: 2016 IEEE International Conference on Software Testing, Verification and Validation Workshops. Paper presented at IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 11-15 April 2016, Chicago, IL, USA (pp. 112-121). IEEE Computer Society
Open this publication in new window or tab >>On Strong Mutation and Subsuming Mutants
2016 (English)In: Proceedings: 2016 IEEE International Conference on Software Testing, Verification and Validation Workshops, IEEE Computer Society, 2016, p. 112-121Conference paper, Published paper (Refereed)
Abstract [en]

Mutation analysis is a powerful technique for software testing but it is also known to be computationally expensive.The main reason for the high computational cost is that many of themutants are redundant and thus, do not contribute to the quality of the test suite. One of the most promising approaches toavoid producing redundant mutants is to identify subsumption relations among mutants, preferably before these are generated.Such relations have for example, been identified at an operator level for mutants created by the ROR operator. This reduced set of non-redundant mutants hasbeen used in several recent studies and is also the default option in at least one mutation testing tool that supports strong mutation. This raises questions on whether the identified subsumption relations between the mutants hold in a context ofstrong mutation or variants of weak mutation that require some limited error propagation (firm mutation).

We have conducted an experimental study to investigate the subsumption relations in the context of strong or firm mutation.We observed that it is possible to create a test suite that is 100\% adequate for the reduced set of mutants while not being 100\% adequate for the complete set. This shows that the subsumption relations do not hold for strong or firm mutation. We provide several examples on this behavior and discuss the root causes. Our findings are important since strong and firm mutation both are frequently used to evaluate test suites and testing criteria. The choice of whether to use a reduced set of mutants or an entire set should however, not be made without consideration of the context in which they are used (i.e., strong, firm or weak mutation) since the subsumption relations between ROR mutants do not hold for strong or firm mutation.Just as redundant mutants can give an overestimation of the mutation score for a test suite, using the reduced set of mutantscan give an underestimation if used together with strong or firm mutation. Results reported from such studies should therefore, be accompanied by information on whether the reduced or complete set of mutants was used and if the researchers used strong, firm or weak mutation.

Place, publisher, year, edition, pages
IEEE Computer Society, 2016
Keywords
mutation testing, redundant mutant, strong mutation, subsumption
National Category
Computer Sciences
Research subject
Distributed Real-Time Systems
Identifiers
urn:nbn:se:his:diva-12766 (URN)10.1109/ICSTW.2016.28 (DOI)000382490200017 ()2-s2.0-84992184216 (Scopus ID)978-1-5090-1826-0 (ISBN)
Conference
IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 11-15 April 2016, Chicago, IL, USA
Projects
TOCSYC
Funder
Knowledge Foundation, 20130085
Available from: 2016-08-09 Created: 2016-08-09 Last updated: 2019-01-22Bibliographically approved
Organisations

Search in DiVA

Show all publications