Technical Report
 

Mutant census : an empirical examination of the competent programmer hypothesis

Public Deposited

Downloadable Content

Download PDF
https://ir.library.oregonstate.edu/concern/technical_reports/fb494967q

Descriptions

Attribute NameValues
Creator
Abstract
  • Mutation analysis is often used to compare the effectiveness of different test suites or testing techniques. One of the main assumptions underlying this technique is the Competent Programmer Hypothesis, which proposes that programs are very close to a correct version, or that the difference between current and correct code for each fault is very small. Testers have generally assumed, on the basis of the Competent Programmer Hypothesis, that mutation analysis with single token changes produces mutations that are similar to real faults. While there exists some evidence that supports this assumption, these studies are based on analysis of a limited and potentially non-representative set of programs and are hence not conclusive. In this paper, we investigate the Competent Programmer Hypothesis by analyzing changes (and bug-fixes in particular) in a very large set of randomly selected projects using four different programming languages. Our analysis suggests that a typical fault involves about three to four tokens, and is seldom equivalent to any traditional mutation operator. We also find the most frequently occurring syntactical patterns, and identify the factors that affect the real bug-fix change distribution. Our analysis suggests that different languages have different distributions, which in turn suggests that operators optimal in one language may not be optimal for others. Moreover, our results suggest that mutation analysis stands in need of better empirical support of the connection between mutant detection and detection of actual program faults in a larger body of real programs.
  • Keywords: testing tools, testing and debugging, empirical analysis, test frameworks, mutation operators, real faults
Resource Type
Date Available
Date Issued
Series
Subject
Rights Statement
Publisher
Peer Reviewed
Language
Replaces

Relationships

Parents:

This work has no parents.

Items

Thumbnail Title Date Uploaded Visibility Actions
file details acm_sigproc.pdf 2017-07-18 Public Download