Threats to Validity in TDD Research

Timm Gross. Threats to Validity in TDD Research. Bachelor’s thesis, University of Bern, May 2020. Details.


Context: Test driven development (TDD) is an iterative software development technique where unit tests are defined before production code. Recent quantitative empirical investigations into the effects of TDD have been contrasting and inconclusive. Additionally studies have shown that TDD is not as widely used as expected. At the same time the body of research contains anecdotal evidence about the usefulness of TDD in practice. This makes it difficult for decision makers in development teams to use the research as the basis for the decision of whether or not to apply TDD. Objective: We present a study designed to uncover the threats to validity in previous studies that prevent them being usable in decision making processes. In order to do that we first studied what values practitioners associate with software testing. Method: We first conducted 15 hours of ethnographically informed qualitative interviews with a small development team to capture the perceived benefits of testing. Then we analysed the threats to validity mentioned in the body of research in a literature review. Results: The interviewed developers put equal emphasis on quality related aspects (i.e. productivity, internal and external code quality) and non quality related aspects (i.e. collaboration, confidence, knowledge transfer, etc.) of testing. In contrast the analyzed research papers focus almost exclusively on quality related aspects of TDD. In addition we identified the common threats to validity in the following areas: participants selection, task selection, context of the study, threats to validity regarding quality, length of observation, amount of iteration, comparisons to other techniques, measuring the adherence to TDD and a lack of qualitative research. Conclusion: Contrasting the views of practitioners on testing and the common threats to validity in TDD research allows us to highlight opportunities for further research. Especially for researchers aiming to provide scientific support for decision making processes of how and when to apply TDD in practice, this study summarizes important aspects to consider.

Posted by scg at 27 May 2020, 9:15 am link
Last changed by admin on 21 April 2009