TestSmells in a Nutshell

(Test)Code Smells

  • Conditional Test Logic
    • ?: conditionals and loops in test-code
    • leads to: Buggy/Obscure/Fragile Tests, maybe unrepeatable/non-deterministic tests, complex code, hard to understand
    • fixed by: refactoring tests, Guard Assertions, delegation, Mock Object, Doubles, Stubs
  • Hard to test code
    • caused by: highly coupled code, asynchronous code
    • leads to: manual testing
  • Hardcoded Test Data
    • ?: magical/suspect numbers and strings
    • leads to: Obscure Tests, hard to extract behavior/documentation
    • fixed by: fixtures, creation mthods
  • Obscure Test
    • ?: long test, much logic
    • caused by: not spending enough time in designing tests, Eager Tests, General/Shared Fixture, Mystery Guests (external resources), Verbose Test,
    • leads to: Buggy/Fragile Tests, increased maintenance costs, debugging due bad bug localization
    • fixed by: splitting up the test into smaller units, delegation
  • Test Code Duplication
    • caused by: copy-past, not reusing code or bad test design
    • leads to: Obscure/Fragile Tests, increased maintenance costs
    • fixed by: Custom Assertions, Verification Method, generic reusable code, test inheritance, Expected Objects - using objects instead of fields, delegation
  • Test Logic in Production
    • caused by: Test Hooks, code for tests only, test dependencies, equality pollution
    • leads to: hidden bugs
    • fixed by: special subclasses exposing behavior, utilities, clean up organisation/modularisation, custom assertions

Behavior Smells

  • Assertion Roulette
    • ?: which assertion caused the failure -> localization
    • caused by: Eager Tests,
    • leads to: debugging
    • fixed by: splitting up the test
  • Erratic Test
    • ?: determinism, sometimes tests pass, sometimes not
    • caused by: Interacting Tests/Suites, Shared Fixtures, execution order of tests, Lonely Test, Resource Leakage/Optimism, Unrepeatable Test
    • fixed by: fresh fixtures
  • Fragile Test
  • Frequent Debugging
    • caused by: lack of defect localization
    • leads to: manual testing, long debugging sessions
    • fixed by: test automation, small tests
  • Manual Intervention
    • ?: test need user interaction/input
    • caused by: manual fixtures/setup, event injection
    • leads to: unrepeatable tests
    • fixed by: simulate human interaction
  • Slow Tests
    • caused by: Obscure Tests, only high level tests, shared Fixtures,
    • leads to: decreased productivity
    • fixed by: split up in smaller parts

Sensitivity Smells

  • Behavior Sensitivity
    • ?: test A suddenly fails after B was modified
    • caused by: introducing a new feature, bug ’fixed’, setup modified
  • Context Sensitivity
    • ?: test A suddenly fails after the state of some other components has been changed
    • caused by: time/date dependencies, dependencies on other components
    • leads to: non-deterministic tests
  • Data Sensitivity
    • ?: test A suddenly fails after the data has been changed
    • caused by: data require new functionality, new data, data removed
    • leads to: increased effort in maintaining tests/data
  • Interface Sensitivity
    • ?: test A suddenly fails after the interface of the SUT has been modifies
    • caused by: changed application/API
    • leads to: no refactorings (’don’t touch it’)
    • fixed by: encapsulate the API/SUT behavior

Project Smells

  • Buggy Tests
    • ?: tests fail although they shouldn’t and vice versa
    • caused by: Fragile/Flexible/Obscure Tests, hard to test code, too much logic in the test, source changes too fast
    • leads to: Fragile Tests and high maintenance costs
    • fixed by: write test first (->requirements), refactor legacy code, learn testing
  • Developers not writing Tests
    • caused by: missing time, schedule, hard to test code, wrong automation strategy
    • leads to: Fragile/Obscure Tests, manual debugging/testing
  • High Test Maintenance cost
    • ?: it takes long to introduce new behavior
    • caused by: Fragile/Obscure Test, spending much time with debugging, manual testing, badly test strategy, missing test automation
    • fixed by: refactoring tests before continuing
  • Production Bugs
    • ?: too many tests fail, many bugs found in source
    • caused by: untested code, unfrequently run tests, not enough tests, lost/missing tests
  • Productivity Smell
    • caused by: frequent debugging, missing automation, slow tests
Last changed by admin on 21 April 2009