Detecting TestSmells

Conditional Test Logic (metric, simple)

• parse testcases to detect conditionals and loops
• parse non-testcases (fixtures, utilities) in TestClass for conditionals and loops

Hard to test code

• could be detected by Christo -> code having no coverage, however this is no guarantee for "hard to test code", just a pointer to something missing

Hardcoded Test Data (metric, simple)

• scan testCases for numerics, strings, symbols and characters
  • -> primitive exposed behavior/state
  • -> broken encapsulation
  • -> missing OO

Obscure Test (metric, simple, many possibilities ?)

• parse the testcase and apply different metrics on the source. Question: how valuable are they ?

  • absoluteLength ( test-code )
  • assertion-code / non-assertion-code (statement or token based)
  • num ( testcases / non-testcases )

• too many source-methods covered by the test-code -> eager test or a more abstract scenario

• long test, but doesn't cover a lot -> some primitive/simpler tests missing, fixture code in the test, missing code reuse

Test Code Duplication (metric, simple)

• similarity analysis, levensthein (?), intelligent hash function (?)
• detect accessors being used for assertions -> field access instead of object comparision
• between classes -> can this be realized -> efficiency ?
• system supervision -> detect when a user is copy-pasting code from/to a testcase

Test Logic in Production (dynamic)

Assertion Roulette (dynamic)

• extend sunit in a way that a testcase knows which assertions failed -> better localization
• keep track about 'permamently' failing tests (history how they develop) (?)

Erratic Test (dynamic)

Fragile Test

Frequent Debugging

• system supervision for user debugging code, what code, timeintervals, ...

Manual Intervention (dynamic)

• detect execution halts/resumes for user interactions/events

Slow Tests (env)

• profile tests and save how long a test need for execution, keep a history (?)
  • -> how often does the user execute such tests
  • -> are tests raraly executed slow tests ? ...

Behavior Sensitivity Context Sensitivity Data Sensitivity Interface Sensitivity

Buggy Tests

• consequence of a fragile/obscure/flexible test

Developers not writing Tests (metric)

• just scan the working sets for tests

High Test Maintenance cost (env)

• if the developer spends alot of time in changing testcode (instead of adding new)
  • detect different kinds of changes: sth added/removed, code fixed ->still very similar after accepting it, ...

Production Bugs (env)

• frequency of test runs, coverage, num of tests, ....

Productivity Smell (env)

• how often does the user use the debugger (?) ...

---

Optionals (not very serious)

Naming Conventions

• parse the working set for source/test associations

  • class A should have a test called ATest, better: ATest should have a class called A -> easier documentation/organization

• parse for test-selectors containing numbers (e.g. test1, test2, test3, ...) -> not good for documentation

Test Organization

• intuitive category names

  • class A is in category Cat, so ATest should be in Cat-Tests

• category consistency

  • classes A and B are in Cat, then ATest and BTest should be in the same category, too

• separation: A and ATest should not be in the same category

Weak/organizational coverage

• associations: each class A should have a test named ATest, vice versa