Simple Statistics

updated: October 14, 2006

  • TestSmells/TestLint still runs too slow -> optimize run time
  • Aconcagua (supposed to be very "good") - collecting results of all rules, 1 afternoon ;( ;( —> smells and interesting things:
    • some untested behaviors sent/used
    • some behaviors untested (no coverage, no tests)
    • some behaviors implemented, but never/nowhere used
    • some behaviors only used in tests but not elsewhere (this is no problem for a system extension)
    • lots of cascaded test assertions (complexity, debug-ability)
    • a few "too" long tests (eager)
    • tremendous amount of magic literals (encapsulation break)
    • some dangerous literal conversions and comparisions (encapsulation break)
    • some non-effictive object comparisions (e.g. 1 = 1)
    • 1 hidden-shared fixture (shared fixture, but no (correct) setup/teardown)
    • no strict testing conventions (naming, .......)
    • lots of enumerations in tests (complexity, debug-ability)
    • most parts probably not done by TDD (fuzzy)
  • Network (well ... see Squeak Graveyard) - collecting results of all (except 1) rules, <5 min —> smells and interesting things:
    • only a very few tests, poor coverage, most behaviors untested
    • one early returning test (e.g. "^self" instead of running assertions)
    • some enumerations (complexity, debug-ability)
    • lots of equivalent tests ( -> partial ordering / coverage)
    • 2 hidden-shared fixtures (shared fixture, but no (correct) setup/teardown)
    • tremendous amount of literal conversions and comparisions (-> can be expected as Network is quite lowlevel, encapsulation break)
    • some logic operations in tests (ifTrue / ifFalse / ..., repeatability)
    • tremendous amount of magic literals ( -> can be expected as Network is quite lowlevel, encapsulation break)
    • rather complex and long tests with lots of comments (complexity, readability, documetation)
    • 1 outcommented test code, but wouldn’t break if comment removed - lucky ;)
    • poor names for test selectors (e.g. test1, test2, test3, ...)
    • much behavior only used for tests
    • not done in TDD (fuzzy)
  • MSE-Exporter - collecting results of all (except 1) rule —> smells and interesting things:
    • some enumerations (complexity, debug-ability)
    • some behaviors only used in tests but not elsewhere (this is no problem for a system extension)
    • 1 non-effictive object comparisions (e.g. 1 = 1)
    • many literal conversion/comparisions (encapsulation break)
    • many magic literals
    • some test-code put into comments
    • 1 outcommented test that would fail on comment removal - UPS ! -> MSEExportTest>#testClassCategory
    • 3 very long tests (size and runtime)
    • 1 test with too many comments (more comments than actual test code)
    • 2 classes with bad selector names (complexity, readability, documetation)
    • some behaviors implemented, but never/nowhere used
    • some untested behaviors sent/used
    • some behaviors untested (no coverage, no tests)
  • Intelligent Games
    • tremendous amount of magic literals ( -> expected as part of the "game")
    • some enumerations (complexity, debug-ability)
    • some behaviors implemented, but never/nowhere used
    • some untested behaviors sent/used
    • some behaviors untested (no coverage, no tests)
  • Nebraska
    • only a very very few tests > lots of behaviors untested (no coverage, no tests)
    • some behaviors only used in tests but not elsewhere (this is no problem for a system extension)
    • some magic literals
    • 1 organization convention ignored
    • 2 bad names for testselectors
    • ? none of the fixtures could not be analysed -> instrumentation error ?
  • Files / Filesystem
    • many tests returning "too early" (->not all assertsions/statements executed -> ^self/^nil/.... )
    • many logic operators (-> droping assertions -> ifTrue: / ifFalse: / ...)
    • many magic literal (expected)
    • 2 mystery guests (access to an external file)
    • tests contain rather complex than simple assertions
    • 3 tests with outcommented code
    • organization convention ignored (almost everywhere, development was a bit sloppy ?)
    • lot of comments in tests, 1 really really really big one (6 lines)
    • one testclass does only have a teardown (that doesn’t do what it is supposed to do) -> ok, but not really nice -> sideffects of tests
  • ICal
    • 1 method overrides the default perform-behavior of sunit
    • some test-methods with outcommented code
    • some test-methods with outcommented assertsions that would fail on comment removal - UPS !!!
    • 1 testmethod is completely outcommented
    • many enumerations (complexity, debug-ability)
    • many missing tests
    • at least 1 testclass having hidden shared fixtures (shared fixture, but no (correct) setup/teardown)
    • 1 class with a fixture not used enough (mostly unused)
    • many literal conversions and comparisions
    • tremendous amount of magic literals ( expected)
    • 4 testclasses with "too many" instance variables, only 2 of them have a too large fixture
    • many testmethods not using the shared fixture (unbalanced use)
    • 2x organization convention ignored
    • 1 test-method with no assertions at all
    • some compex assertions (nested, cascaded, block, ...)
    • (unsure) 1 shared fixture instead on fresh fixtures
    • many bad test-selector names
    • unmanaged variables instead of using a fixture
    • some behavior that is nowhere use nor tested
    • 2 shared fixtures with variables/behavior never used/accessed by any test-method
  • RoelTyper
    • some long testcases / manual tests -> no test selectors
    • some enumerations (complexity, debug-ability)
    • many missing tests / bad coverage
    • 1 hidden-shared fixture (shared fixture, but no (correct) setup/teardown)
    • some logic operators (-> droping assertions -> ifTrue: / ifFalse: / ...)
    • tremendous amount of magic literals
    • many bad test-selector names
    • organization convention ignored
  • Cryptography
    • >>> quite difficult to analyse as ’Cryptography’ is not just one package but in fact a bundle of seperate packages - maybe a set for analysis would be appropriate, tests are not together with the code ...
    • average coverage in generel: ASN (0%), Core (27%), DES (59%), DSA (23%), ELG (88%), MD4 (84%), MD5 (90%), RC4 (75%), RSA (32%), RiJ (51%), SHA (58%), SHA256 (94%), SSL (0%), X509 (0%), Base (62%)
    • only a very very few very abstract tests (~30 for such a big package !, bad: complexity, debug-ability, bug-tracking)
    • all tests have magic literals (expected)
    • some very long tests (runtime and code)
    • 1 test with literal conversion (encapsulation break)
    • many enumerations (complexity, debug-ability)
    • test-organization convention ignored
    • many untested behaviors
  • Tsunami
    • medium-good coverage
    • some literal conversions
    • organization convention ignored
    • more abstract tests than primitive ones, inly a few tests
  • OpenDocument
    • some enumerations
    • tests with too many referenced / coupling
    • many literal conversions
    • rather complex tests with a lot of control logics
    • tremendous amount of magic literals
    • some very long tests
    • organization convention ignored
    • many tests have too many/large comments
    • some tests with outcommented code and assertsions
    • 1 test with outcommented assertions that would fail on comment removal - UPS
    • bad test selector names
    • much untested behavior (sent)
Last changed by admin on 21 April 2009