Reducing conditional code makes assessing coverage or debugging difficult

by Peter Wood   Last Updated February 22, 2017 03:05 AM

Working in Python, I've written a few utilities to assert pre-conditions and to re-interpret exceptions, and have found my code doesn't have so many branches.

I've also tried to move other code path selection into maps, so depending upon data conditions, an appropriate object will be looked up to carry out the operations.

My code is mainly just a sequence of statements.

I now have difficulty placing breakpoints or assessing code coverage.

Is this normal when you move to a more declarative style? (obviously I'm still procedural)

As an example, instead of raising an exception upon some condition, I pass all those details to a function, insist.

How it looked before:

if not condition:
    raise MyError('details {something} or {other}',
                  something=something,
                  other=other)
else:
    # rest of code

How it looks now:

insist(condition, MyError, 
       'details {something} or {other}',
       something=something,
       other=other)

# rest of code

I also have something similar for eliminating try/except blocks when I know I just want to raise a new exception.



Answers 1


Having less branch technically makes it easier to attain a high branch coverage, but as you suspected, higher coverage does not always mean the tests were good enough. To an extreme, you could play all sorts of tricks to eliminate branches or make it easier to cover them, only boost the score not the actual quality of your tests. Other structural coverage criteria also suffer from this limitation however fancy they are (even MCDC does). How the code is structured affects the coverage and thus affect fault finding effectiveness as well (http://ieeexplore.ieee.org/abstract/document/4814127/).

A remedy to this problem is to carefuly write test oracles, or assertions, together with test inputs or driver code. Intuitively speaking, you're more likely to catch any bug if you check more, given the same number of test cases. Some metrics are suggested by academics to measure oracle coverage, requirement coverage, or checked coverage, etc., but no ready-to-use tool exists that I know of.

Taejoon Byun
Taejoon Byun
February 22, 2017 02:48 AM

Related Questions