Do You Need to Test Everything?

You get a test, you get a test, everyboooody gets a test!

Testing’s dark side

  • Investment: an ever-growing test suite has maintenance costs. If the time invested in writing a test nets a positive return, then you’ve added value. But there are no guarantees that the ROI won’t be negative at times.
  • Less readability: testable code is often less readable. Abstractions like inversion of control, dependency injection, mocking, or stubbing are common and make for hard-to-read code.
  • No silver bullet: at times, code makes us jump through hoops to write a test. For instance, a controller in an MVC application needs extensive mocked requests and responses, even to test a little bit of logic.
  • Prototyping: until you have a working prototype, testing is more of a burden than a boon. Testing starts to make sense once the system matures and interfaces stabilize.
  • False positives: tests are also code and they can be buggy, causing false positives and making you hunt for nonexistent bugs.
  • Green mania: I call this the compulsion to reach 100% coverage. While seeing a full green bar is satisfying, no amount of tests can prove that a system is error-free. Testing everything can generate the illusion of a perfect codebase, but it doesn’t mean that there are no problems.
  • Speed: while there are methods to make tests run very fast, they do still take time and resources to run. So you should make each one count.
  • Cargo cult: some developers feel shame if they don’t write tests for everything.

The goal is to write code that works

  • Do I have a fast test suite? Does your CI/CD pipeline run in under 10 minutes? If not, allocate some effort to streamlining it before extending the suite.
  • Am I following a specification? Is the test covering business requirements? Do you have a specification defined? If so, you must write enough tests to ensure you’re following it.
  • How critical is my code? Not every piece of code is equally important; some may not even deserve a test. Consider what’s the worst that can happen if your code has errors.
  • Is my code interesting? Is the code under test complex enough to warrant a test? Avoid testing trivial code, and make sure you’re not testing the compiler.
  • Am I heavily refactoring only for the sake of testing? How difficult is it to implement a test? In other words, is the test more complex than the actual code tested? If you need to add several levels of ad hoc code, maybe the cure is worse than the disease.
  • Do I control all elements in the test environment? Do you fully own the system? External dependencies lead to flakiness. If you depend on third parties, maybe you need some form of contract testing instead.
  • Do I need excessive setup? Do you need to set up a ton of things to test? It could mean that your code is too tightly coupled, or you’re trying to do unit testing where integration or end-to-end tests would be a better fit.

Final thoughts

--

--

--

Supporting developers with insights and tutorials on delivering good software. · https://semaphoreci.com

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

How to Become a Better Software Developer

Send AMP Emails with Ruby on Rails

AWS Features

Config Driven Reactive Programming in Ruby

The value of math operations inside ManyChat bots - Round 2

Using RAPIDS Memory Manager with CuPy

Difference Between Frameworks and Libraries

JAAA, Joint Account Autonomous Agent

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Semaphore

Semaphore

Supporting developers with insights and tutorials on delivering good software. · https://semaphoreci.com

More from Medium

Cucumber Founder Aslak Hellesøy on TDD and BDD

The Single Best Indicator of Software Quality Is Automated Testing

Debugging the debug process

Why do unit tests matter?