• Skip to content
  • Skip to footer

Craftsmanship Counts

We're Hardcore About Software

  • Home
  • Courses
  • Blog
  • About
  • Contact Us

Eddie / July 30, 2016

The Agile Testing Pyramid: A Strategy for Success

The Agile Testing Pyramid: A Strategy for SuccessIn case you’ve forgotten, the second declaration of the Agile Manifesto states a preference for:

Working Software Delivered Over Comprehensive Documentation

With this in mind, it should be no surprise that there is such an emphasis on testing in projects being built with Agile methodologies.

While it’s important to have testing, and to automate as much of it as possible, it’s also important to create tests with a strategy in mind. One great strategy to apply is Mike Cohn’s Agile Testing Pyramid.

Mike Cohn’s Agile Testing Pyramid

Mike’s Agile Testing Pyramid provides a great strategy for the types of tests to create and automate. It also provides guidance on how many of the different types of tests you should have.

Let’s take a brief look at the types of tests that make up the pyramid. Once we’ve done that, we’ll circle back and dig more deeply into each type of test, why it’s important, and how to write it.

The Foundation: Unit Tests

The foundation of Mike’s Agile Testing Pyramid is unit tests. Perhaps seventy percent of all your tests should be unit tests. I say perhaps to ensure you will feel free to flex a bit one way or the other.

The important thing is that most of your tests are unit tests. Shoot for 100% coverage with these, but ensure you’re at least in the high 90s.

The Middle: Service Tests

Mike defines a service as a business operation. In his example of a calculator, he suggests that a service would be an operation like add or multiply. If you’re using Clean Architecture, this would align with a single Interactor or Story.

Aim to have at least one service test for each service you’re testing, and make sure each one of these is an subcutaneous end-to-end test.

The Top: UI Tests

The top of Mike’s Agile Testing Pyramid are UI tests. Where the service tests are subcutaneous, these comes straight through the UI, using Selenium or something similar.

If you think about it, it’s likely that the same UI drives several service operations. Since these tests are sanity checks on the wiring between the UI and services, you should need far fewer of these.

Unit Tests – A Foundation to Build On

As development proceeds, unit tests give a developer confidence that they are building the thing right. Each build confirming that things work as the developer intended.

Because unit tests are applied directly to a single class – disconnected from its collaborators by installing test doubles – there should be one test for each outcome a class can produce. Because these tests have checks for each value that is emitted during each outcome we call them direct tests.

The result? When something breaks, you have a specific test method that fails. You can tell exactly what went wrong. Discovery time is reduced, and – instead of guessing – a developer can put their energy into fixing things.

This feedback enables a developer to move quickly, with high confidence, knowing that the code they are building works.

Service Tests – Proof That Everyone Gets Along

Unit tests are a great start, but they are only one piece of a good automated testing strategy. You also need to automate tests that prove the units work well together, and that’s where service tests come into play.

Service tests are subcutaneous, meaning they come in “under the skin” (the UI) and operate the entire stitched-together system. They do this by calling the controllers directly.

A lot of people want to keep the database out of these tests – just as with the unit tests. However, it’s really important to know that all the pieces of the system are working properly, so I’d probably have a specific test database instance that is pre-initialized with any static data and that has all dynamic or transactional data removed between test executions.

Otherwise, you’re not really testing the whole system.

UI Tests – Validation of UI Integration

Once service tests have executed, the only things left unknown are whether the UI can talk to the services, and whether the UI acts appropriately given certain outcomes (rosy vs exceptional path). If you consider that several error responses result in the same error behavior, you probably don’t need to check the error path for everything.

The same is probably true for the rosy path.

UI tests are generally the most fragile and the most expensive. Err on the light side in the start. You can always add more if you feel they are warranted.

Wrapping It Up

If you think about the relationship between these different types of tests, there are some things you can know that might not be obvious. When unit tests fail, the cause is clear. However, once we get to service tests – and especially once we get to UI tests – the cause may become more murky.

For UI and service test failures, it is possible that the error relates to that specific integration step (UI code for UI tests and service/component code for service tests) or that the problem is a result of the developer implementing the next level “down” incorrectly.

For example, a failing UI test may come about because a developer has incorrectly implemented the UI to service integration. However, failing this, it could also result from someone having stitched the service together incorrectly.

Just because a test passes doesn’t mean the code it tests is implemented properly. It simply means that it meets the developer’s expectation. For this reason, if you feel you should omit the UI tests, you should make sure that you do not skip the service tests.

Otherwise, the only integration testing you have is manual execution. While it certain can work, it will be much less efficient than an automated test. Particularly when you consider that you can factor defects back into tests and know that you are avoiding regressions.

Closing Thoughts

Remember that, no matter how comprehensive your test suite is, they were written by humans. Humans documented the requirements that were used to implement the test and code too.

Generally, be somewhat suspect of your unit tests until you have integration tests that prove the units were written properly. And, realize that this does not diminish the value of the unit test. It simply means that humans are imperfect and computers do what we tell them to.

That’s it for this week! Until next time, this is Eddie Bush with Craftsmanship Counts reminding you to keep your tests passing and your code clean!

Filed Under: Featured Tagged With: Agile Testing

Save 52% From Black Friday to Cyber Monday!

Looking for a Clean Architecture course that covers more than theory? Check out Java Clean Architecture by Example! Click Here to Save 52% Through Cyber Monday!

Weekly Newsletter

Our number one goal is to help you improve a little bit each time we press publish. If that sounds like fun to you, just fill out the following form and click 'Yes! Sign Me Up!'

By joining the Craftsmanship Counts Newsletter, you agree to our Privacy Policy.

Reader Interactions

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Footer

Contact Us

Craftsmanship Counts
13364 S 4328
Chouteau, OK 74337

918-373-3594

About Us

Craftsmanship Counts is a botique consultancy offering Agile leadership, development, and clean coding services.

Learn More

Follow Us

  • Google+
  • LinkedIn
  • Twitter

© 2016 Craftsmanship Counts, LLC - All Rights Reserved

  • Privacy
  • Anti-Spam