Since Pact was first written in 2013, it has been adopted by many organizations around the world. A great number of these have found that Pact has helped speed up their testing cycles, reduced the number of integration bugs, and helped them increase their speed to market for new features and changes. But there are also many organisations that have been disappointed in their Pact implementations. Let's go through some of the reasons why Pact implementations fail to deliver on their promised benefits, and how you can avoid these pitfalls.
Lack of buy in
The number one reason that Pact implementations fail is a lack of buy in from all the parties that will be using Pact. Using Pact requires collaboration and commitment from each consumer and provider team. If only one team for an integration is invested in the process, then it going to be very unlikely to succeed.
What can you do to avoid this?
Good communication is the key to success in almost everything, and Pact is no different! Clearly outline the goals, the benefits and the costs of implementing Pact to your teams. Watch some of the many online presentations together. Do a proof of concept, and share it around.
Lack of education
Understanding how to write good Pact tests can make the difference between Pact tests that are fast, lightweight and helpful, and Pact tests that make you wish you'd stuck with integration testing. It is easy to write brittle, burdensome Pact tests if you think that there is no more to contract testing than understanding the syntax of the Pact library. Contract testing is a new paradigm to most people, and getting your head around what to test, how to test it, and just as importantly, what not to test, takes time and effort.
Using Pact also requires:
- strong development skills
- a good understanding of how APIs work under the hood
- understanding how to write, execute and debug unit tests
- understanding how to use CI/CD software and build release pipelines
What can you do to avoid this?
Take the time to educate all the developers and testers who are going to be using Pact, so they understand the contract testing tools and mindset. Make the most of the documentation at docs.pact.io. Read the best practice guides. Do some workshops together. Join the Pact Slack community and ask people what pitfalls they've experienced and how to avoid them.
The people paying the cost don't get to experience the benefits
When we say "paying the cost here" here, we mean: the people who are responsible for the project budget; but also the people who are doing the work of integrating Pact into the CI/CD pipeline, writing the tests and dealing with the verification failures. These people are typically the developers, although sometimes testers take on this responsibility. A valuable reward for team members who are investing time in Pact is improved "flow", achieved by faster tests, fewer bugs and fewer dependencies, leading to shorter release cycles, and an increased sense of satisfaction in their work. A valuable reward for a project manager would be seeing a reduction in project cost or release cycle length.
Unfortunately, the team leaders or members paying the cost may not get to experience the satisfaction. This might happen for one of two reasons.
Firstly, the people paying the cost may be different to the people experiencing the benefits. For example, if the developers are already responsible for the maintenance of automated end-to-end tests, then (after the initial overhead of set up) ongoing maintenance and execution time of the end-to-end tests should be less, increasing the aforementioned "flow" that developers crave. Any developer who has spent large blocks of time fighting with flakey browser based end-to-end tests would relish being able to delete some of them in favour of faster running, reliable contract tests.
If, however, the responsibility for integration testing lies solely with the testing team, then developers may only experience the increase in work associated with the contract tests, without experiencing the benefits themselves.
Secondly, there may be other constraints that stop the full benefits of the contract tests from being experienced. For example, an application with multiple integrations may only have a contract with one of them, meaning that all the other integrations still need to be tested using traditional integration tests.
What can you do to avoid this?
See if you can couple the "cost" to the "reward" by getting the people who are experiencing the pain of the current status quo to work on the Pact implementation. If the current "sufferers" are the testers, but they are not familiar enough with the codebase to be able to implement Pact themselves, this would be a good opportunity for developers and testers to pair together on the task. Make sure the developers understand the benefits to themselves, and to the overall project.
When you are deciding whether or not to use Pact for a particular integration point, think through all the pressures that are currently being placed on the release cycles of the services involved, and be fairly confident before you start that Pact is going to provide benefit for that particular integration.
The value it provides is not perceived as valuable
The value of using consumer driven contracts lies in ensuring that changes made to the provider do not break existing consumers. However, some people do not actually think that provider changes breaking internal consumers is a problem. They take the view that it is the responsibility of the consumer teams to ensure that they are up to date with the latest version of the provider API. In this situation, failing contract tests are seen as an impediment to their workflow.
What can you do to avoid this?
If someone thinks that a provider should be able to make backwards incompatible changes, then it's probably better to work on changing that mindset first before trying to implement Pact! Organisations where providers are allowed/expected to make breaking changes are often associated with long release cycles, as the mitigation for this approach is to have an extensive system integration testing phase. Contract testing is only one part of a more agile (to use an overused term!) approach to software development that puts a strong emphasis on being able to respond to change quickly.
Conclusion
Hopefully this information can help you avoid all these pitfalls, and successfully use Pact to help ship your code faster. If you've experienced any of these, or have any you'd like to add, come and chat to us on the Pact Slack workspace!
