In April 2022, SmartBear acquired PactFlow, adding contract testing to our API platform. This meant different things for our company, but for me it meant diving head-first into the contract testing methodology and the Pact framework. This was so I could become fluent in explaining it to software development teams whose only experience with Pact might be a blog post or hearing it mentioned in conversation. Simple, right?
Thankfully, there were plenty of workshops and articles to help me learn the ropes. Once I had a solid understanding of the fundamentals, it was time to start talking to teams interested in the technology, then help them figure out if contract testing could solve some of their software delivery challenges. With over a year’s worth of conversations, I’d seen many success stories, but I also saw plenty of teams and companies who couldn’t fully unlock the value of contract testing. In the latter scenario, it’s usually down to one or more of the following things:
1. The return on investment wasn’t clear enough
When I’m supporting a team with their contract testing proof of concept (POC), the first thing I ask them to do is review their integration test suites. The primary value of contract tests comes from the fact that, in the long run, they’re cheaper to run and maintain than traditional end-to-end integration tests. Ahead of a POC, I’ll ask teams to think about how much time they currently spend on things like:
- Building and maintaining test environments
- Writing and executing integration tests
- Debugging integration tests failures
- Fixing flakey integration tests
This will give the team and the wider organization a clear idea of the value of contract tests, specifically how they save development and compute time while providing faster feedback. Once these metrics are well-defined and understood, it’s easier to communicate to other teams the return on investment contract testing can provide.
Ask one of the friendly SmartBear sales team for your own PactFlow POC template – a customizable tool to help you set out your POC goals in advance to convey the return on investment.
2. The purpose of contract tests wasn’t defined well enough
I’ve lost count of the number of times I’ve said it over the last year; contract tests aren’t functional tests. The best analogy I’ve heard so far goes like this: Â
Let’s say I walk up to you on the street, hand you an envelope and ask you to post it for me. A contract test would verify you understood my request, but wouldn’t verify you  post the letter, the letter reaches its destination, or that I get a reply, etc.
Those things (we could call them system side effects) would be better verified by using functional tests. Ideally, contract tests focus on the messages flowing between a consumer and provider and ignore other elements of the system. A misunderstanding of the purpose of contract tests can lead to frustration amongst the developers and testers who are tasked with writing and maintaining them. Â
This blog demonstrates the purpose of contract testing in the context of the testing pyramid conveying the difference between unit, integration, E2E and contract tests.
3. The scope of your contract tests was too wide
The best examples of contract tests will focus on the data access layer of the consumer and, even more specifically, the part of that layer we would consider the API client. If I’m working with a team that isn't immediately sure what part of their code this is, I ask them the following question: Â
What part of your code is responsible for translating your application’s business domain objects and data into HTTP requests getting sent to the provider?
That’s where contract tests are going to be most effective. A good contract test shouldn’t be interested in the consumer’s business logic or the provider’s business logic - that would still be the realm of the respective teams’ functional tests. The risk here is, once again, teams get frustrated with brittle and difficult to maintain contract tests which, if you remember from earlier, are exactly what we’re trying to avoid.
4. There wasn’t enough communication between consumers and providers
A less technical, but no less important, reason I see organizations fail to maximize the benefits of contract testing is due to poor communication between the teams involved. One benefit of contract testing is you can decouple teams that otherwise would depend on (and potentially be blocked by) each other’s work. However, contract testing isn’t a substitute for open and easy communication between different development teams within an organization.
For example, I’ve had conversations with provider teams who are eager to implement contract testing with Pact and PactFlow but who don’t have lines of communication open with the consumer(s) of their application. Worse, they can’t tell me who these consumers might be. It takes two to tango, and in the world of contract testing, both sides of an integration need to be sold on contract testing for it to provide value to the organization.
Having a shared understanding of the value means consumers and providers need to be able to communicate and collaborate effectively. This can also result in a lack of buy-in and integration into your organization’s engineering culture. A lot of the companies I work with see pockets of success but struggle to unlock the next level of value because contract testing isn’t truly part of their ways of working.
Some ways to facilitate this are setting up communities of practice and internal templates/educational materials tailored to your organization’s tech stack and business domains. Without these mechanisms, you’ll struggle to embed contract testing at your company.
5. You didn’t know about the Pact Foundation Slack workspace
Finally, a shameless plug. PactFlow wouldn’t exist without the work of the community behind Pact. The Pact Foundation Slack workspace is filled with nearly 5,000 practitioners who are hands-on with the various implementations of Pact and the related technologies (such as the open-source Pact Broker and PactFlow.) The invite link to this workspace is one of the first things I’ll send to a team embarking on a contract testing POC, because it’s the best place to look to get answers to implementation-related questions from people who have done it before and seen positive results.
So, you’ve read some articles, watched some videos, talked to colleagues who’ve done it before and you’re ready to find out if contract testing could save time and money for your team. Keep these five points in mind, and you’ll be in Pact Nirvana in no time!
Getting started:
- Create a free PactFlow account to spin up your POC in no time
- Complete the contract testing courses in the SmartBear Academy and get your contract testing certification
