Verification Results

As enterprises begin to scale their microservices across their business, for a variety of reasons not all teams may have access to each others CI systems. This can be problematic when using a consumer-driven contract approach, as when a consumer publishes a contract that breaks a providers' build, they may not be able to see why the failure has occurred - was it a breaking change to the contract that was an accident? Did we use a state that didn't yet exist or was data not populateed as expected? Or did the build fail for other unrelated reasons?

Even with Webhooks, there may not be a good automated feedback loop in place to deal with this situation. Until now.

Today, we're announcing our Verification Results interface, that will help teams quickly identify the interactions that are causing test failures for a provider build and get them rectified.

Verifications background

A while back, we introduced Verifications into our open source tooling and the Open Source Broker, to help create better feedback loops and automate some of the most painful aspects of a release - checking if one or more components was compatible with another. This was the birth of one of our most powerful features, the "can-i-deploy" tool.

The basic mechanism is this: when a pact is verified against a provider, the outcome of that verification (pass or fail) needs to be made available to the consumer and provider teams, so they know whether or not the code in either project can be safely deployed.

Read more about verification results.

With this in place we were able to show, at a glance, which integrations and which contracts were verified, unverified or had failed and therefore were unsafe to release:

Introducing Verification Results interface

Whilst this at-a-glance view is useful to make the decision as to whether or not it is "safe" to release, it doesn't help you get to the bottom of why it's not safe. Until today.

Navigating to the Pact view for a broken contract, you'll now be able to see inline, any interactions that failed and a visualisation of the cause of that failure:

In the example above, each item in the response body was missing the required first_name property.

As part of this uplift, we've also made a few UX tweaks to the Pact view.

1. We've broken down at an interaction level, whether or not it was successfully verified or not
2. We've removed the popups and now display the pacts in the same tab you were working in.
3. For each interaction, we've modified the way we highlight request and response objects to make it clearer, by giving them their own section boundaries and better syntax highlighting

Available Now

Verifications are available now and you can start using them today. All of our OSS tooling is capable of publishing this information to Pactflow. If you've already been using this feature with Pactflow, you'll be able to see past results immediately. If not, you're missing out!

In most libraries, it's just a matter of setting a single property or environment variable. For example, setting the publishVerificationResult property to true in Pact JS is all that is required:

describe("Pact Verification example", () => {
  it("validates the expectations of Matching Service", () => {
    const opts = {
      provider: "Animal Profile Service",
      providerBaseUrl: "http://localhost:8081",
      pactBrokerUrl: "https://<yourbroker>.pact.dius.com.au/",
      pactBrokerToken: "123..............xyz",
      tags: ["prod"],
      publishVerificationResult: true, // <- enable this in CI
      providerVersion: "1.0.0", 
    }

    return new Verifier(opts).verifyProvider().then(output => {
      console.log("Pact Verification Complete!")
      console.log(output)
    })
  })
})

Got feedback for us? We'd love to hear from you. Simply drop us a line at hello@pactflow.io or follow us on Twitter