Reading:
Schemas can be contracts

Schemas can be contracts

Matt Fellows
Updated
Schemas can be contracts
With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.
Hyrum Wright
https://xkcd.com/1172/

If you've worked with APIs long enough, you've lived this. You changed something that wasn't in the contract — an optional field that appeared when it usually didn't, an undocumented header quietly removed — and somewhere, silently, a consumer broke. Not because you violated the spec. Because they'd built on the gap between the spec and reality.

This is the thing schemas have never been able to protect you from.

Schemas are not contracts.

A schema tells you the shape of your data. It validates structure at a point in time. What it cannot do is verify that your running service continuously honours its own spec as the codebase evolves, or give you any visibility into what your consumers are silently depending on.

Put another way: your provider contract is not your API. It's a map — it's not the terrain.

Contract testing shines a light in the darkness of the gaps between specification and real-world usage. But to understand why, we need to look at exactly where those gaps appear, and why they're harder to close than they might seem.

A gap with two sides

The gap between schema validation and contract testing isn't just a tooling inconvenience, it's a fundamental problem with two distinct faces.

On the provider side, there are actually two classes of problem.

The first is straightforward: your API drifts from its spec, even with the best intentions. Teams that generate specs from code can end up with incorrect definitions if annotations fall out of date, a genuine issue we've heard from customers, including a large security organisation who discovered their generated spec no longer matched their implementation. Sometimes there are gaps in spec generators that require manual configuration or intervention. Sometimes there are simply bugs in the wiring between code layers that only surface during testing. In every case, your schema still passes. Your tests are green. But your spec is now a fiction.

The second class of problem is subtler and harder to solve: certain API behaviours simply can't be fully expressed in a schema. Polymorphic payloads ( anyOf, oneOf, and the like) are a good example. Under which conditions can a consumer expect a particular shape in the response? OpenAPI doesn't specify the preconditions for these behaviours. Nor does it fully capture the semantics of optional fields, enum values in context, or ordering guarantees. The map has edges. Beyond them, there be dragons.

On the consumer side, your consumers are navigating that incomplete map and making assumptions about how it can be used. They build on undocumented behaviours. They treat optional fields, or polymorphic variants, as if they're guaranteed under certain conditions. They rely on ordering you never promised. Hyrum's Law in practice: the observable behaviour of your system becomes the real contract, whether you intended it or not.

This is exactly why we've long argued that specification testing and consumer contracts are both necessary and complementary. Specification testing gives providers confidence that their docs are a reliable, accurate map of the API. Consumer contracts illuminate the subtle behaviours the map can't express: the real, observed usage patterns that no schema alone will ever capture.

And here's where it gets practical: teams aren't just doing schema validation, they're already doing functional API testing too. The challenge is that they don't see why they should also adopt contract testing on top of what they're already doing. More tools. More workflows. More effort. Our answer to that is simple: you shouldn't have to choose. Done right, spec-driven testing and contract testing shouldn't be parallel tracks. They should be one workflow. We'll come back to that.

Building the bridge with bi-directional contract testing

Consumer-driven contract testing — the idea that consumers should define and publish their expectations — has been an answer to the consumer-side problem for over a decade. Pact pioneered the principles of making consumer behaviour explicit, verifiable, and visible to the providers who need to honour it.

But adoption by some teams was slow, not because of the idea, but because of the cost. Teams had already invested in OpenAPI. They had schemas, documentation, existing test suites. Starting over with a new paradigm wasn't always viable.

Bi-Directional Contract Testing (BDCT) was the bridge. It let teams use their existing OpenAPI specs and existing test tools as the foundation for provider verification, dramatically lowering the barrier to entry. Teams using BDCT report more than 50% reduction in the time needed to create contract tests, and over 40% of our customers have now adopted it.

But one gap remained. BDCT could tell you whether your spec was compatible with consumer expectations. It couldn't verify that your running service actually matched the spec it was being compared against. The provider side still had no enforcer and finding one wasn't easy.

Teams had to source their own solutions. Some already had tools in place. Others asked us what we recommended. For years, Dredd was the go-to open source answer, but it's now officially archived and no longer maintained, leaving a real gap in the ecosystem. Alternatives all came with trade-offs: Postman requires custom schema validation scripts added to each and every request ( pm.response.to.have.jsonSchema(...) ), introducing manual effort and additional maintenance. Language-specific tools like RestAssured work well within a single team's stack but fracture as organisations grow. What teams actually need is a single solution they can adopt, educate around, and scale consistently across teams and technologies.

Introducing Drift

Today, we're excited to introduce Drift — spec-driven API testing for the provider side.

Drift runs deterministic, automated checks to verify whether your API implementation actually conforms to its OpenAPI definition.

Here's what that looks like in practice:

  • Spec-driven: your OpenAPI definition is the source of truth — and all schema tests are derived directly from it
  • Plug-and-play CLI: download and run in seconds, locally or in any CI pipeline
  • Declarative and version-controlled: deterministic test suites that live alongside your code, not in a separate tool
  • Extensible: customise behaviour using scripting, or extend capabilities through the plugin architecture
  • AI-powered: automatically generate a full test suite from your spec in minutes, and use AI to maintain and fix tests as your API evolves

Used standalone, Drift is a fast, lightweight way to catch provider drift before it reaches your consumers.

Used alongside PactFlow's BDCT workflow, it becomes something more powerful. Drift enforces the provider spec against real, running behaviour. PactFlow surfaces what consumers actually depend on. And PactFlow's can-i-deploy capability gates your deployments, ensuring that no change goes out that would break a known consumer. All at enterprise scale.

Together, both sides of the contract are held to account simultaneously, unifying spec-validation an contract testing into a single workflow, allowing teams to evolve both provider and consumer systems safely over time, with confidence.

Drift in action

Why this matters more than ever

The agentic AI era isn't on the horizon. It's here.

AI systems are generating APIs, writing specs, and consuming endpoints as autonomous tools at machine speed, with no human in the loop to notice when something quietly diverges. An LLM can produce a complete OpenAPI document in seconds. But without enforcement, that spec is just text. A statement of intent with no mechanism to verify it.

In a human-paced development cycle, drift gets caught in a code review or a QA cycle. In a machine-paced cycle, it gets caught in production by a customer, or by an AI agent behaving in ways nobody anticipated.

The volume and velocity of API creation now demands quality tooling that is automated, continuous, and grounded in the spec you already have. This is what SmartBear means by Application Integrity — in the AI era, software must demonstrably do what it says. Not just document it. Prove it, automatically, at every stage of the lifecycle.

Drift is that evidence layer at the API boundary.

Schemas can be contracts

Schemas cannot become contracts in and of themselves. But we now have the tooling to make them a meaningful part of a contract testing solution.

A contract requires two things: a shared agreement, and a mechanism to enforce it from both sides. PactFlow combined with Drift ensures the map matches the terrain.

For the first time, the full picture is available in one workflow. Design your API. Verify it conforms. Know what your consumers depend on. Deploy with confidence.

Drift is the keystone of a new standard for API integrity built for the agentic API lifecycle, where APIs are created faster, consumed by machines, and trusted at a scale that demands more than documentation. Whatever the technology, whatever the protocol, PactFlow and Drift will be there to ensure every API is high quality, trustworthy, and safe to release.

Drift is available now, to all PactFlow and Swagger Contract Testing customers.

Try Drift today →

arrow-up icon