Saturday, April 17, 2010

The importance of contract-first development

In enterprise IT projects, software services are typically consumed by other teams or even organisations than those who programmed those services. For example, a Billing system can be instructed by a User Profile system to allocate billing details for a new user profile. Or a Web Frontend system can ask a Session Management system for the data pertaining to a certain session ID.

This is typical service-orientation.

Expectation management
A challenge in such a service-oriented approach is making sure that all involved parties – and that can be or become a lot more than just 2 – have the same expectations from the service. In my experience this is the number one source of frustrations and issues.

Before a service is built or extended, the parties involved at that point in time typically reach agreement what the new functionality should do and how it should be accessed. It is then quite common to have those teams work in parallel, because time-to-market is key.

But even if the teams do not work in parallel, when the service provider delivers functionality for testing and integration, expectations from the other parties can be very hard to meet. For example, service consumers may have expected different formats or may have expected necessary transformations would be done by the service provider, while the service provider is pushing the requirement to the callers.

When this happens, project deadlines and stress levels are at stake.

And the more consumers and the more time it takes for the service provider to deliver a first version, the higher the risk.

Working with contracts
The common solution to this is to work with a contract that defines the agreements between the provider(s) and the consumer(s). Such contracts vary from informal (e.g. Word-documents lacking strict interface definitions) to formal (WSDL with very strict constraints).

Such contracts can be written at the start or they can be generated from existing code.

Generating a contract from existing code is generally a bad idea. The contract is then focused on the implementation, which is subject to change. Then if the implementation changes, what happens to the interface? If it should also change, this violates the common principle of Separation of Concerns. And it is clearly the wrong way around, the contract is the agreement, not the implementation chosen by the provider.

Another issue with contract generation is that the contract can be very much focused on the implementation, including specific data types tailored towards an implementation within a certain platform or programming language. This has an impact on the callers and, in the long term, also on the consumer, since the latter may be forced to stick with legacy implementation choices.

An implementation-independent contract
A first step towards a complete solution to this challenge is to author a contract that is implementation-independent. Such a contract can be agreed upon from the start by all involved parties. Real separation of concerns: the contract is the common agreement, while providers and consumers are free to make their own choices within the bounds of the contract.

This may appear a complete solution.

However, in practice, contracts change, possibly even during the initial implementation project. So what should be done in such a case? Is the contract still leading or will the contract by bypassed by reality?

Based on my experience in professional environments I dare say that if the contract can be violated, it will be. With impact on stability/reliability and on stress levels. In the battle between contract-orientation and project deadlines, the former is easily sacrificed.

Contract-first development
The only way to really guarantee that you can keep the promise of an independent contract is to integrate your contract-orientation in your development process, making the implementation(s) dependant on the contract, preferrably on both sides of the contract (providers as well as consumers).

One way to achiebe this is to make your code link to the contract somehow, for example by having a URL to your WSDL files in all implementations, making sure that failures to adhere to the contract result in errors.

Validation at runtime
To avoid the contract being bypassed, implementations should check all service invocations to make sure the contract is respected.

This can be achieved by generating server- and client-side code from the contract. Having access to the contract (in some form) the implementation then checks all aspects of the invocation. Not only the  preconditions (consumer requests), but also postconditions (provider responses). If either a consumer or a provider violates to the contract, that must be treated as a fatal violation. Only then your contract will actually be leading.

Validation on the server versus on the client
So if you want to validate your contract, where should you do it: on the client (consumer) or on the server (provider)?

The best approach to this is to do it in both locations, for good reasons:
  1. Separation of concerns: both ends should assume their responsibility, which is to adhere to the contract and consider violations fatal. Neither end can be sure the other end will validate the contract, so they should not assume that.
  2. The provider and the consumer may have access to different versions of the contract. Validation on both ends will likely catch any incompatible differences in the contract versions.
Your contract is your treasure!
Now when you have accomplished a true contract-oriented development process, you will find the contract is your gold mine. The contract is typically orders of magnitude simpler than implementations, making it easy to read and change. But since it describes an extremely important integration point so well (concise) and so correctly, it is easy to reuse it for all sorts of of purposes. For example:
  • generation of documentation, e.g. Word-documents or web pages, etc.
  • generation of forms or programs for manual use (including human testing)
  • generation of unit-, performance- and load-tests
  • generation of IDE projects, such as Visual Studio, Eclipse, NetBeans, XCode, etc.
To enable certain possibilities, the formal part of the contract needs an additional informal part. The informal part can include examples and human-readable descriptions. They can also help humans understand the contract and its purpose faster and more easily.

Examples of contract-orientation
While WSDL is an example of a contract definition language for web services, it does not provides a full contract-orientation process, requiring the contract to be held.

The only fully contract-oriented projects I am aware of are those I initiated: XINS and Logdoc. The first (XINS) is an open-source web services framework (that generates server- and client-side Java code), while the latter (Logdoc) implements contracts for logging, again including a Java-implementation.

Do you have experience with contract-orientation? Do you know of other projects that put contract-orientation in the heart of their approach? Please let me know in the comments!

1 comment:

  1. As you correctly describe in your post: defining a good interface is one. However managing the realised service is another. I got great value from Todd Biskes' book. May be it is worthwhile for you to read:

    http://www.amazon.com/SOA-Governance-Todd-Biske/dp/1847195865/ref=sr_1_1?ie=UTF8&s=books&qid=1271879149&sr=1-1

    ReplyDelete