Comment to "Versioning Services"

Since spammers have discovered blog comments as a way to get to you, I disabled them.

Honestly, spammers should be summarily and ceremoniously executed.

Anyway, my colleague Ade Oshineye sent me this comment:

Have you seen this article: http://martinfowler.com/articles/consumerDrivenContracts.html where Ian Robinson talks about the gritty issues involved in versioning services?

Sure. I am not dismissing the complexities of versioning mature services. The point of this series of posts is to get people to stop worrying about details like these until they have obtained a firm grasp of what it is like (and what it means) to build applications with services.

In true Agile sense, plan and build what you understand. Analyze and design what you can't get your head around yet later.


It is this engineering habit of wanting to have ALL the answers to ALL the questions that is killing efforts to change the way we do things in software development.
That's why Agile/XP is such a beautiful thing, because it does away with that notion and accepts that you CAN'T know all the questions up front, let alone have all the answers.

Many times have I come across budding SOA efforts that were mired in pointless analysis and design. "But we have to have our message headers defined!".

NO, NO, NO! What does a full definition of message headers teach you about making SO applications? Nothing! Where's the learning? Where's the difference? Where's the business value?

The point is to learn how to do things differently and discover how you and your team can benefit from it. The many issues that undoubtedly will arise are there to be SOLVED, not avoided. If early man had this attitude of wanting to have all the problems solved before embarking on something new, we'd still be living in caves, without fire, mind you.

What does the sentence: "With Web services, commercial registries can handle much of the versioning" actually mean? Are you suggesting that using a commercial registry to locate business-critical services is a viable approach? Even though the first I'll know about a (possibly incompatible) service upgrade is when the registry notifies me? At this point the upgrade has already happened which is far too late to do anything about it.

Commercial registries like Systinet have the concept of service release stages. For this you need to install more than one registry and add the newly versioned service to the non-production registry first. This very action will notify interested parties that a new version of a service is pending, so they can take action if required.

There are several sound strategies for introducing a new version of a service. One is to leave the previous version intact while the new one comes on-line and set a phase-out schedule.

Of course, this sort of approach requires that all parties have a means of communicating. And here now is the crux of a successful SOA implementation: it's the PEOPLE:

As per the Agile Alliance: (http://www.agilealliance.org/intro)

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

As you can see from Ian's article it's not enough to just append extra content on the end given that most of the tools people use to interact with web services insist on validating against the entire schema.

Sure, if you validate against an entire schema, you are shackling your consumer to a service, i.e. coupling it tightly. While schema validation has its place, if you are in an environment where message formats change frequently (e.g. in options trading) then that coupling is undesirable.

I'm not pro or con validation, I'm just suggesting to use common sense.

Moreover putting version numbers on each message sent is not in itself an acceptable versioning strategy unless we know that our consumers can actually handle the new version. For instance in an investment banking context if there were a service that published newly booked trades it would be unacceptable to silently upgrade that service. If business critical applications suddenly stopped working because of a silent upgrade it would not be enough for the service provider to point to the error queues full of messages in the new format. Each upgrade has to be verified against each consumer in some way because in practice the levels of coupling of between service provider and consumer are still just as high in an SOA. The difference is that the coupling is often implicit and only exists at runtime. This creates situations where there appears to be low-coupling at the architectural level but support staff have to manually resolve the high levels of coupling that exist at runtime in the production environment. This usually ends up taking the form of increased communication between the humans who support the service providers and consumers e.g. co-ordinated releases, staging areas, sandboxes for integration testing by replaying messages from the live system.

Again, use common sense.

• Introduce a new version of a service in a staging environment
  
• Release it to production on a different URL or topic, if you can
  
• If new data is added to a message, put it somewhere so that it doesn't interfere with existing consumers
• ...and version numbers in messages DO work. As a consumer of somebody else's service, I know that I'm beholden to the format it sends its data. So if version numbers are available, it would behoove me to build my consumer to check for that and do something useful with it
  

Indeed, until we can write software that can figure out what to do with new data all by itself, we have to have proper human communication. This need not be heavy-weight and procedure-laden. But every time one human provides something to another human, the provider has to choose between either improving it and coordinating with the consumer, or leave it stagnant and risk the consumer walking away.

If you feel I'm being too light-hearted and cavalier about this, maybe you oughtta try that yourself. It's quite liberating, and very effective.