SOA Governance Evaluation Guidelines

Original publication date: Oct 29, 2007

A pragmatic approach to assess and compare alternative solutions

Today I will try to outline the general principles which might be useful when selecting or architecting a comprehensive SOA Governance Solution. They have helped my team in the past and hopefully could be of some use to others as well.

Depth and Scope

By definition Service Oriented Architecture represents an array of business and technical strategies that the IT organization of an enterprise pursues to promote the exposure of business functionality within and between enterprises in a consistent, published, secure and contractual way. Yet most “SOA enabler” products including Governance Solutions are centered around Web Services, WSDL and UDDI – none of which are an intrinsic part of SOA. The reason for this is that due to its recent popularity, SOA has become a must have on all product spec sheets and architecture proposals, and the vendors are trying to limit the scope of SOA to match their offerings. The following table summarizes the common misconceptions about the substance, scope and depth of SOA and some of its aspects:

SOA ≠ Web Services or Any Other Technology SOA ≠ Integration or OOP or MOM Service Registry ≠ UDDI Service Description ≠ WSDL Service Monitoring and Control ≠ JMX Service Management ≠ Component Deployment Service Versioning ≠ Repository Source Control Service Security ≠ SSL or XML Firewall or Security Appliance Service Accountability ≠ System Logs

Service QOS ≠ HA Solutions Service Metering, Chargeback ≠ Usage Logs Service Routing ≠ BPEL or MOM or Java Code Service Interoperability ≠ HTTP and SOAP Service Composability, Orchestration ≠ Published WSDL Service Discoverability, Dynamic Binding ≠ UDDI Service Fault Handling & Analysis ≠ Java Exception Stack or WSDL Fault or System Logs Service Throttling ≠ Firewall or JMS or Application Server Configuration

When considering investment in a SOA Governance Solution, it is important to make sure that prospective vendor or architecture team have not artificially narrowed the scope and thus future usability by repeating any of these mistakes.

Solution Drivers

The following drivers should be decisive in evaluating the architecture and functionality of a governance solution:

1. Does it define a comprehensive Governance Information Model (GIM) which contains information required by all stakeholders and actors in the SOA landscape? Such service model should be radically different form the metadata-based approaches to SOA Governance adopted by the majority of existing solutions: it should define a strongly typed, easily validated view of the governance domain with enforced referential integrity, while the latter simply adorns the service WSDL. The SOA service model should be extendible with metadata to allow users capture additional service attributes, such as geography or pricing, which are pertinent to their business model, but the model itself should be defined as the first-rate data.
2. Does it separate the views of a service from the Technical, Business and Governance domains and allowing them to be expressed and maintained independently?
3. Does it provide support for all core SOA characteristics as well as any additional ones required by the enterprise without forcing the users to implement the characteristics that have no current business value in their domain?
4. Does it maintain complete separation between service implementations and service aspects implementing the SOA characteristics, allowing incremental adoption and rollout of governance solutions?
5. Does it facilitate the support of all existing and future SOA-related standards without forcing users to lock into any particular standards or profiles? Does it allow users to switch between competing standards or between proprietary and standard aspect implementations? In other words: can it be used as hedging against uncertainty in the SOA standards space?
6. Does it support packaging policies into reusable Governance Contracts to prevent uncontrollable policy application and proliferation? Does it help to prevent unnecessary service proliferation by exposing the same service implementation to different constituencies with different contracts?
7. Does it facilitate true service reusability through direct support for service virtualization and refinement? Service Refinement refers to the ability to slightly change the interface, vocabulary, granularity, semantics or behavior of enterprise services without changing the service implementation or affecting existing service consumers.
8. Does it provide complete support for the Web Services technology stack (HTTP, SOAP, WSDL, UDDI), while allowing service producers and consumers to use the technologies of their choice when implementing and consuming enterprise services? Does it minimally support JMS, EJB and Java service bindings, and allow users to define and implement new binding types.
9. Is it Aspect-based? Does it provide SOA adopters with the means to support all aspects of service governance without forcing them to accept any arbitrary decisions on which governance aspects they should implement or how these aspects should be implemented. Does it come out of the box with a reasonable governance model and a set of aspect implementations providing a useable starting point for recent adopters of SOA? At the same time is it equally applicable for advanced service-oriented enterprises with established governance model, standards and toolsets?
10. Does it support aspect delegation: many of the service governance aspects including: security, versioning, monitoring and management, etc.; have existing packaged solutions broadly adopted in the prospective SOA marketplace? The examples of such solutions are Sun’s Access and Identity Managers and CA’s eTrust for security, IBM’s Tivoli and CA’s Unicenter for monitoring and management, CVS and ClearCase for version control. Does it allow users who have already adopted a packaged solution in one or more aspect spaces to delegate the implementation of these aspects to those solutions, provided that the solutions themselves support such delegation? At the same time it should not rely on any external packaged solutions, providing independent implementations for all core aspects.
11. Are the services exposed through the solution consumable without any custom client code? Is it compatible with industry-standard tools like SoapScope? There is a contention between the need for interoperability and decoupling versus convenience and the need to hide complexities. Can the solution resolve this contention by offering service consumers a spectrum of options for invoking governed services? These options should include:
    • Access points, which allow service invocation without any client-side components, but require client implementation to manage the complexities of dealing with find-bind-execute cycle and compliance with the governance policies.
    • Client interfaces, which provide service-generic but client platform-specific solution components that insulate consumers from most of the complexities when invoking any exposed service.
    • Convenience APIs, which provide both service- and client platform-specific layer for zero-effort invocation of designated services.
12. Does it manage the cost of compliance? To be able to satisfy all of the above drivers would require a solution of significant level of complexity - does it compromise overall viability of the SOA implementations it is designed to support? Specifically:
    • Does it introduce performance bottlenecks to underlying service implementations?
    • Does it introduce scalability bottlenecks to underlying service implementations?
    • Does it introduce a single point of failure or negatively affect the availability of underlying service implementations?
    • Does it introduce any additional vulnerabilities or negatively affect the security of underlying service implementations?
    • Does it negatively affect the testability of underlying service implementations?

Brownfield-friendly SOA Governance

Original publication date: Nov 07, 2007

The time is now!

The real life usage of such governance solutions is evolving even compared to the last year. Until recently most companies that introduced SOA Governance did so in some form of pilot project, which usually represented a "greenfield" environment: service consumers, composite applications and often the services themselves were being developed at the same time. So the key factors on which SOA Governance solutions were evaluated were centered around their design- and run-time capabilities and operational characteristics but did not include the "brownfield" environment friendliness factor. Consequently SOA Governance vendors had little incentive to invest in those capabilities of their products. But all of this is about to change, and the ability to support effective and painless introduction into existing IT environments will soon become one of the key differentiators in the SOA Governance marketplace. We have experienced this first hand during a recent implementation of Sun Service Governance Framework (SGF) at a large European media company, during which it turned out that the majority of issues with development, implementation and rollout of the governance solution were directly related to the "brownfield" category. Specifically they included:

1. The need to reconcile and integrate Service Governance with the SDLC used by the client’s IT organization.
2. Ability to support service governance across multiple development, testing, staging and production environments.
3. The need to provide support for “uncooperative” clients – the ones which are impossible or not feasible to change to accommodate governance-related service changes.
4. The need to quickly and efficiently bring large numbers of existing services under the control of the Governance Solution.
5. The need to support safe and effective sharing and migration of governance data between multiple environments.

It was estimated that without the above capabilities, the total effort required to introduce the Governance solution into the IT SOA landscape would exceed half of a man-year.

State of the union

I have not seen any full-spectrum SOA Governance products or technologies that provide noteworthy "brownfield" environment friendly capabilities described above. There are number of design-time only governance products (Registries) that provide federation capabilities which could be utilized to support some form of migration and reconciliation of governance data across multiple environments. There are also some run-time only governance products which allow import and export of governance data and can be used to ease some of the pains of reconciling Governance with the SDLC. But that’s about it! Existing governance methods and solutions are focused on governing services in the context of established SOA environment complete with underlying governance infrastructure. Let me bring an example: WebMethods in their definitive whitepaper on the subject write: "Ensure that governance capability-related milestones are synchronized with SOA adoption milestones so that you do not end up trying to retro-fit governance after the fact" and "the right time for governance is before you put any services into place" - great advice if you only deal with clients that have never played with SOA before! This situation is most representative for "greenfield" environment and is highly atypical for real-life enterprise IT. This static nature of service governance can become a significant barrier for its introduction (and consequently the success of SOA overall) in "brownfield" environments with its existing sets of services, legacy consumers, third-party composite applications and established software development processes and practices.

The Answer

When we first recognized this problem (and the shortcomings of our own governance solution) we set out to define the list of capabilities and enhancements to SGF that would solve the challenge of [near] painless introduction of SOA Governance into existing SOA-based IT environments. This is what we end up with:

1. Staging-aware SOA Governance which aims to resolve the disconnect between the fact that governance is essentially an oversight activity and thus should happen in (or at least as close as possible to) production with the need to put governance artifacts through the same QA processes as the rest of the IT assets.
2. SDLC support in Governance which addresses the fact that transition from so called monolithic or siloed applications to SOA has in fact, from the SDLC point of view, made the entire IT environment even more monolithic than it was before that transition. In the past at least these applications were independent form one another and could have been taken through SDLC phases one-at-a-time. As companies embrace SOA they are facing potentially infinitely connected mesh of services, consumers and composite applications and the only guaranteed safe option becomes to take through SDLC the entire snapshot of enterprise IT, making it more difficult and costly then ever to introduce new changes required by the business. Extending Governance solution with SDLC capabilities makes it possible to take individual services, consumers and entire composite applications through the lifecycle stages as required by the IT practices and procedures.
3. Transparent Governance Mode which resolves the tension between the need for a Governance platform to transform services and the need to support legacy clients that can not (easily) change to accommodate governance-related changes to service interfaces. For example declaring that a certain service has to be authenticated with WS-Security requires changing the WSDL to reflect the fact that it now needs wsse-compliant header.
4. Bulk operations which would allow to quickly and consistently bring under the umbrella of governance groups of existing services, spread throughout the Enterprise.

I believe that brownfield-friendliness will be a decisive differentiator amongst the SOA Governance products in the coming year so I am planning to talk about each of these capabilities in more detail in future posts.

SOA Governance Scorecard

I stumbled on an unanswered question how can we evaluate a governance solution on all the factors given by you? for the SOA Governance Evaluation Guidelines article on my old blog that somehow survived my separation from Sun few years ago and its subsequent swallowing by Oracle. In fact, I have been getting a lot of inquiries lately about the stuff described in that blog. And, although I have not been focusing on SOA Governance and Security for the past two years, every time I do cursory research to answer a question I get the impression that things have not changed much since I left that scene.

So, in case Amit still needs the answer to his question, I have dug out the SOA Governance Scorecard which I have defined five years ago to measure and track us against the competition. It uses 62 evaluation criteria organized in three-level taxonomy. Quite a few people have found it useful and a year later Accenture even adopted it almost verbatim as their Accenture SOA Governance Vendor Analysis v 1.0. As completed my archeological excavations, extracted the scorecard from the dig and gave it a gentle cleaning my trusted brush, the emerged criteria seemed surprisingly relevant: the only things I would have changed if I were writing it today would be expand the aspect list in section 2.1 and added a new top-level section on Operational Readiness along the lines of requirements outlined my article on Brownfield-friendly Governance.

A sample scorecard based on this criteria can be downloaded here.

Scorecard Criteria

1. Governance

overall weighted rating for the entire Governance Section.

1.1. Defines a comprehensive SOA service model

overall weighted rating for the Governance Model used by the solution.

1.1.1. Service Metadata

does the solution have the capability to associate metadata with individual services?

1.1.2. Strongly typed model

does the solution define a comprehensive SOA service model which contains information required by all stakeholders and actors in the SOA landscape? Such service model would be radically different form the metadata-based approaches to SOA Governance adopted by the majority of existing solutions: it should define a strongly typed, easily validated view of the governance domain with enforced referential integrity, while the latter simply adorns the service WSDL.

1.1.3. Model validation

does the solution have the specific capability to validate the model and most importantly the conformance of services and other artifacts? Also takes into the account whether the model only allows after-the-fact validation of published services, or through referential integrity, will prevent users from creating invalid entries.

1.1.4. Enforceable Model

does the solution have the specific capability to enforce the governance information captured by the model during all parts of the governance cycle?

1.1.5. Extendable Model

does the solution offer the capability to extend the model to accommodate customer-specific requirements?

1.1.6. Separate consumer and provider views

does the solution differentiates between the governance data which should be disclosed to the consumers and the data which should only be available to the service providers and infrastructure? For example the fact that certain service requires a valid lease token should be made available to both consumer and enforcement agent, while the policy how to deal with missing and expired lease tokens (whether to fail requests, include warnings, and / or raise alerts) should not be disclosed to the consumers.

1.2. Separates Technical, Business and Governance views of a service

does the solution separate the views of a service from the Technical, Business and Governance domains and allowing them to be expressed and maintained independently? A developer should not be concerned with the cost and subscription model of the services they are developing and consuming, and Business Actors should not worry about the bindings, protocols and lease requirements of the services their organization provides and utilizes. Such separation of concerns and un-concerns should be maintained throughout all aspects of a governance solution starting with the model.

1.3. Separates Service Lifecycle from SDLC

does the solution governance offering maintain clear separation between Service Lifecycle (including service definition, validation, approvals, publication, evolution and retirement) and Software Development Lifecycle of the service implementations (coding, testing, QA, staging, production and maintenance)?

1.4. Breadth of Governance Solution

overall weighted rating for the breadth of Governance functionality addressed by the solution.

1.4.1. Run-time Governance

does the solution support run-time governance functions such as service invocation cycle (find-bind-execute), policy enforcement, security, service virtualization, protocol and binding adaptation, monitoring, management, etc?

1.4.2. Design-time Governance

does the solution support design-time governance functions such as contract, service and service offering definition and lookups, registry-repository access, etc?

1.4.3. Analysis-time Governance

does the solution support service categorization and discovery? Definition and cataloging of reusable governance artifacts such as service and customer tiers, contracts, SLAs, etc.

1.4.4. Lifecycle Governance

does the solution support service lifecycle activities, approval workflows, lease renewal, service evolution and retirement?

1.5. Manages Policies as Reusable Governance Contracts

does the solution support packaging policies into reusable Governance Contracts to prevent uncontrollable policy application and proliferation? Does it help to prevent unnecessary service proliferation by exposing the same service implementation to different constituencies with different contracts?

1.6. Focus of Governance Solution

how focused is the solution on SOA Governance? Higher ratings are given to pure-play solutions focused exclusively on solving the service governance problems, versus other types of solutions such as Management Platforms, ESBs, EAI and Identity Management products and Composite Application Suites that have some SOA Governance capabilities.

2. Support for SOA characteristics

overall weighted rating for the capability of the solution to support various SOA characteristics. Takes into account support for core and additional characteristics, ability of the product to define and support client-specific characteristics and customer's flexibility in choosing which characteristics to implement and how to support them.

2.1. Supports all core SOA characteristics

overall weighted rating for support of the core characteristics.

2.1.1. Security

solution's support for SOA security. Takes into account support for different security aspects: Authentication, Authorization, Integrity, Confidentiality, Accountability, Identity Management and Security Policies; as well as the facets where they manifest themselves: Transport, Message, Application, Asset (Data), Knowledge and Control Security.

2.1.2. Versioning

solution's support for service versioning, including ability to support multiple concurrent versions of the same service, designate primary versions, define version compatibility lists and ability to adapt requests to deprecated versions. Also takes into account to distinguish between version changes that change service interface and those which only affect governance contract.

2.1.3. Lease

ability to support service lease to assure true decoupling of service consumers and producers. Ability to issue, publish and validate lease tokens and implement service lease enforcement policies.

2.1.4. Monitoring

solution's support for service monitoring (both real-time and historic).

2.1.5. Metering and Chargeback

solution's support for collecting service usage information suitable for generating consumer bills. Takes into account support for advanced billing capabilities, such as real-time checks and ability to deny requests for delinquent and overdrawn accounts.

2.1.6. Throttling

ability to define and implement different throttling policies to protect service providers and ensure sustainable service levels. Throttling can include limiting TPS per service, instance, client, etc.

2.2. Supports any additional SOA characteristics

solution's ability to define provision and enforce additional client-specific SOA characteristics.

2.3. Characteristics supported out of the box

number of SOA characteristics which solution supports out of the box (without any additional development).

2.4. Freedom to select any subset of supported characteristics

as the number of SOA characteristics supported by the solution increases, so does potential for overhead. This criterion assesses the ability to support a subset of relevant characteristics during the implementation to ensure that the solution does not introduce any unnecessary overhead.

2.5. Allows incremental rollout of characteristics

solution's ability to change the set of supported and implemented characteristics without affecting any existing service consumers or providers.

3. Architecture

overall weighted rating for architectural maturity and flexibility of the solution and its ability to support all aspects of SOA Governance as the notion of governance continues to evolve.

3.1. Aspect-Based Governance

support for aspect-oriented governance model, which decouples different SOA characteristics and implements them as independent aspect methods.

3.2. Governance Delegation Capability

overall weighted rating for support of the aspect delegation: many of the service governance aspects including: security, versioning, monitoring and management, etc.; have existing packaged solutions broadly adopted in the prospective SOA marketplace. Does the solution allow users who have already adopted a packaged implementation in one or more aspect spaces to delegate the implementation of these aspects to those packages, provided that the packages themselves support such delegation?

3.2.1. Complete Delegation

can the solution delegate entire aspects to an external package?

3.2.2. Partial Delegation

can the solution delegate parts of aspect implementations to an external package?

3.2.3. Orchestrated Delegation

can the solution delegate the entire aspect functionality to and external package but "rearrange" the default functionality through low level APIs?

3.3. Supports SOA beyond Web Services

SOA is often confused with web services, which limits its applicability and adoption. This criterion evaluates how applicable is the solution to other SOA implementation technologies.

3.4. Supports Different Binding Types

overall weighted rating for support of different types of service bindings.

3.4.1. Web Services

support for the standard web service bindings (SOAP over HTTP/HTTPS).

3.4.2. JMS

support for JMS service bindings.

3.4.3. EJB

support for EJB service bindings.

3.4.4. POJO (Java)

support for POJO (pure java) service bindings.

3.5. Customer Defined Binding Types

ability to add new customer-specific binding types, such as .NET, AJAX, COBOL, etc.

3.6. Supports Dual Binding

does the solution support dual (consumer and provider side) bindings? This allows service creation and consumption to occur in multiple environments characterized by different platforms, technology preferences, skill sets, stages of lifecycle, etc. If path to interoperability is seen through the use of single underlying service implementation technology, such as Web Services, it would take unacceptably long time to reach the level of adoption, necessary for SOA to succeed.

3.7. Provides Complete Spectrum of Client Solutions

there is a contention between the need for interoperability and decoupling versus convenience and the need to hide complexities. Can the solution resolve this contention by offering service consumers a spectrum of options for invoking governed services?

3.7.1. Compatible with industry-standard tools

are the services exposed through the solution consumable without any custom client code? Is it compatible with industry-standard tools like Mindreef SOAPScope?

3.7.2. Provides Access points

does the solution provide access points, which allow service invocation without any client-side components, but require client implementation to manage the complexities of dealing with find-bind-execute cycle and compliance with the governance policies?

3.7.3. Provides Client interfaces

does the solution provide client interfaces, which provide service-generic but client platform-specific solution components that insulate consumers from most of the complexities when invoking any exposed service?

3.7.4. Provides Convenience APIs

does the solution provide convenience APIs, which provide both service- and client platform-specific layer for zero-effort invocation of designated services?

3.8. Manages the cost of compliance

does the solution helps to manage the cost of compliance? Is it able to satisfy all of the above criteria would require a solution of significant level of complexity - does it compromise overall viability of the SOA implementations it is designed to support? Specifically:

• Does it introduce performance bottlenecks to underlying service implementations?
• Does it introduce scalability bottlenecks to underlying service implementations?
• Does it introduce a single point of failure or negatively affect the availability of underlying service implementations?
• Does it introduce any additional vulnerabilities or negatively affect the security of underlying service implementations?
• Does it negatively affect the testability of underlying service implementations?

3.9. Service Virtualization

does the solution provide capabilities for building task-specific “virtual” services from existing services? Does it allow to:

• Consolidate one or more operations from different services into one
• Hide selected operations of an existing service
• Rename selected operations of an existing service
• Hide bindings and other details of the service implementation

3.10. Service Refinement

does the solution provide capabilities to slightly change the interface, vocabulary, granularity, semantics or behavior of enterprise services without changing the service implementation or affecting existing service consumers?

3.11. Non-invasive Architecture

is the solution build on a non-invasive architecture, or does it requires to rebuild, re-instrument or redeploy existing service to bring them under the aegis of SOA Governance?

4. SOA Standard Support

overall weighted rating for standard compliance and capability to support SOA related standards. 4.1. Complete support for the Web Services technology stack

overall weighted rating for standard compliance of the existing solution (HTTP, SOAP, WSDL, UDDI / ebXML).

4.1.1. Standard Service Registry

service registry supports UDDI and/or ebXML standards.

4.1.2. Service Repository

solution includes a Service Repository (there is no standard for repositories except ebXML Registry-Repository).

4.2. Capable of supporting all existing SOA-related standards

does the solution has an architecture in place to add support any existing SOA-related standard not currently supported out of the box.

4.3. Capable of supporting any future SOA-related standards

does the solution has an architecture in place to add support any future or custom SOA-related standard not currently supported out of the box.

4.4. Provides hedging against uncertainty in the SOA standards space

overall weighted rating how the solution helps to address and mitigate the uncertainty in the WS Standard space and lack of many needed SOA standards.

4.4.1. Allows to switch between competing standards

does the solution allows to switch implementation of a particular aspect between competing/complimentary standards?

4.4.2. Allows to switch between proprietary and standard aspect implementations

does the solution allows to switch implementation of a particular aspect between proprietary and standard or draft compliant?

Service Refinement

The Philosopher’s Stone of SOA

Original publication date: Oct 30, 2007

This paper is dedicated to the problem of Service Refinement in the Enterprise SOA implementations and outlines my approach to solving this problem in the context of the comprehensive SOA Governance Solution. Let's start with a definition:

Service Refinement refers to the ability to slightly change the interface, vocabulary, granularity, semantics or behavior of enterprise services without changing the service implementation or affecting existing service consumers.

The Challenge

One of the core benefits of SOA is reuse, which means that ideally every service is implemented once and then used throughout the enterprise. In other words every business service, such as InvoiceCustomer, would have to be developed once and then could be used in hundreds of places across dozens of applications for multitude of purposes. It is unlikely that the exact same interface, granularity, vocabulary and level of abstraction will be right for all possible service uses. Thus creating a single implementation that would out of the box satisfy all existing and future uses throughout the company is not realistic - it would have to be infinitely flexible to work on different levels of abstractions, granularities, with different sets of defaults and assumptions, etc. Infinite flexibility means infinite complexity: to design and build such a service would be very costly, require a very long time, and it would still likely miss some of the use cases.

Another approach is to define and create a reasonable 80/20 service and apply some glue as needed on the consumer side. This is also a very costly approach, potentially requiring service consumers to repeatedly re-write the missing 20% of the code in every place where the service is consumed. Furthermore, such glue code will be very tightly coupled with the service itself, so whenever the latter changes in any visible way, the owners of all composite applications that consume such service will have to go through every instance of that glue code and change it accordingly.

Neither of these approaches is acceptable in enterprise-level post-pilot SOA since both result in overcomplicated and fragile solutions, which often forces the adopters of SOA to take an even less desirable route of service replication thus ultimately defeating the purpose of SOA to save money and improve quality through reuse.

The Answer

The best way to implement service refinement is in the context of a comprehensive aspect-based SOA Governance solution. This architecture naturally supports service refinement through well defined, reusable refinement aspects. Service producer defines and implements a reasonable service implementation, which is published and can later be combined with one or more refinement aspects into a number of consumable service offerings. Service consumers then select and bind to the offering which best matches their usage scenarios.

An even more powerful solution would combine the use of refinement aspects that implement common refinement concerns with support of decorating individual services with individual refinement methods or filters to address one-off requirements. When a new use case with some unique requirements is discovered, a new service offering can be created by combining the existing service implementation with a new refinement method, which can be developed by service consumer, producer or a third party.

The best part of the solution is that neither service producer not any of existing consumers of basic or refined service offerings are affected by this change. Any future fixes and improvements of the service implementation would be immediately effective to all of the consumers.

The Uses

Below are some of the real-life SOA challenges that have been successfully solved through service refinement:

·        Customer Information Service: following the SOA best practices, a company implemented a single coarse-grained service getCustomerInfo which provided consolidated customer information, instead of multiple fine-grained services like getCustomerName, getCustomerAddress, etc. This worked well for most consumers except a remote application which required only minimal customer information and was accessing the service over an unsecured, low bandwidth, high cost wireless connection. A simple granularity refinement method allowed to reuse the common lookup service for this remote application.

·        Employee Lookup Service: a company developed a service to lookup employee information based on a number of criteria. The service proved to be very useful and was immediately utilized in a number of applications run by different departments, including payroll, HR and training. However it became apparent that although all of them have a concept of employee, each attached a different meaning: in payroll system employees were everyone who gets paid, including permanent and contractors, training system only considered permanent full-time employees who were eligible for training, and HR software was required to also keep track of former employees. A number of straightforward semantic refinement methods allowed to implement a common lookup service, which was not tied to any particular definition of employee thus reducing coupling. When a new system was discovered in Legal, which had to track employees based on nationality, the same lookup service was reused without any changes.

·        The same Employee Lookup Service: was easily made compliant to the new privacy regulations with a use of vocabulary refinement method that transparently translated employee social security numbers that were used as employee IDs into a surrogate IDs for the consumers invoking the service from the overseas subsidiaries.

Conclusion

Support for service refinement should be a crucial part of any SOA enablement platform. Similarly to service versioning, which allows services to evolve in time to meet the changing consumer needs, refinement allows services to adapt to different usage scenarios without unnecessary duplication and service proliferation. A universal SOA governance solution allows to manage service offerings based on refinement in the same way as those based on different security models or SLAs.

SOA Governance Terminology and Positioning

Original publication date: Nov 01, 2007

I have been wrestling with this for a while but I think I have finally found an elegant solution to the SOA Governance terminology and positioning conundrum.

First, this is a short recap of the problem: we have approached solving the problem of SOA governance very methodically and developed a model that identified a number of well-defined entities that exist in the problem domain. Since most of the SOA terminology was coined by the analysts and other industry pundits, it is very imprecise and often downright ambiguous. As a result we decided to introduce our own terminology that has not been marred by multiple contradictory connotations in the context of SOA governance. That worked fine and allowed us to develop and successfully communicate the solution to technically apt listeners, however as soon as we widened the audience, those same analysts and pundits started picking on us for lack of support for the things like policies and service contracts broadly accepted as must-haves for any SOA governance solution. Ironically, when we did an about-face and renamed key artifacts of our solution to match the closest industry-standard monikers, we opened ourselves to criticism that our policies and contracts differ from those talked about in the analysts’ reports and vendor’s white papers.

I always felt that the solution should be to keep the buzzword-compliant nouns and prepend them with a distinguishing qualifier: akin turning democracy into functioning democracy. And in one late night revelation I found that qualifier: Governance. Now everything falls into place:

• Policies in general are mutable business-related aspects of service behavior, which developer has to externalize in order to allow analysts and other non-technical players to define and change them without changing the service implementation. Examples of policies include:
1. Service should authenticate users and should not accept UnerNameToken with unencrypted password or self-issued certificates.
2. Service should be available 99.9% of time during business hours and should be able to sustain 10 TPS during this time.
3. Service should accept requests for the previous, but not any of the earlier versions.
4. Service Leases should be tamper proof and should not exceed 30 calendar days.
5. Purchase order in excess of $50,000 should be routed to a VP for an approval.
6. In the absence of available rooms, reservation for a platinum level guest should bump an existing reservation of a lower level guest.
7. Orders from a client with more than 5 outstanding invoices should be routed through the collections department.

Policies in general are inherently application-specific and should be interpreted by the application itself. Thus they can be expressed in standardized format that lacks semantics and strong validation capabilities, such as described in WS-Policy. Generic policies (examples 5 through 7 above) are best represented as business rules and workflows and their enforcement (implementation) should be delegated to a Business Rule Engine and/or Workflow Solution as a part of the regular application architecture. There is usually little need to advertise these policies as a part of service contract, since the consumers will unlikely be able to understand and interpret them the same way as the service producer.

• Governance Policies (examples 1 through 4 above) represent common non-functional characteristics of SOA services, which should be declared, validated, interpreted and enforced in a centralized and uniform way outside of and independently from service implementations. Thus Governance Policies have to be expressed in a strongly typed semantically bound format ideally described by a Governance Information Model and need to be coupled with executable Enforcement Agents (aspect methods).
• Service Contract is an amorphous conglomeration of interface descriptions, invocation conventions (such as required SOAP headers), bindings and ports (WSDL), adorned with generic policies and other assorted metadata which has to be parsed and interpreted by service clients and implementers.
• Governance Contract is a complete set of non-functional characteristics of an enterprise service expressed in the form of Governance Policies, which is unambiguously interpreted and enforced by the Governance Solution. Governance Contract represents a business-centric view of a service and combined with a developer-centric Interface Description defines a holistic Service Offering.

Unfortunately this approach will not work for governance itself: we can not call our solution Governance Governance, in order to distinguish it from the amorphous offerings of the competition – it would be too … Nabokov. But we do need a differentiator here as well, and with my innate hubris I suggested True Governance, but for some reason it was not very popular L… Closed Loop Governance is already taken – not that liked it that much or even fully understood the term: it vaguely suggests automation, but governance should be a deliberate human oversight activity. Other possibilities might include Executable Governance, Model-driven Governance and Proactive Governance.

Armed with this terminology and message we should be able to successfully position and communicate our solution to a wide spectrum of audiences including our own leadership, analysts, technologists and executives.

Multiple Personalities of SOA Governance

Two roads diverged…^[1]

Not a typical shrink's couch

In an earlier post, I asked a question: “is Governance worth the trouble?” and left a cliffhanger for an answer that it depends on how one looks at it. Today, I will try to untangle the issue further. But first let’s look at the diagnosis. Imagine yourself as a psychiatrist who encounters a patient that exhibits the following symptoms:

1. is perceived very differently by different observers
2. exhibit drastically different behaviors under identical circumstances
3. possesses some advanced capabilities one day and completely lacks them the next
4. radically changes the ways in which it relates to and interacts with the others

If you are an American^[2], you will undoubtedly diagnose the patient with the Multiple Personalities Disorder. Wouldn’t you agree that all of the above apply to SOA Governance? I admit that at some point the parallels end: there was no documented early abuse, hypnosis or memory loss, but in my book there are enough similarities, to warrant looking at Governance from this angle. And, as in real life, the doctor needs to identify, isolate and unravel each of the personalities before they can make progress with their patient, let’s try the same approach with ours. Now that we’ve got Governance comfortable on our couch, let us start with the history…

Not means of transportation

I remember attending a webinar, where Anne Thomas Manes, the recognized matriarch of SOA Governance, was warming up the crowd for a pitch by HP/Systinet, where she used to be the CTO. She started by saying “SOA Governance is about risk mitigation” which almost made me to do the Parisian Tailor (leave), but instead got me thinking. She is the one of the most recognized authorities in the field, so she can’t be wrong. Yet how could she be right, and talking about the same Governance to which I have dedicated the last several years of my career? Imagine yourself hearing one of the German car manufacturers to say: “Automobile is a means of ground transportation" – hard to argue with, true, yet somehow completely unfathomable. But there is a number of car companies, that live and compete (although perhaps not with the Germans) by that formula. That is when the idea for this article first hatched. Perhaps the theological disputes about the nature of Governance (how many policies can dance on the head of a PEP?) miss the point that the latter presents different personalities to different observers.

Construction workers

In my professional observation I identified two distinct personalities, which I named the Hauler and the Builder as a reference to one of my favourite parables attributed to Roberto Assagioli. In it a traveller walked past a construction site and saw three men doing the same work. He stopped and asked them: what are you doing? The first one replied – “Hauling stones”. The second – “Earning a living”, and the third one – “Building a Temple”. When I ran a recruitment operation, I used to tell it to some of the candidates when critiquing the resumes and discussing interviewing techniques. The diagram below illustrates how the two personalities manifest themselves to the main constituencies: technology vendors and customers.

Customer	Builder		Vendor
	Strategic Transforming Services into Business Assets; Building Enterprise-grade SOA Ecosystem;	Offensive Delivering on the original SOA promise; Creating strategic opportunities;
	Tactical Adding Security, Logging, etc. to web services; Virtualizing endpoints; Creating System of record;	Defensive Closing gaps in product lines; Checking off the boxes on RFPs Placating the analysts;
Point of View	Hauler		Point of View

If I had to summarize each personality in a single word, Builder would be transformative and Hauler – incremental. Not that there is something wrong with the latter: both represent steps into the right direction, and any kind of improvement is way better then none.

Now, when we have identified the personalities we can analyze them in a bit more detail. Builder is a visionary and likes to relate to the same: given a choice she would go straight to the CIO/CTO. Hauler is a pragmatist, and prefers those who would benefit immediately: operations, security, etc. Builder tries to emphasize flexibility and potential, while Hauler concentrates on convenience and stability (often coming as an appliance). Builder is a generalist, trying to address all aspects of Governance in the broadest possible sense, Hauler usually tries to narrow down the problem, do one thing and do it well. Builder likes to talk about business challenges and opportunities; Hauler concentrates on technical capabilities and supported standards. Builders often see themselves as unique and irreplaceable, while Haulers tend to seek strength numbers and form or join all kinds of unions.

In retrospect it makes sense that Anne Thomas Manes chose the Hauler, when she introduced Systinet, who is the perceived leader amongst Service Registries, and thus most interested in maintaining status quo. This leads us to the final question: which of the two you (as a customer, or perhaps, a vendor) should be dealing with? The answer, as always, is highly individual but for me it feels too crowded on the Hauler’s side. And we can easy guess whose pay is better

---

¹ This is a re-post of an rticle originally published on Jun 27, 2008 on my old Sun blog.

² Until very recently, the phenomenon whose proper name is Dissociative Identity Disorder was almost exclusively confined to North America.