soa

Importance of a SOA Maturity Model

I talked a little about the Maturity Model concept in my blog entry about Simplifying Build Management – Build Once, Deploy Anywhere. One of the more well know Maturity Models is called CMMI (Capability Maturity Model Integration). In the presentation Capability Maturity Model Integration (CMMI) Version 1.2 Overview, there are descriptions of a “Process Model” that helps lay the foundation of the benefits of a Maturity Model.

A process model is a structured collection of practices that describe the characteristics of effective processes.
A process model is used

  • to help set process improvement objectives and priorities
  • to help ensure stable, capable, and mature processes
  • as a guide for improvement of project and organizational processes
  • with an appraisal method to diagnose the state of an organization’s current practices

The Maturity Model helps to define an architectural framework, what different levels of implementation can be achieved with the framework, ability to measure improvement of its usage, and the ability to assess the state of that architectural framework. The Maturity Model puts a subject like SOA into a framework and creates multiple maturity levels that define initial stages, all the way to advanced stages of the SOA process. This tutorial site provides a clear definition around what a maturity level is:

A maturity level is a well-defined evolutionary plateau toward achieving a mature software process. Each maturity level provides a layer in the foundation for continuous process improvement.

So we essentially have two maturity models that we are leaning on with regards to the standup of our SOA initiative. The initial example is from a document written in 2005 by Sonic Software Corporation (A NEW SERVICE-ORIENTED ARCHITECTURE (SOA) MATURITY MODEL). This provides an excellent example view into the different layers of a SOA Maturity Model and then finer grained discussions around each level. The following is the example Service-Oriented Maturity Model from this document:

soamm

We can see from this model that we have 5 essential levels for achieving SOA. The 5 maturity levels defined here are Initial Services, Architected Services, Business/Collaborative Services, Measured Business Services, and Optimized Business Services. These different maturity levels allow us to evolve our SOA initiative over time and validate that we are correctly progressing through those levels. For instance, one of the recommended steps for implementing an SOA is to utilize a Enterprise Service Bus to stand up some initial services (in order to create a starting point). It’s easy to determine the success of the overall SOA initiative based on this initial phase, which might give a view that the SOA initiative is not successful. This might be because the right types of services are not being exposed, or composite services aren’t being created, advanced standards aren’t developed, and so forth. But in the context of a Maturity Model, it allows us to evaluate our current SOA initiative against the different levels and determine that we have successfully achieved Level 1, lining up what is necessary for us to move towards Level 2 of the Maturity Model.

primobolan depot

Another Maturity Model for SOA is provided at Griffiths Waite – the Oracle SOA company. This is defined more clearly in the resource document SOA Maturity Model – Quick Reference Guide.

SOA-maturity-model-large

This Maturity Model also provides 5 phases called Opportunistic, Systematic, Enterprise, Measured, and Industrialised. This documents the evolution of SOA from initial services, to the development of standards, to utilizing BPEL for integration of multiple services, then utilizing BAM to monitor and measure services, to the ability to be completely agile via the previously successfully implemented 4 phases.

The following article provides a landing page for quite a few different SOA Maturity Models.

The fact is that starting a SOA initiative is a massive undertaking with many moving pieces and steps. It is easy to become overwhelmed as to the correct processes and architecture necessary to achieve the SOA end state. The benefit of aligning these initiatives with a SOA Maturity Model is that we are able to partition our SOA intiative into individual levels/stages so that we can achieve SOA in more manageable chunks. At the end of each phase of our SOA, we can determine whether or not we have evolved to the next phase of the SOA Maturity Model and what is left to accomplish our current level.

Share and Enjoy

Dangers of a Domain Model as the Messaging Model

When designing a services layer, we can relate this to an n-tier architecture where the presentation layer exists on a separate server (possibly developed under a different technology). So the services layer will essentially be like the Session Facade down to the integration layer for interactions between databases, file systems, etc. In this design, there is usually a decision as to whether or not the domain models will be exposed out of the business layer, or whether those domain models will be mediated into a different object representation like a DTO (Data Transfer Object). In this discussion, Domain Models represent our persistence representations while the Messaging Model represents our models exposed via a Service Contract.

In the design of the models themselves, the drive is towards creating a Canonical Model as described in XML Standards for Service Modeling. In this specific implementation, the decision was to utilize the Domain Model as the Canonical Model and expose those models via the Service Contract. As different parts of the application architecture evolved, this began to expose itself as being the cause of some issues.

First off, the technology being used in this case is CXF/JAX-WS/Spring/Hibernate/JPA/AOP. The CXF/JAX-WS frameworks/APIs are used for exposing SEIs as web services for invocation. JPA/Hibernate are being used to provide the persistence framework through JPA ORM files and Hibernate as the Provider. Spring is being used to wire together the CXF/JAX-WS endpoints with the business logic and persistence layer, as well as using AOP with JTA to provide transactionality.

One of the first issues to be hit related to the fact that models representing persistence may be over complex for the needs of a UI Framework (which would potentially be more successful with a flattened model). The persistence model however has complex relationships like bidirectional relationships that allow us access to Parent/Child interactions without having to loop through an object model and keep track of them ourselves. The issue this brings to light though is that a bidirectional relationship will many times consist of a Parent with a Collection of Children, where each instance of the Child in the Collection has a reference back to the Parent, which internally has a Collection of Children and on and on and on. So when it comes time for JAXB to marshal the objects into XML to produce the SOAP Response being sent out of the CXF/JAX-WS framework, the process consumes all memory and begins to run out of Memory and the server crashes. This occurs because the recursive nature of a bidirectional relationship and as JAXB walks the object model to persist it, it will start into a recursive loop of serialization as it walks that bidirectional relationship and the callbacks from the Child to the Parent and so on. This issue is discussed in some detail in the following two posts:

JPA Entities to XML – Bidirectional Relationships
JAXB Mapping cyclic references to XML

The Child object will most likely need to have an @XmlTransient annotation on the relationship it has back to the Parent. Therefore when JAXB is marshalling the object model into XML, it will convert the Parent and then Collection of Children, but not the Child’s relationship back to the Parent. The client of the calling service needs to be aware then that the Child does have a Parent (the containing Parent) even though over the wire (with relation to the SOAP message) it appears that the Child does not have a reference back to a Parent. There maybe a need to think through the same sort of concept for the equals/hashCode/compareTo/toString, as to which fields are being used and how much of the object graph is being walked in order to utilize that field.

Another issue is related to what happens within an application with regards to how Hibernate handles relationships. Hibernate helps the initial load and overall performance of an Entity by providing lazy loading features. Hibernate Without a ton of background experience, it was easy to incorrectly configure Hibernate ORM files by trying to make all the Collections eagerly fetched. But when the WAR is deployed and Spring begins to bootstrap the loading of the persistence.xml and orm.xml files, we receive an error related to (Caused by: org.hibernate.HibernateException: cannot simultaneously fetch multiple bags).

Hibernate and Multiple Bags
Hibernate Exception – Simultaneously Fetch Multiple Bags

So we have to correctly label Collections as LAZY, by default they are supposed to be. This seems to fix our issue with the Multiple Bag Exception. The problem is that now our Collections that are not EAGER hold a reference to Hibernate’s PersistentBag (used as a proxy for the lazy loading of the relationship) and the One-to-one Objects that are not EAGER hold a Hibernate Proxy (used as a proxy for the lazy loading of the relationship). This becomes an issue in web applications related to whether or not the session is open so that when those proxies get touched, they can retrieve the subsequent relationship. Otherwise we will receive the Hibernate LazyInitializationException. There are quite a few articles on how to resolve this issue:

Hibernate – Lazy Load exception – no session or session was closed
How to solve lazy initialization exception using JPA and Hibernate as provider (with newer ways to solve it).

Even though we are not running in a complete MVC web application, we have the same situation occurring because the models we are exposing through the web service are Domain Models tied to a Hibernate Sessions (because of the EntityManager and ORM files). So like a typical SessionFacade, the service that is exposed as a web service has AOP Pointcuts to add JTA Transactionality. What occurs is that when that WAR receives a web service request, JAX-WS/JAXB/CXF do their work and then hand off the Object to the appropriate SIB (Service Implementation Bean) which has an AOP Pointcut tied to it, so a JTA transaction has been started. All data access that is done within that transaction ties the Hibernate Session (EntityManager) to that transaction, because of:

1
<entry key="hibernate.current_session_context_class" value="jta"/>

Therefore when all the logic is complete and the service method returns the Object to the calling framework (essentially the business logic is done and now the JAX-WS/JAXB/CXF will take over), the AOP Pointcut is complete and closes the Transaction (and therefore the Hibernate Session). So when the JAXB part of the process marshals the Object into XML and walks the object graph, it will run into the Hibernate Proxies and PersistentBag instances, with no Hibernate Session from which to retrieve the relationship. Therefore the LazyInitializationException occurs.

In some of these examples they talk about the OpenEntityManagerInViewFilter or called the Open Session in View pattern. This is one of the typical ways to solve this issue from within a web application.

This can be done by making the following modifications (as an example)

1
<entry key="hibernate.current_session_context_class" value="jta"/>

is changed to

1
<entry key="hibernate.current_session_context_class" value="org.springframework.orm.hibernate3.SpringSessionContext" />

And add the following to the web.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<filter>
        <filter-name>OpenEntityManagerInViewFilter</filter-name>
        <filter-class>org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter</filter-class>
        <init-param>
            <param-name>entityManagerFactoryBeanName</param-name>
            <param-value>sampleEntityManagerFactory</param-value>
        </init-param>
        <init-param>
            <param-name>singleSession</param-name>
            <param-value>true</param-value>
        </init-param>
        <init-param>
            <param-name>flushMode</param-name>
            <param-value>AUTO</param-value>
        </init-param>
    </filter>

This actually can work for us. Another way to do it is to create a CXF Interceptor to start the transaction at a certain point in the Phases of unmarshalling data when the request is received, or closing the transaction when the marshalling process has completed. Since we hadn’t tried the Open Session in View pattern, the resolution that occurred was creating a touch method that would utilize a ReflectionUtils.doWithFields functionality to walk the whole object graph and touch any Hibernate Proxies that were not EAGERLY loaded. This would essentially force selects to occur immediately when the object was retrieved from the database (by adding a step in the framework to perform a touch of an object before it was returned to the calling method).

Another approach was to use the concept in service design where we could use a single fine grained method to produce multiple types of responses based on the data the user wanted. So for instance, if we had a method that was getPerson and retrieved a Person, their family, their addresses, etc, we could change the amount of data returned from that one getPerson method by supply a list of Filters/Scopes for limiting data. This would allow a user to call getPerson with a List of Filter fields like address and family, so that when the getPerson retrieved a Person object, it did not also retrieve the Person’s Address and Family (would be additional select statements to other tables). Therefore, the we could take the fact that Hibernate will not allow more than one Collection to be EAGER as an advantage because time was not wasted pulling in additional resources the user might not want. The Person object would be retrieved and have its Hibernate Proxies (for doing lazy loading of the Collections) and if the List of Filters was populated, the code could determine if one of the Filtered fields was a Hibernate Proxy, then that reference could be set to null (dereferencing that Hibernate Proxy). Therefore when the object was marshaled via JAXB, it would not throw a LazyInitializationException on that field because the reference was null and we did not need that data returned to the user.

This approach actually seemed to produce less selects than by using the Open Session in View pattern. This simplified the architecture by leaving the AOP and JTA Pointcuts as they were originally developed.

Share and Enjoy

XML Standards for Service Modeling

When designing web services, it becomes apparent that it is necessary to lock in standards for designing the service contracts as well as the service models. In an effort to standup the initial phase of the SOA Enterprise, many services begin to expose themselves on the ESB that either are driven by a legacy system (such as a mainframe generated service) or by developer building the models in isolation. Since the goal is to drive towards a common Canonical Model, it is important to define a set of standards around how Schemas are designed and rules that need to be followed when creating them.

If we do a little googling, you might land on the Oasis XML Coverpages website, which is a great resource for XML based standards groups. The XML Coverpages site has a landing page for Naming and Design Rules. One of the resources made available in the table listing of this page is the “UN/CEFACT XML Naming and Design Rules”, which has two great resources.

UN/CEFACT 8 XML Naming and Design Rules Technical Specification 9 Version 3.0
UN/CEFACT/UBL XML Naming and Design Rules Analysis

The “UN/CEFACT/UBL XML Naming and Design Rules Analysis” is a spreadsheet that can be downloaded in DOC format. It refers to another document called Universal Business Language (UBL)
Naming and Design Rules
. These documents provide some very detailed discussions around the standard uses of XML Schemas and the implementation of the XML Schema to create Models. This might be as detailed as to the types of names of elements, what attributes are expected, and whether or not features in the XML Schema Specification can be used at all (like xs:all).

Another one of the interesting examples is the Department of the Navy
XML Naming and Design Rules
because of the great detail an explanation of all the different types of rules and usage standards.

In reading these documents, it becomes quite apparent how difficult it can be to produce the ideal environment for Service Modeling, let alone the complexities/capabilities in utilizing the XML Schema Specification. These concepts are very important in a SOA that pushes a Contract-First methodology, where the WSDLs and Schemas will be designed and developed first. One of the books for Java web service development talks about a bridged approach between the Contract-First and Code-First called Code-First/Contract-Aware.

Here is the book

This is really about taking advantage of the great annotation support in JAX-WS and JAXB to replicate the standards for Service Modeling within the annotations themselves. While you have more power in developing the XML Schemas directly, the simplicity that is provided through the JAXB annotations cannot be understated. Even with the power of JAXB annotations, standards will be necessary for the names/features that are used with the framework, along with defining how to implement complex JAXB representations such as Maps/Enums/Collections.

For some businesses, these documents and their detail maybe overkill, but there is probably some value at least in having some generalized rules for XML naming and design (such as not having verbs in the Schema names except for the introduction of operations within the WSDL, or the names of elements and what detail they are, etc). Having a set of some core rules and definitions will ease the integration of development teams into the enterprise SOA.

Share and Enjoy