Cloud Computing Bible (2011)
Part IV: Understanding Services and Applications
IN THIS PART
Chapter 13
Understanding Service Oriented Architecture
Chapter 14
Moving Applications to the Cloud
Chapter 15
Working with Cloud-Based Storage
Chapter 16
Working with Productivity Software
Chapter 17
Using Webmail Services
Chapter 18
Communicating with the Cloud
Chapter 19
Using Media and Streaming
Chapter 13: Understanding Service Oriented Architecture
IN THIS CHAPTER
Using Service Oriented Architecture
Defining message-based transactions
Understanding processes and transactions
Managing SOA distributed applications
Service Oriented Architecture (SOA) describes a standard method for requesting services from distributed components and managing the results. Because the clients requesting services, the components providing the services, the protocols used to deliver messages, and the responses can vary widely, SOA provides the translation and management layer in an architecture that removes the barrier for a client obtaining desired services. With SOA, clients and components can be written in different languages and can use multiple messaging protocols and networking protocols to communicate with one another. SOA provides the standards that transport the messages and makes the infrastructure to support it possible. SOA provides access to reusable Web services over a TCP/IP network, which makes this an important topic to cloud computing going forward.
You don't need SOA if you are creating a monolithic cloud application that performs a specific function such as backup, e-mail, Web page access, or instant messaging. Many of the large and familiar cloud computing applications are monolithic and were built with proprietary technologies—albeit often on top of open source software and hardware. However, as cloud computing applications expand their capability to provide additional and diverse services, SOA offers access to ready-made, modular, highly optimized, and widely shareable components that can minimize developer and infrastructure costs.
For over a decade now Service Oriented Architecture has been part of a collaborative effort on the part of both large and small vendors to come up with a common solution to architecting complex business software processes efficiently. As cloud computing matures and the applications offered become more capable, the key to being competitive and offering users the capability to customize their environments lays in the standardization that Service Oriented Architecture offers. The influence of SOA in cloud computing is therefore likely to grow.
This chapter provides the basis for understanding what SOA is, how SOA operates, what limitations and capabilities are part of the architecture, and the vocabulary that you need to know in this subject. SOA is an architecture, first and foremost, so in essence it is a blueprint for creating a system conforming to this standard. The environment it creates is a virtual message-passing system with a loose coupling between clients and services. The products that support SOA are a diverse lot. SOA components are meant to be modular and easily added to a business process, and this modularity makes them good candidates for Computer Aided Software Engineering (CASE) modeling tools. This chapter describes the software used to support SOA and some of the important ways in which the architecture is interpreted.
Cloud computing is not the next evolutionary step beyond SOA, as some think because of SOA's longer history. The two technologies are complementary. Whereas SOA can be used to construct large and complex applications that scale both horizontally and vertically, cloud computing applications tend to be scaled vertically. Horizontal scaling refers to applications with a large number of different business processes operating. Vertical scaling refers to large applications with a limited number of business processes operating. SOA techniques may be applied in both instances.
Introducing Service Oriented Architecture
Service Oriented Architecture (SOA) is a specification and a methodology for providing platform- and language-independent services for use in distributed applications. A service is a repeatable task within a business process, and a business task is a composition of services. SOA describes a message-passing taxonomy for a component-based architecture that provides services to clients upon demand. Clients access a component that complies with SOA by passing a message containing metadata to be acted upon in a standard format. The component acts on that message and returns a response that the client then uses for its own purpose. A common example of a message is an XML file transported over a network protocol such as SOAP.
Usually service providers and service consumers do not pass messages directly to each other. Implementations of SOA employ middleware software to play the role of transaction manager (or broker) and translator. That middleware can discover and list available services, as well as potential service consumers, often in the form of a registry, because SOA describes a distributed architecture security and trust services are built directly into many of these products to protect communication. Middleware products also can be where the logic of business processes reside; they can be general-purpose applications, industry-specific, private, or public services.
Middleware services manage lookup requests. The Universal Description Discovery and Integration (UDDI) protocol is the one most commonly used to broadcast and discover available Web services, often passing data in the form of an Electronic Business using eXtensible Markup Language (ebXML) documents. Service consumers find a Web service in a broker registry and bind their service requests to that specific service; if the broker supports several Web services, it can bind to any of the ones that are useful.
This architecture does not contain executable links that require access to a specific API. The message presents data to the service, and the service responds. It is up to the client to determine if the service returned an appropriate result. An SOA is then seen as a method for creating an integrated process as a set of linked services. The component exposes itself as an “endpoint” (a term of art in SOA) to the client.
The most commonly used message-passing format is an Extensible Markup Language (XML) document using Simple Object Access Protocol (SOAP), but many more are possible, including Web Services Description Language (WSDL), Web Services Security (WSS), and Business Process Execution Language for Web Services (WS-BPEL). WSDL is commonly used to describe the service interface, how to bind information, and the nature of the component's service or endpoint. The Service Component Definition Language (SCDL) is used to define the service component that performs the service, providing the component service information that is not part of the Web service and that therefore wouldn't be part of WSDL.
Note
Whatever protocol is used to negotiate a transaction, the formal definition of the transaction is referred to as the “contract.” Indeed, the notion of a contract implies a certain level of service that is available to clients and that may be part of any paid service in SOA.
Figure 13.1 shows a protocol stack for an SOA architecture and how those different protocols execute the functions required in the Service Oriented Architecture. In the figure, the box labeled Other Services could include Common Object Request Broker Architecture (CORBA), Representational State Transfer (REST), Remote Procedure Calls (RPC), Distributed Common Object Model (DCOM), Jini, Data Distribution Service (DDS), Windows Communication Foundation (WCF), and other technologies and protocols. It is this flexibility and neutrality that makes SOA so singularly useful in designing complex applications. These services and the manner in which they interact in regards to SOA have been codified by a number of standards organizations, and some of the more prominent efforts are described later in this chapter.
Tip
To read the IBMs SOA Foundation White Paper, see http://download.boulder.ibm.com/ibmdl/pub/software/dw/webservices/ws-soa-whitepaper.pdf.
SOA provides the framework needed to allow clients of any type to engage in a request-response mechanism with a service. The specification of the manner in which messages are passed in SOA, or in which events are handled, are referred to as their contract. The term is meant to imply that the client engages the service in a task that must be managed in a specified manner. In real systems, contracts may specifically be stated with a Quality of Service parameter in a real paper contract. Typically, SOA requires the use of an orchestrator or broker service to ensure that messages are correctly transacted. SOA makes no other demands on either the client (consumer) or the components (provider) of the service; it is concerned only with the interface or action boundary between the two. This is the earliest definition of SOA architecture.
FIGURE 13.1
A protocol stack for SOA showing the relationship of each protocol to its function
Components are often written to comply with the Service Component Architecture (SCA), a language- and technology-agnostic design specification that has wide, but not universal, industry support. SCA can use the services of components that are written in the Business Process Execution Language (BPEL), Java, C#/.NET, XML, or Cobol, and can apply to C++ and Fortran, as well as to the dynamic languages Python, Ruby, PHP, and others. This allows components to be written in the easiest form that supports the business process that the component is meant to service. By wrapping data from legacy clients written in languages such as COBOL, SOA has greatly extended the life of many legacy applications.
Tip
To read David Chappel's white paper on SCA, go to http://www.davidchappell.com/articles/Introducing_SCA.pdf.
Components are coded with their service logic and their dependencies, QoS is established, and the service is instantiated. In the SCA model, data and messages are exchanged in a Service Data Object (SDO). This system of messaging using objects and services is sometimes referred to as a Data Access Service (DAS). Figure 13.2 shows how components of different types can communicate using different protocols as part of SOA.
When you combine Web services to create business processes, the integration must be managed. Two main methods are used to combine Web services: orchestration and choreography. In orchestration, a middleware service centrally coordinates all the different Web service operations, and all services send messages and receive messages from the orchestrator. The logic of the compound business process is found at the orchestrator alone. Figure 13.3 shows how orchestration is managed.
FIGURE 13.2
SOA allows for different component and client construction, as well as access to each using different protocols.
By contrast, a compound business process that uses choreography has no central coordination function. In choreography, each Web service that is part of a business process is aware of when to process a message and with what client or component it needs to interact with. Choreography is a collaborative effort where the logic of the business process is pushed out to the members who are responsible for determining which operations to execute and when to execute them, the structure of the messages to be passed and their timing, and other factors. Figure 13.4 illustrates the nature of choreography.
What isn't clear from Figure 13.2, but is shown in Figure 13.3 (orchestration) and Figure 13.4 (choreography) is that business processes are conducted using a sequence, in parallel, or simply by being invoked (called to). An execution language like WS-BPEL provides commands for defining logic using conditional statements, loops, variables, fault handlers, and other constructs. Because a business process is a collection of activity graphs, complex processes are often shown as part of Unified Modeling Language (UML) diagrams. UML is the modeling language of the Object Management Group that provides a method for creating visual models for software in the form of 14 types of diagrams. Some of the diagram types are structure, behavior, class, component, object, interaction, state, and sequence.
Tip
You can find a primer on BPEL by Matjaz Juric on Oracle's Web site at http://www.oracle.com/technology/pub/articles/matjaz_bpel1.html. For information on SOA modeling, refer to Service-Oriented Modeling: Analysis, Design, and Architecture, by Michael Bell, Wiley, 2008, and Bell's later book SOA Modeling Patterns for Service-Oriented Discovery and Analysis, Wiley, 2010.
FIGURE 13.3
An orchestrated business process uses a central controlling service or element, referred to as the orchestrator, conductor, or less frequently, the coordinator.
Most mature SOA implementations favor orchestration over choreography for a number of reasons. With orchestration a single central service manages the various processes, and changes to the business logic can be made in that one location. The integration of Web services into the architecture is easier than with choreography because these services don't need to know anything about the business process. Centralizing the business logic also makes it easier to put error handling mechanisms in place and to account for, manage, and analyze events that occur outside the business process that relate to a part of the process. Event handling is part of event-driven SOA or SOA 2.0, which extends Service Oriented Architecture to include both random and scheduled events that are triggered by a business process outside of a business process.
One way of performing orchestration is through the use of an Enterprise Service Bus or ESB. An ESB provides a middleware software layer for event management with a messaging infrastructure. ESBs are described later in the section called “The Enterprise Service Bus.” An ESB isn't required by SOA, but it is often used to create a compliant and efficient service architecture.
FIGURE 13.4
With choreography, business process execution is a cooperative affair.
Event-driven SOA or SOA 2.0
Event-driven SOA or SOA 2.0 is an extension of the Service Oriented Architecture to respond to events that occur as a result of business processes or perhaps cause and influence a business process. For example, in a business process, sales at a certain Web site are processed. If the business process recognizes the rate at which sales are occurring, it could perform an analysis to determine what events might influence the buying decision. This is the sort of analysis that event-driven SOA is meant to address. SOA 2.0 can allow low-level events to trigger a business process, correlate events with information contained in the SOA design, inhibit a business process if the appropriate events don't appear, or invoke a reaction or response based on a trigger.
To perform these tasks in SOA 2.0, a Causal Vector Engine (CVE) with some built-in artificial intelligence must be added to the SOA design. Events are analyzed in terms of event sequences, event relationships, and event timing to establish whether a certain condition has occurred. The CVE then determines how to react to the condition using a set of rules that are built into the system. Many CVE systems display events in a console in different contexts so that an observer can analyze the display and take appropriate actions. A CVE application may include the ability to query event data in the same way that a stock ticker or trading application can query trading data. The CVE application provides the same kind of heartbeat and correlation functionality that a stock trading application does.
From the standpoint of the service requestor or consumer (client), the client simply needs to know the form required to initiate the action of the provider (service) and how to interpret the results returned from the service provider. The nature of the component's processing is unknown, the location where the processing is done is unknown, and the various operating systems and applications involved are unknown. The client is responsible for validating that the service returned the results that were expected. The SOA component is essentially a black box to the client. That is, SOA makes no demands of the component other than to conform to the rules of a standard endpoint. This level of abstraction offers operational advantages to Web service providers in that components can be continually upgraded, replaced, or moved to improve efficiencies without disrupting the clients that depend on those services, and the Quality of Service for that service can be accurately measured and delivered. In SOA, the service has been virtualized.
Cross-Ref
Communication protocols are discussed in detail in Chapter 3, “Understanding Cloud Architecture.”
Any network service that spans different application types is a candidate for componentization. Consider a logon or authentication system. It would be wasteful to implement the same authentication functionality in several different applications when a single unified module could serve the same purpose. A single sign-on system for an accounting package, payroll module, or production database replaces three separate modules with attendant efficiencies in the amount of code that needs to be written and managed and the level of system overhead that is involved. SOA provides the rules so each application can access the authentication module in its own way, as required. What you gain with SOA is the ability to add significant capabilities with a fraction of the cost or effort and to federate applications if you desire. What you lose with SOA is the ability to perform fundamental customization of the service itself when that service is provided by a third party.
The Enterprise Service Bus
In Figure 13.5, those aforementioned hypothetical three different applications are shown interfaced with an authentication module through what has come to be called an Enterprise Service Bus (ESB). An ESB is not a physical bus in the sense of a network; rather, it is an architectural pattern comprised of a set of network services that manage transactions in a Service Oriented Architecture.
You may prefer to think of an ESB as a set of services that separate clients from components on a transactional basis and that the use of the word bus in the name indicates a high degree of connectivity or fabric quality to the system; that is, the system is loosely coupled. Messages flow from client to component through the ESB, which manages these transactions, even though the location of the services comprising the ESB may vary widely.
An ESB is necessary but not essential to a Service Oriented Architecture because typical business processes can span a vast number of messages and events, and distributed processing is an inherently unreliable method of transport. An ESB therefore plays the role of a transaction broker in SOA, ensuring that messages get to where they where supposed to go and are acted upon properly. The service bus performs the function of mediation: message translation, registration, routing, logging, auditing, and managing transactional integrity. Transactional integrity is similar to ACID in a database system—atomicity, consistency, isolation, and durability, the essence of which is that transactions succeed or they fail and are rolled back.
FIGURE 13.5
An SOA application of a shared logon or Authentication module
An ESB may be part of a network operating system or may be implemented using a set of middleware products. An ESB creates a virtual environment layered on top of an enterprise messaging system where services are advertised and accessed. Think of an ESB as a message transaction system. IBM's WebSphere ESB 7.0 is an ESB based on open standards such as Java EE, EJB, WS-Addressing, WS-Policy, and Kerberos security, and it runs on the WebSphere Application Server. It is interoperable with Open SCA. WebSphere ESB contains both a Service Federation Management tool and an integrated Registry and Repository function.
These typical features are found in ESBs, among others:
• Monitoring services aid in managing events.
• Process management services manage message transactions.
• Data repositories or registries store business logic and aid in governance of business processes.
• Data services pass messages between clients and services.
• Data abstraction services translate messages from one format to another, as required.
• Governance is a service that monitors compliance of your operations with governmental regulation, which can vary from state to state and from country to country.
• Security services validate clients and services and allow messages to pass from one to the other.
Figure 13.6 shows how these different services in an SOA relate to one another.
The difference between a repository and a registry in the context of a Service Oriented Architecture is subtle. Repositories and registries are both data stores, but a repository stores references to the components of the SOA, their source code, and linking information that are used to provide SOA services. An SOA registry contains references to rules, descriptions, and definitions of the services—that is, the metadata of the components.
A repository serves the role that a name server does in a network operating system infrastructure, while the registry plays the role of a directory service (domain). The service broker uses the rules contained in the SOA registry to perform its function as translator and delivery agent. For developers, the registry serves as the central location to store component descriptions that allow composite applications to be created and the place in which services may be published for general use.
These services in an SOA also include the provider interfaces and standard sets of network protocols that were mentioned previously. Developers may also choose to create a Business Process Orchestration module to coordinate the access and transactional integrity of multiple business applications that integrate into a larger platform, described in the next section in more detail.
FIGURE 13.6
This figure shows a network services model infrastructure for an SOA, which is based on the SOA meta-model of the Linthicum Group, 2007. A slightly different version of this diagram appears in Networking Bible by Barrie Sosinsky, Wiley, 2009.
Service catalogs
Finding any particular service and locating the service's requirement in a large SOA implementation can involve a large amount of network system overhead. To aid in locating services, SOA infrastructure often includes a catalog service. This service stores information on the following, among other things:
• What services are available, both internal and external
• How to use a service
• Which applications are related to a particular service (dependencies)
• How services relate to one another
• Who owns the service and how a service is modified
• The event history of a service, including service levels, outages, and so on
• The nature of service contracts
Service catalogs are dynamic and under constant modification. Catalog servers have these features:
• They can be standalone catalog servers serving a single site.
• They serve the role of a global catalog service where two or more catalog servers are merged to include several sites. A global service usually requires some sort of synchronization or update to maintain a unified data store across the servers involved.
• They can be part of a federated catalog service where two or more global catalog servers have access to one another's information through a trusted query relationship.
Catalog services have an enormous impact on large system performance and eventually become essential as a SOA internetwork system grows. An internetwork is a network that is constructed through the consolidation of separate networks, in the same manner that the Internet has been built.
Note
The Information Technology Infrastructure Library (ITIL) developed by the United Kingdom's Office of Government Commerce (OGC) has developed a Service Catalog Management design specification as part of the ITIL v2 Service Design Package (SDP). Service Designs present best practice guidance for planned services. The ITIL Web site may be found at http://www.itil-officialsite.com/home/home.asp.
Defining SOA Communications
Message passing in SOA requires the use of two different protocol types: the data interchange format and the network protocol that carries the message. A client (or customer) connected to an ESB communicates over a network protocol such as HTTP, Representational State Transfer (REST), or Java Message Service (JMS) to a component (or service). Messages are most often in the form of the eXtensible Markup Language (XML) or in a variant such as the Simple Object Access Protocol (SOAP). SOAP is a messaging format used in Web services that use XML as the message format while relying on Application layer protocols such as HTTP and Remote Procedure Calls (RPC) for message negotiation and transmission.
The software used to write clients and components can be written in Java, .NET, Web Service Business Process Execution Language (WS-BPEL), or another form of executable code; the services that they message can be written in the same or another language. What is required is the ability to transport and translate a message into a form that both parties can understand.
An ESB may require a variety of combinations in order to support communications between a service consumer and a service provider. For example, in WebSphere ESB, you might see the following combinations:
• XML/JMS (Java Message Service)
• SOAP/JMS
• SOAP/HTTP
• Text/JMS
• Bytes/JMS
The Web Service Description Language (WSDL) is one of the most commonly used XML protocols for messaging in Web services, and it finds use in Service Oriented Architectures. Version 1.1 of WSDL is a W3C standard, but the current version WSDL 2.0 (formerly version 1.2) has yet to be ratified by the W3C. The significant difference between 1.1 and 2.0 is that version 2.0 has more support for RESTful (e.g. Web 2.0) application, but much less support in the current set of software development tools. The most common transport for WSDL is SOAP, and the WSDL file usually contains both XML data and an XML schema.
REST offers some very different capabilities than SOAP. With REST, each URL is an object that you can query and manipulate. You use HTML commands such as GET, POST, PUT, and DELETE to work with REST objects. SOAP uses a different approach to working with Web data, exposing Web objects through an API and transferring data using XML. The REST approach offers lightweight access using standard HTTP command, is easier to implement than SOAP, and comes with less overhead. SOAP is often more precise and provides a more error-free consumption model. SOAP often comes with more sophisticated development tools. All major Web services use REST, but many Web services, especially newer ones, combine REST with SOAP to derive the benefits that both offer.
Contained within WSDL are essential objects to support message transfer, including these:
• The service object, a container where the service resides.
• The port or endpoint, which is the unique address of the service.
• The binding, which is the description of the interface (e.g. RPC) and the transport (e.g. SOAP).
• The portType, or interface that defines the capabilities of the Web service, and what operations are to be performed, as well as the messages that must be sent to support the operation.
• The operation that is to be performed on the message.
• The message content, which is the data and metadata that the service operation is performed on. Each message may consist of one or more parts, and each part must include typing information.
• The types used to describe the data, usually as part of the XML schema that accompanies the WSDL.
Tip
A set of primers on WSDL may be found on the W3C Web site. The latest version, “Web Services Description Language (WSDL) Version 2.0 Part 0: Primer” may be found at http://www.w3.org/2002/ws/desc/wsdl20-primer.
The code that follows is the WSDL 2.0 document that is created and analyzed in the W3C WSDL primer referenced in the accompanying tip.
<?xml version=”1.0” encoding=”utf-8” ?>
<description
xmlns=”http://www.w3.org/ns/wsdl”
targetNamespace= “http://greath.example.com/2004/wsdl/resSvc”
xmlns:tns= “http://greath.example.com/2004/wsdl/resSvc”
xmlns:ghns = “http://greath.example.com/2004/schemas/resSvc”
xmlns:wsoap= “http://www.w3.org/ns/wsdl/soap”
xmlns:soap=”http://www.w3.org/2003/05/soap-envelope”
xmlns:wsdlx= “http://www.w3.org/ns/wsdl-extensions”>
<documentation>
This document describes the GreatH Web service. Additional
application-level requirements for use of this service --
beyond what WSDL 2.0 is able to describe -- are available
at http://greath.example.com/2004/reservation-documentation.html
</documentation>
<types>
<xs:schema
xmlns:xs=”http://www.w3.org/2001/XMLSchema”
targetNamespace=”http://greath.example.com/2004/schemas/resSvc”
xmlns=”http://greath.example.com/2004/schemas/resSvc”>
<xs:element name=”checkAvailability” type=”tCheckAvailability”/>
<xs:complexType name=”tCheckAvailability”>
<xs:sequence>
<xs:element name=”checkInDate” type=”xs:date”/>
<xs:element name=”checkOutDate” type=”xs:date”/>
<xs:element name=”roomType” type=”xs:string”/>
</xs:sequence>
</xs:complexType>
<xs:element name=”checkAvailabilityResponse” type=”xs:double”/>
<xs:element name=”invalidDataError” type=”xs:string”/>
</xs:schema>
</types>
<interface name = “reservationInterface” >
<fault name = “invalidDataFault”
element = “ghns:invalidDataError”/>
<operation name=”opCheckAvailability”
pattern=”http://www.w3.org/ns/wsdl/in-out”
style=”http://www.w3.org/ns/wsdl/style/iri”
wsdlx:safe = “true”>
<input messageLabel=”In”
element=”ghns:checkAvailability” />
<output messageLabel=”Out”
element=”ghns:checkAvailabilityResponse” />
<outfault ref=”tns:invalidDataFault” messageLabel=”Out”/>
</operation>
</interface>
<binding name=”reservationSOAPBinding”
interface=”tns:reservationInterface”
type=”http://www.w3.org/ns/wsdl/soap”
wsoap:protocol=”http://www.w3.org/2003/05/soap/bindings/HTTP/”>
<fault ref=”tns:invalidDataFault”
wsoap:code=”soap:Sender”/>
<operation ref=”tns:opCheckAvailability”
wsoap:mep=”http://www.w3.org/2003/05/soap/mep/soap-response”/>
</binding>
<service name=”reservationService”
interface=”tns:reservationInterface”>
<endpoint name=”reservationEndpoint”
binding=”tns:reservationSOAPBinding”
address =”http://greath.example.com/2004/
reservation”/>
</service>
</description>
Notice the major elements of the document. The XML document sets up the namespace, defines the interface, specifies the binding, names the service, provides the documentations for the service, and then supplies a schema that may be used to validate the document. In the message descriptions, the message types are declared. XML schema can be separate files, but what we see here is normal for WSDL, an inline schema that is part of the WSDL document. For a much more complete step-by-step description, refer to the W3C tutorial.
A WSDL file contains essential message data for a transaction, but it doesn't capture the full scope of a Service Oriented Architecture design. Additional requirements need to be specified. The functional requirements for message passing between client and service in SOA are embodied in the concept of a service contract. A service contract codifies the relationship between the data to be processed, the metadata that accompanies that data, the intended service, and the manner in which the service will act upon that message.
Messages therefore must have some of the following pieces of information contained inside them:
• Header: The header contains the name of the service, service version, owner of the service, and perhaps a responsibility assignment. This is often defined in terms of a RACI matrix where the various roles and responsibilities for processes are spelled out in terms of a set of tasks or deliverables. The acronym designates the Responsible party or service, the Accountable decision maker, the Consulted party, and person(s) or service(s) that must be Informed on the use of the service.
• Service Type: Examples of service types include data, business, integration, presentation, and process types.
• Functional Specification: This category includes the functional requirements, what service operations or actions and methods must be performed, and the manner in which a service is invoked or initiated. Invocation usually includes the URL and the nature of the service interface.
• Transaction attributes: A message may define a transaction that may need to be managed or tracked or be part of or include another transaction operated at a specific Quality of Service and under a specific Service Level Agreement (SLA). Security parameters also are part of a transaction's attributes, as are the role the message plays in a process and the terms or semantics used to describe the interaction of the message with a service's interface.
Depending upon the degree of formalization a service contract may require messages to carry a variable amount of information in order to be successfully transacted. You can see, therefore, how SOA expands the definition of a Web service transaction from WSDL.
Business Process Execution Language
If a message represents an atomic transaction in a Service Oriented Architecture, the next level of abstraction up is the grouping and managing of sets of transactions to form useful work and to execute a business process. An example of an execution language is the Business Process Execution Language (BPEL) or alternatively as the Web Service Business Process Execution Language (WS-BPEL), a language standard for Web service interactions. The standard is maintained by the Organization for the Advancement of Structured Information Standards (OASIS) through their Web Services Business Process Execution Language Technical Committee (WSBPEL-TC; see http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsbpel.)
BPEL is a meta-language comprised of two functions: executable commands for Web services and clients, and internal or abstract code for executing the internal business logic that processes require. A meta-language is any language whose statements refer to statements in another language referred to as the object language. BPEL is often used to compose, orchestrate, and coordinate business processes with Web services in the SOA model, and it has commands to manage asynchronous communications.
BPEL uses XML with specific support for messaging protocols such as SOAP, WSDL, UDDI, WS-Reliable Messaging, WS-Addressing, WS-Coordination, and WS-Transactions. BPEL also builds on IBM's Web Services Flow Language (WSFL) and Microsoft's XLANG for data transport; the former is a system of directed graphs, while the latter is a block-structured language adding to additional verbs and nouns specific for business processes to BPEL, which were combined to form BPEL4WS and are being merged with BPEL. A version of BPEL to support human interaction is called BPEL4People, and it falls under the WS-HumanTask specifications of OASIS.
BPEL was designed to interact with WSDL and define business processes using an XML language. BPEL does not have a graphical component. A business process has an internal or executable view and an external or abstract view in BPEL. One process may interact with other processes, but the goal is to minimize the number of specific extensions added to BPEL to support any particular business process. Data functions in BPEL support process data and control flow, manage process instances, provide for logic and branching structures, and allow for process orchestration. Because transactions are long-lived and asynchronous, BPEL includes techniques for error handling and scopes transactions. As much as possible, BPEL uses Web services for standards and to assemble and decompose processes.
Business process modeling
SOA was created by the industry to solve a problem: how to make disparate, diverse, and distributed services talk to disparate and diverse clients. The final result of an SOA project isn't the access of services, per se; it is the creation of a business process. In a complex business project, the developers juggle many clients and many services, which can make visualization of the overall system difficult. To address this problem, various modeling tools have been developed to support SOA development and optimization, system and process management, change and life-cycle management.
Several methodologies have been developed to model SOAs. Working in a software package to model your business processes is similar in approach to designing and optimizing a relational database in entity-relationship, object-role modeling package, or another Computer Aided System Engineering (CASE) tool for data storage—and equally as valuable.
Commonly encountered system models include the following:
• Unified Modeling Language (UML): The UML standard is the work of the Object Management Group (http://www.omg.org/). UML creates graphical representations of software systems in the form of a set of diagram types. Elements in a UML architectural blueprint include actors, business processes, logic modules (components), program routines, database schemas, software components, and activities. UML diagrams are separated into seven structural types and four behavior types; structure types model the components of the system, while behavior types model states, actions, and events. UML is widely used in the industry for software system modeling. A developed system model can be reduced automatically to code.
• XML Metadata Interchange (XMI): XMI is another standard of the Object Management Group (OMG) and is used to exchange metadata using the Extensible Markup Language (XML). Metadata is structured into a metamodel that fits into the OMG's Meta-Object Facility. UML models often use XMI as their interchange format, although they can be used by other languages. XMI files are not generally interchangeable between the different modeling languages that can use them. XMI has been codified as an international standard by ISO, as IS0/IEC 19503:2005.
• Systems Modeling Language (SysML) is an open-source extension of the part of the UML system dealing with profiles. It is smaller, more focused, and easier to learn and work with than UML itself. SysML reuses 7 of UML 2.0's 13 diagrams. The effort to develop SysML was rolled into OMG in 2008, but remains open source. SysML can use XMI and is developing toward support of ISO 10303, which is the Standard for the Exchange of Product Model Data (STEP) AP-233. STEP aims to create the mechanisms for sharing information between the different software engineering tools described in this section (and others).
• Business Process Modeling Notation (BPMN) is a methodology for representing business processes as a set of connected visual objects that illustrate workflow in a Business Process Diagram (BPD). It is similar to UML. Originally developed in the Business Process Management Initiative (BPMI), it was incorporated into the Open Management Group in 2005. A BPD can be reduced to the OASIS standard WS-Business Process Execution Language, which is an executable language for information transfer between different Web services. However, this mapping is tool specific and standardized at this point.
• Service-Oriented Modeling Framework (SOMF): This framework was proposed by Michael Bell and combines a modeling language with a graphical display of the various SOA components so the system can be viewed as a map of objects and associated relationships. SOMF software allows developers to create an action plan to implement their business processes and can be valuable in system and architecture optimization, tracing message pathways, positioning software assets correctly, and providing a language for describing through abstraction and generalization how the processes operate. SOMF software not only allows you to determine what needs to be done, but also allows you to run “what-if” scenarios to see how changes will impact your SOA system.
Tip
The Enterprise Architect (EA) offered by Sparx Systems (http://www.sparxsystems.com/) is an example of a unified modeling language tool that supports software systems modeled in these various languages.
A well-known SOMF modeling technology is IBM's Service-Oriented Modeling and Architecture (SOMA), introduced in 2004. SOMA reduces services to a set of service objects and breaks down relationships into three components: the services themselves, the service components that make use of those services, and the information flows required to interact between them. Flows consist both of processes as well as their internal composition. In SOMA, domains and functional groups are identified, variables that affect processes are analyzed, and component development and object oriented analysis are used to model specific cases. SOMA is meant to provide information on service and service boundaries, service granularity, and asset analysis.
In SOMF, the modeling is based on the elements of the service life cycle:
• Conceptualization: Defining the processes that will be supported
• Discovery: Determining how components are exposed
• Analysis: Identifying the relationships services have to clients
• Integration: Implementing the messaging infrastructure that connects clients to components
• Logical design: Building the system and process logic and determining how orchestration or choreography will be performed
• Architecture design: Creating the system component architecture and reducing design to software components with specified interfaces
• System implementation: Building and testing the system components
Figure 13.7 shows the various practices brought together in the SOMF modeling discipline.
Four modeling topologies for message passing are used in SOMF:
• Circular topology: A circular topology is one where message passing is carried out in a circular fashion. There is no orchestrator in this system, and each component providing a service is responsible for knowing which message to act on and where to send a message next. That is, the choreography of the system is maintained at the component.
• Hierarchical topology: In a hierarchical topology, services are arranged in a tree pattern with parent/child relationships. Messages from one service to another must traverse up the branch of the tree and down another branch from the root until the matching service is found. Hierarchical topologies offer the advantage of a well-defined set of relationships, a central location (the root) where logic may reside, and a clearly stated address space. The disadvantage of a hierarchical topology is that the overhead of passing a message from one service to another isn't optimized.
• Network topology: A network topology has a many-to-many relationship between services and their clients. The advantage of a network topology is that the overhead associated with message passing has been minimized (often there is a direct path), but there is considerable overhead built into the system in order to maintain the many links needed.
• Star topology: In a star topology, services are designed to connect through a central service. The star model is favored in orchestration processes and is useful for services that use broadcasting or multicasting services, publish and subscribe, and other related systems.
FIGURE 13.7
SOMF 2.0 practices, principles, and design methodologies
Figure 13.8 shows an interaction diagram of these four different topologies.
A service in SOMF that is granular and very narrow in scope is referred to as an atomic service. An atomic service cannot be decomposed into smaller services that provide a useful function. An example of an atomic service is a customer lookup that contains a customer ID and the customer's name and address. An address that doesn't contain the customer's name or lists a customer name without an ID wouldn't have much value, so this service stands alone. In Figure 13.8 an atomic service would be represented by one of the circles in the diagrams.
FIGURE 13.8
The four different message-passing topologies used in SOMF are shown above. The lines and arrows indicate message-passing pathways and relationships.
Rarely do atomic services stand alone. A service that provides ID, name, and address might be part of a set of services that describe a bank account, customer purchase record, or any number of services. A collection of services that work together is referred to as a composite service. The Enterprise Service Bus described previously in this chapter is another example of a composite service; it contains functions for message routing, message interchange and translation, and process orchestration. A composite service is usually organized as a hierarchical topology and is multi-functional or a coarse-grained entity.
SOA describes a distributed collection of services performing business process functions. A collection of composite services that would form a process module is referred to as a service cluster. Service clusters may be composed of both atomic services and composite services. Large functions such as payroll modules would normally be composite SOA services.
Tip
To view a structured presentation on how to build an SOMF diagram, go to http://www.modelingconcepts.com/pdf/SOMF_ANALYSIS_MODELING.pdf. The white paper “Enacting the Service Oriented Modeling Framework (SOMF) Using Enterprise Architect” by Frank Truyen (http://www.sparxsystems.com/downloads/whitepapers/EA-SOMF_Introduction.pdf) shows different SOMF diagram types.
In an SOMF model, each of these three service types (atomic, composite, and clusters) appears as a specific shape, and connections are made between them that generalize, specify, expand, or contract the services they provide. Services are typed, granular services are identified, and then services may be aggregated, decomposed, unified, intersected, or subtracted from other services to suit the needs of the business process being modeled. The SOMF modeling notation has a symbol for each analysis that relates one service to another. As you build a business process, you add services to the model and connect them in ways that make sense for your workflow. When the model is complete and optimized, it is reduced to a conceptualized service that relates the business process to the specific implementation chosen. Some modeling technologies allow for the reduction of the model to executable code.
Managing and Monitoring SOA
Software for monitoring and managing an SOA infrastructure plays an important role in large SOA deployments. While SOA offers a logical design and reusable components, it does not make the task of network management any easier. If anything, SOA management requires proactive oversight because you can't wait for a particular application to fail before taking corrective action. Therefore, tools for managing SOAs tend to be multifaceted and run constantly.
SOA management tools
There are a number of network management frameworks products and suites, notably these:
• HP Software and Solutions OpenView SOA Manager (https://h10078.www1.hp.com/cda/hpms/display/main/hpms_content.jsp?zn=bto&cp=1-10^36657_4000_100)
• IBM Tivoli Framework Composite Application Manager for SOA (ITCAM; see http://www-01.ibm.com/software/tivoli/solutions/),
• Oracle BPEL Process Manager (http://www.oracle.com/technology/bpel/index.html)
These products have SOA tools for network management. IBM's product specializes in change management and SOA lifecycle development, and it integrates with a WebSphere and other Tivoli systems. HP SOA Manager provides dynamic mapping, monitoring, and optimization of SOA services such as Web services, software assets, and virtual services. These framework products create a central console with a variety of management views. Oracle's BPEL Process Manager and WebSphere are process managers for creating an Enterprise Service Bus.
The SOA management software technology is dynamic, with many small vendors' products some of which have been purchased and rolled into (or are being rolled into) larger systems. Oracle's recent acquisition of AmberPoint's SOA Management System is an example of this trend. BMC Software's AppSight (http://www.bmc.com/products/product-listing/BMC-AppSight.html) is an automated SOA problem-resolution package, as is Tidal Software's Intersperse package, which has root cause analysis services. The CA Wily SOA Solution (http://www.ca.com/us/eitm/solution.aspx?id=8254) is a monitoring and discovery service that can map SOA transactions and dependencies and discover components such as ESBs, Web portals, and various Web services. iTKO's LISA (http://www.itko.com/products/index.jsp). Enterprise SOA Testing platform specializes in testing Web service components that are used in SOA. Another example of an SOA transaction manager is OpTier's CoreFirst (http://www.optier.com/corefirst_overview.aspx).
Configuration and change management present a particular challenge in the area of SOA (and cloud computing in general). In addition to the fact that elements of an SOA infrastructure can be highly distributed and therefore require good discovery mechanisms, these environments also are highly virtualized. As workloads vary, solutions often provision virtual servers as needed and move these virtual servers' processing across physical servers. Virtualization will continue to challenge SOA management software well into the future.
SOA security
Any system that sends hundreds or thousands of messages across an internetwork as SOA does is subject to attack in all the traditional ways that network traffic is hijacked, spoofed, redirected, or blocked. Because SOA eliminates the use of application boundaries, the traditional methods where security is at the application level aren't likely to be effective.
Cisco has a family of products that enforce rules and policies for the transmission of XML messaging that they have named Application Oriented Networking (AON; http://www.cisco.com/en/US/products/ps6480/). A similar policy based XML security service may be found in Citrix's NetScaler 9.0 (http://www.citrix.com/English/ps2/products/product.asp?contentID=21679) Web application delivery appliance.
To address SOA security, a set of OASIS standards (http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=security) was created, which includes the following:
• Security Assertion Markup Language (SAML) is an XML standard that provides for data authentication and authorization between client and service. The SAML technology is used as part of Single Sign-on Systems (SSO) and allows a user logging into a system from a Web browser to have access to distributed SOA resources.
• WS-Security (WSS) is an extension of SOA that enforces security by applying tokens such as Kerberos, SAML, or X.509 to messages. Through the use of XML Signature and XML Encryption, WSS aims to offer client/service security.
• WS-SecureConversion is a Web services protocol for creating and sharing security context. WS-SecureConversion is meant to operate in systems where WS-Security, WS-Trust, and WS-Policy are in use, and it attaches a security context token to communications such as SOAP used to transport messages in an SOA enterprise.
• WS-SecurityPolicy provides a set of network policies that extend WS-Security, WS-Trust, and WS-SecureConversion so messages complying to a policy must be signed and encrypted. The SecurityPolicy is part of a general WS-Policy framework.
• WS-Trust extends WS-Security to provide a mechanism to issue, renew, and validate security tokens. A Web service using WS-Trust can implement this system through the use of a Security Token Service (STS), a mechanism for attaching security tokens to messages and a set of mechanisms for key exchanges that are used to validate tokens and messages.
Another approach to enforcing security in SOA is to use an XML gateway that intercepts XML messages transported by SOAP or REST, identifies the source of the message, and verifies that the message was securely received. Providing XML Gateway SOA security requires a Public Key Infrastructure (PKI) so that encryption is enforced by digital signatures. Progress Software's Actional 8.0 (http://web.progress.com/en/actional/index.html) now includes Mindreef's SOAPscope Server and has an XML middleware service that performs diagnostic testing and Web services governance, adding that component to Actional's ability to monitor and map XML appliances and application servers.
The Open Cloud Consortium
The Open Cloud Consortium (OCC; see http://opencloudconsortium.org/) is an organization comprised of several universities and interested companies that supports the development of standards for cloud computing and for interoperating with the various frameworks.
OCC working groups perform these functions:
• They develop benchmarks for measuring cloud computing performance. Their benchmark and data generator for measuring large data clouds is called MalStone (http://code.google.com/p/malgen/).
• They provide testbeds that vendors can use to test their applications, including the Open Cloud Testbed and the Intercloud Testbed that are part of the work of the Open Cloud Testbed and Intercloud working groups.
• They support the development of open-source reference implementations for cloud computing. The Working Group on Standards and Interoperability For Large Data Clouds extends the architecture for data storage with a distributed file system, table services, and computing using MapReduce following the model that is part of Google's offering.
MapReduce is Google's patented software framework that supports distributed large data sets organized by the Google File System (GFS) accessed by clusters of computers. The Apache Hadoop (http://hadoop.apache.org/) open-source system is based on MapReduce and GFS.
• They support the management of cloud computing infrastructure for scientific research as part of the Open Science Data Cloud (OSDCP) Working Group's initiative.
Relating SOA and Cloud Computing
Cloud computing is still in its infancy, and although Web services can implement a Service Oriented Architecture, it is not a requirement. Most of the large implementations of cloud computing described in this book are single-purpose applications that have been optimized on a grand scale: Carbonite's backup, Google's Gmail e-mail, and Twitter's Instant Messaging (IM) are several examples. Applications of those types have less of a need for the flexibility and loose coupling that SOA provides. As cloud applications become more diverse in scope, SOA offers an architectural blueprint for accessing diverse optimized services through a loosely coupled standardized method that provides an ability to evolve that is difficult to implement in any other way.
SOA is loosely coupled because the service is separated from the messaging. If a component doesn't provide the capabilities required, it is an easy task to switch to a different component, and switching requires almost no programming. Developers lose some of the ability to customize modules, but gain a significant advantage in simplifying their applications. Taken as a whole, applications that rely on SOA components can be very complex and appear to be tightly coupled, when in reality they are not.
SOA components are often best-of-breed service providers that can provide a measured service level and can play a role in Business Process Management (BPM) systems. The separation of services from their design allows for much easier system upgrades and maintenance.
Many Web 2.0 applications use SOA components, and SOA will become increasingly useful in larger applications that require many Web services. Web 2.0 is an acronym coined by Tim O'Reilly to describe Web services that allow for user input and modification. These applications often rely on REST and feature AJAX components in a user interface that supports Web syndication (think of the Google customizable user page), blogs, and wikis. Some people regard mashups as Web 2.0 applications as well. A mashup is the combination of data from two or more sources that creates a unique service. The layers added to Google maps are examples of mashups.
AJAX stands for Asynchronous JavaScipt and XM. AJAX is a set of development tools that allow for client input into Web applications; it is not a standard. Rather AJAX describes a group of technologies that leverage HTML and CSS for styling, Web objects in the Document Object Model (DOM) for data, XML and XSLT for data interchange, the XMLHttpRequest for asynchronous communication, and JavaScript commands to request data from data sources.
The challenge SOA faces in designing systems to support Web 2.0 is the lack of standardization in how components in Web 2.0 are used. However, many people believe that SOA will play a role in creating what has been dubbed an “Internet of Services” where complex services will be available for use as a set of building blocks based on the convergence of SOA and Web 2.0. The Gartner Group refers to this trend as the development of “Advanced SOA,” but features of SOA that are event-driven have been part of many vendors' middleware offerings for several years now.
Summary
This chapter described Service Oriented Architecture (SOA). SOA offers a design methodology for creating distributed applications using diverse clients and components. SOA defines a message-passing infrastructure from clients or consumers to and from service providers. Making SOA work correctly requires a certain set of middleware products in your infrastructure. These servers may aid in transaction management or brokering, message translation, or other services. Taken as a whole, these services are referred to as an Enterprise Service Bus (ESB).
Most message-passing protocols are based on a version structured XML, although that is not required in SOA. A variety of transport protocols are used, but SOAP and RPC are the most common ones. The nature of SOA messaging was explored. In a complex system of message passing and services, system management and security is an important consideration. Tools for setting up and running an SOA infrastructure were described.
Finally, this chapter described the relationship between SOA and cloud computing. You don't need to use SOA to build a massively scaled cloud computing application, but as cloud computing applications become more capable and user configurable, the logic and structure that SOA design imposes on infrastructure will prove to be invaluable to cloud applications. The two areas of technology benefit from their mutual convergence.
In Chapter 14, I consider the topic of transactional Web applications in cloud computing systems. That subject builds on what you have learned about SOA in this chapter, extending the discussion into the command Web APIs that are in use today.