In Depth: What You Need To Know About SOA Management Suites

Managing a SOA isn't for the faint of heart. Services crop up like mushrooms and comprise multiple touch points (operations) that may require different policies based on the person or application using the service. Forget conventional methods of managing and monitoring Web resources--they can't offer visibility into the operation being executed within a service. Most are capable only of recording and monitoring URIs, and log-culling products simply cannot provide a transactional view of service use for your service-oriented architecture.

You need an application that can monitor services at the operation level, perform authentication and authorization, apply security policies that safeguard the privacy of data, conform to regulations such as HIPAA and Sarbanes-Oxley, and ensure availability of services using SLA management and enforcement. Oh, and it must also integrate into your existing architecture.

Traditional APM (application performance monitoring) models in the form of agents residing on enterprise service platforms, such as BEA Systems' WebLogic or IBM's WebSphere, don't fit the bill because they haven't evolved along with SOA. APM agents are focused on URIs and don't speak XML or SOAP, both must-haves to collect metrics and apply policies accurately based on XML-specific standards and best practices, such as encryption, data transformation and monitoring at the operation level. They're also not proficient at enforcing policies that may modify requests and/or responses, nor can they perform authentication and authorization. Give 'em the boot.

Play Reveille for Me

If you're squirming because we just described your SOA management strategy, wake up and smell the mess hall coffee. Web services are not only not conventional Web-based resources, they aren't even always Web-based. Many Web services, especially those exposed by middleware, such as ESB (enterprise service bus) suites, are accessed using messaging protocols like JMS (Java Messaging Service) and not through HTTP, making life difficult for some SOA management products--and leaving many IT pros shaking their heads at the erroneous use of "Web services" to describe services within a SOA. JMS (Java Messaging Service) advocates are particularly irritated at the misnomer and plan to launch a counterstrike to rename all SOAP transports "messaging services." You heard it here first.

If you're serious about SOA, you need the right weaponry to keep services under control. But in our reader poll for this article, only 33 percent of respondents said a management product was in their future; 26 percent said they were undecided.

Let us help: If your SOA strategy doesn't include a SOA management product and you expect to implement more than 25 services, rethink your strategy. If you need to enforce service-level agreements on those services, rethink your strategy. And if your organization requires last-mile security for services, you really need to rethink your strategy; APM products simply can't provide the fine-grained security and SLA enforcement required for SOA deployments. And you must make the leap sooner rather than later. Putting management products into a service after the fact adds complexity and increases the time to complete a deployment.

Although you can employ code-based security and some facets of management within a service, managing code when faced with multiple access scenarios that address the differing needs of various types of uses can actually degrade performance. Worse, it destroys the ability to quickly adapt to changing business needs, such as welcoming new partners, applications or users. SOA management suites address these needs neatly by placing the onus on external systems that are readily modified, simply by changing or creating policies that can be applied in hours rather than weeks.

The market is ready; SOA management products have been evolving and consolidating for several years. We brought two SLA management suites, Actional's Looking Glass and SOA Software's Service Manager, into our NWC Inc. business applications lab in Green Bay, Wis. (see "NWC Reports: SOA Management" below, for highlights and go to www.nwcreports.com for our complete analysis). We dug beyond the bells and whistles of real-time monitoring and eye-candy reports, and found that both are ready for prime time. The interoperability issues, poor standards compliance and immature implementations of SLA enforcement and notification mechanisms we saw two years ago have been resolved. The field also has narrowed considerably, with the remaining players in this space evolving through consolidation with security players (Actional-Westbridge), partnerships (DataPower-CA), and acquisitions (SOA Software-Flamenco Networks/Blue Titan Software). In this case, a smaller field really is better.

There's still room for improvement, of course. Most products remain light on enterprise-class alert and notification mechanisms, choosing instead to integrate with corporate-standard network and systems management products, such as CA Unicenter, HP OpenView and IBM Tivoli. For smaller organizations, the alert and notification systems of SOA management products are likely good enough. Policy management needs work across the board, but it's not so poorly implemented as to be a show-stopper. In fact, SOA Software's Service Manager does an excellent job of distributed policy management, and the version control in Actional's SOAPStation is an excellent example of the maturity in this arena.

What we aren't seeing yet is consistency in depth and breadth of feature sets. Service Manager's version control isn't as mature as that of Actional and Reactivity, but Actional's policy management isn't as centralized as SOA Software's implementation. However, products are stable, and their core functions--managing and monitoring services--are ready for battle.

The cost of a SOA management suite--an average of $120,000 for our NWC Inc. scenario--and the complexity of implementation mean the decision to purchase one of these puppies must be a collaborative one between IT and business users. That's the case for readers responding to our poll for this package: 57 percent say Web services technology purchases are initiated jointly.

The Business Perspective

One benefit of a SOA is that it offers business users a view into processes that, in the past, have been relegated to basement-dwelling admins who may or may not understand business priorities. A SOA management suite is not going to give business users control over your SOA. But they can, and do, offer visibility into the services in a business-friendly manner, much like business activity monitoring offerings. BAM suites monitor transactions from a business view by digging into the message and pulling out business-relevant details, like stock levels or order totals. Although SOA management products don't yet provide the eye-candy-laden executive dashboards found in BAM, they can monitor and route messages based on the same business-relevant details, making them a perfect complement to a full-blown BAM implementation.

SOA management products also let IT enforce SLAs based on SLOs (service-level objectives) or in conjunction with business metrics, such as the value of a customer. If given the ability to determine which to service first, a purchase order for $10,000 or one for $100, we want to choose the former. This type of SLA enforcement is flexible and more dynamic than typical techniques because it can be based on the content of the message instead of less-business-oriented values, such as response times and bandwidth metering.

Basic SLA policies can be created and enforced in all SOA management products, but very few give business owners a view of services that makes sense to them--all too many present data using operational IT jargon. The ability to disseminate information in layman's terms is unique to Actional's Looking Glass, which comes as close to a poor-man's BAM suite as we've seen. Business users view services relevant to their customers or processes and define key performance indicators that can be culled from messages and presented as "Top 10" views of business metrics. Actional calls these "key business indicators," but we suspect that terminology will change in the future to better align with business terminology.

SOA Software's Service Manager uses JSR-168-compliant portlets to provide business users with real-time views of service performance via any JSR-168-capable portal. (JSR-168 is a standard for portlets that goes hand-in-hand with WSRP (Web Services for Remote Portlets) to make it easy to integrate portlets into enterprise portals.) These views, like those in competing products, are driven by Flash interfaces that update every four or five seconds and provide a constant view of operational performance metrics.

Indeed, dashboard views are becoming more ubiquitous as business users increasingly demand such visibility. Of course, there isn't a darn thing business users can do about problems they find using their flashy interfaces, but at least they'll have the knowledge they need to proactively deal with customers who may be experiencing poor performance, while IT resolves the issue.

The IT Problem

IT faces two main hurdles when attempting to manage the many services being deployed across the enterprise: enforcement of operational policies and monitoring performance. Policies need to be applied specifically to operations (functions) instead of at the service endpoint. Although general organizational policies regarding user authentication and authorization may be feasible at the service level, avoid applying policies that require digital signatures or element-level encryption on all services. The negative impact on performance will be unacceptable. Of course, including a SOA management product in your SOA strategy makes applying security features at the service level possible and should be considered a "must have."

The first hurdle is high because IT must consistently enforce operational policies across all services, regardless of where they may be used. Composite applications, BPM (business process management) suites, ESBs, even Excel spreadsheets are just a few possible points of entry; IT must somehow ensure, regardless of an end user's method of access, that the user is authorized to use the service and that policies regarding data security are enforced--a task that's growing increasingly difficult as policies may differ based on the user. Coding authentication and authorization into the application is a stop-gap solution, one that doesn't scale well or support the goals of agility and loosely coupled systems.

Adding support for new standards, such as WS-Policy, is also more difficult if IT has to recode an application. Agility isn't just about the business being able to react quickly to changing market conditions; IT must also be agile to support changes introduced by new standards or operational policies. Placing support for SLAs, security, policy enforcement, authentication and authorization into a SOA management system--whether it be agent- or proxy-based--gives IT the flexibility to rapidly address changing business needs simply by creating a new policy.

The second problem facing IT is that of monitoring the complex network of services that make up a SOA. As your SOA expands and the operations associated with a given service multiply, it becomes more difficult to ferret out where performance problems are occurring. This is primarily because, in a well-architected SOA, there is a many-to-one relationship among operations and services, much like the relationship between applications and the application server. One application may be performing well, but you wouldn't assume that others on the same server are also in good shape. Likewise, you can't assume that because one operation in a service shows acceptable performance that others in the same service will act the same. You need a system that can monitor at the operational level to pinpoint issues.

Managing a SOA also requires discovery of services, which can be a full-time task in the face of massive SOA initiatives. Agent-based offerings are particularly helpful here. They provide what vendors like to call "autodiscovery" of services, eliminating the need for IT to find and enter services manually into the central SOA management server. The ability of products, like those from Actional and SOA Software, to automatically discover--and then bring under management--services residing in a variety of application servers helps shorten implementation time and makes introduction of errors less likely. Service Manager excelled in this portion of our testing thanks to its ability to apply a default policy to all services automatically when they're discovered and brought under management, something Looking Glass couldn't do because of its less centralized policy-management architecture.

Build Me Up

Most APMs rely on agents to manage and monitor applications in a distributed environment. SOA management suites began as pure proxy-based systems but are evolving to support both proxies and embedded agents. The products from AmberPoint, Actional and SOA Software all use a hybrid management architecture (see "Hybrid SOA Management Architecture," at right with both proxies and agents, both of which are commonly referred to as "enforcement points" or "management points," depending on which product you're using. In all cases, proxies actually contain an embedded agent, and are generally deployed in either a Jetty, JBoss or Tomcat container in an automated installation. Regardless of terminology, an enforcement point is a specific instance of an agent capable of enforcing policies, monitoring traffic and reporting back to the central console.

Today, market differentiators primarily revolve around how well products manage large numbers--from tens to potentially hundreds--of these enforcement points The last thing an administrator wants to do is individually configure each instance, so centralized management of both policies and agents is paramount to reducing costs. Here, SOA Software is well ahead of Actional. Although some aspects of policy management could be centrally managed using Actional's Looking Glass, core configuration of policies was accomplished by modifying policies on the distributed agents directly. SOA Software's Service Manager, by contrast, let us define policies within its centralized console, and policies were automatically pushed to enforcement points.

Support for agents on enterprise service platforms like those from BEA, IBM and Microsoft varies. Some SOA management products, including Looking Glass, can instrument these platforms across a number of versions, while some, like Service Manager, are more limited and support only specific versions of these platforms. See our analysis of the impact of agents versus proxies on the left.

ESBs in the Cold

Those concerned with the management requirements of their ESBs must wait a while longer before embedded agent support is available. Most ESBs lack comprehensive security and monitoring components, which means visibility and enforcement of policies within orchestrated services is lost. Actional (now owned by Progress Software, which also happens to own Sonic Software) told us that Sonic's ESB 7.0 will be the first ESB supported by Actional, with others to follow. No surprise there. SOA Software is working closely with BEA to deliver embedded agents for AquaLogic Service Bus. But that leaves customers of ESB vendors--such as Cape Clear Software, Fiorano Software, IBM, Oracle and TIBCO Software--with fewer options. They'll be able to manage the services that are controlled by these ESBs over management proxies, but ESBs are already mediators, so inserting an additional proxy into the chain is less desirable than direct support through an embedded agent.

Other SOA management vendors, like Reactivity, are focused on managing the disparate transports used to carry SOAP messages through the enterprise: JMS, RMI and MQ. Although Actional's product can monitor resources, such as JMSs and EJBs (Enterprise JavaBeans), in a JavaEE container, it can't directly manage these resources if they aren't within a managed container. Reactivity, on the other hand, can interface directly with these types of resources and therefore act not only as a service enablement platform--for example, layering a SOAP interface over a conventional data source, such as a database or legacy application--but as a management point for enforcing policies regarding SOAP messages being delivered to messaging layers.

This particular area of market development has become a line of demarcation between diverging product sets: Reactivity and DataPower are in one camp, while Actional, AmberPoint and SOA Software are on the other side of the road, focusing more on providing such features as business monitoring and dashboards, rather than integration with the corporate application infrastructure. This may change with SOA Software's recent acquisition of Blue Titan, whose capabilities in the areas of routing and protocol mediation, including the ability to automatically transform messages from one protocol to another or to accept a SOAP message and send it to a back-end service as a JMS message, for example, could be what SOA Software needs as it moves to provide SOA integration as well as management.

The convergence of conventional integration with management makes sense. Most integration capabilities have been commoditized in recent years, and the ability to provide both service enablement and policy enforcement in a single product is a boon as it simplifies both architecture and management.

If you need to go with a proxy implementation, as we had to in NWC Inc. to support our services residing on BEA's AquaLogic Service Bus 2.1, you must make modifications at the service orchestration or application layer. To take advantage of the features provided by SOA management proxies, you must proxy all your services through the system--including those that might be called from within other services. Although you could gain some functionality by directing clients to access a composite service (a service that calls other services) through a SOA management proxy, you won't benefit from visibility into transactions and the ability to apply policies to services being called directly from within the composite service.

This is perhaps the most compelling argument for agents. Agents immediately provide these capabilities without changing a thing in the application layer. Obviously, routing all requests through a proxy-based enforcement point increases complexity, making development and deployment of services more difficult. Easing this complexity for developers is, in part, the role of a SOA registry/repository. All SOA management products integrate--with varying degrees of depth--with these registries/repositories. SOA Software includes its own UDDI registry, but it should not be confused with a full registry/repository product, such as those offered by Systinet (Mercury) or Infravio. We have several of these registries in our lab right now; watch for reviews and analysis in our Aug. 3 issue. A comparison of a composite service before and after the introduction of a management proxy is at left.

High On Visibility

The reasoning behind SOA Software's more restrictive version-based agents lies, in part, in its architectural decision to implement agents at both ends of the application stack. It does this so agents can use WS-Addressing as a mechanism to more completely track interdependencies among services. Correlation of service relationships is vital when performing dependency analysis, especially in terms of impact and root-cause analysis of performance issues in a complex SOA implementation. One issue that has dogged the network management arena is the accuracy of interdependency audits. In our tests on the SOA side, we found SOA Software's mappings very accurate. Actional was a bit less precise--we weren't absolutely sure who was calling whom--but even so, it was better than nothing.

While competitors place agents as servlet filters or Java interceptors only at the top of the stack, SOA Software's Service Manager also implements what it calls a "SOA Context" that intercepts outgoing service invocations and, when indicated by the deployed policy, adds the appropriate WS-Addressing headers, which are later used to correlate service interrelationships. Although Actional's Looking Glass correctly rendered traffic flows between services residing on different servers, through its interceptor-based agents, it couldn't definitively represent relationships between those services, for example, present a hierarchical invocation representation of messages traveling between services.

A pure proxy-based approach, such as that offered by IBM/DataPower or Reactivity--or when using only the proxy capabilities of Service Manager and Looking Glass--cannot determine relationships between services because proxies have no visibility beyond the services they are managing. To provide that visibility it's necessary to instrument the application to consume and insert additional headers, like those provided by WS-Addressing, or require that all service invocations be routed through the proxy. This tracking could ostensibly be accomplished by routing all service invocations back through the proxy and inserting the appropriate headers, but this will require a good understanding of WS-Addressing and XSLT and the ability to extract headers from messages using XPath. Because the point of using an external entity to manage services is to remove this onus from developers, we strongly suggest the use of embedded agents rather than proxies wherever possible to achieve the highest return on investment.

Performance Art

The decision to use a proxy rather than an agent may also have performance implications. Pure software proxies like those from Actional, AmberPoint and SOA Software can't compete in terms of performance with the likes of the products from DataPower and Reactivity, which take advantage of custom-built ASICs to accelerate XML parsing and processing. IBM's DataPower uses its own silicon, while Reactivity is firmly in the Tarari OEM camp, something we see becoming more and more common as Tarari continues to evolve its custom content processors.

Proxy implementations from Actional and SOA Software use Jetty and Tomcat, respectively. This impacts overall performance because both are limited to the TCP and HTTP processing limits of the platforms on which they are deployed. We made comparisons of response times for the same service using both proxies and agents. The overhead hit of the additional TCP session and XML processing required by a proxy negatively impacted performance--response time was nearly 50 percent higher when using proxies as when we took advantage of embedded agents!

Moreover, if you're in a highly security-conscious organization that implements end-to-end security of transported messages via SSL, the use of proxies will introduce additional hurdles into your SOA network. SOA enforcement points require visibility into the payload of the message, for example, and therefore cannot process SSL-encrypted traffic without extra overhead. The proxy would need to decrypt the message, apply the appropriate policy, then re-encrypt the message before sending it on to the intended receiver. This requires access to certificates and keys; that introduces an additional security risk on top of higher message latency.

What Price Scalability?

Agents win when we factor in performance, but they lose points when considering the cost of scaling SOA management horizontally. Not only must you factor in the cost of the additional enforcement point--a common pricing model for SOA management products--you need to add licensing of the application server instance. This may or may not be significant, depending on your licensing agreements and the flavor of the application server in question; we've seen a range of $2,000 per CPU to $20,000 per CPU.

When deploying additional proxies, the cost is reduced to just the additional enforcement point because the underlying application server instance is distributed at no charge. Hardware costs are a wash regardless of whether you go with a proxy or an embedded agent in an application server. As your SOA implementation grows, this is a serious consideration and one that should be considered as part of the ongoing expense of managing your SOA. Some proxies can be deployed in an existing app server, but that doesn't make sense in the case of SOA management--the reason you'd deploy a proxy is because there is no agent to support the particular version of the app server you have, and in that case, it's unlikely that you could deploy the proxy on the app server, or the product would have an agent. Remember, too, that a proxy becomes a single point of failure in your SOA network. You'll almost certainly want to deploy a second proxy as backup to ensure availability.

Securing the Last Mile

Security and compliance are taking ever larger bites of the IT budget. Some industries, notably finance and health care, require encryption not only of external communications, but internal "last mile" communications as well. Although both proxy and agent architectures can secure messages as they traverse the network, the former requires more resources in terms of development to enable.

Transport-layer communications can be secured easily using well-known protocols such as SSL, but this protects only the communication channel and does nothing for the security of underlying data. A bad purchase order worth millions can still be forwarded using SSL--transport-layer security only protects data from prying eyes; it can't make a determination as to the authenticity of the data being transported. Such security is enabled in a SOA implementation through the use of WS-Security, specifically those profiles that describe the use of XML-DSIG and XML-Encryption to provide digital signatures and encryption of data.

A proxy deployment implementing security, such as element- or document-level encryption or signature verification, requires that the application server absolutely trust the proxy or that the application implement these functions. The latter is not acceptable in some industries, and therefore it behooves you to train developers to understand and implement code to handle the relevant specifications, or deploy an agent on the application server that can perform these functions. The latter approach is most commonly used because tying application code to specific security measures runs counter to the premise of SOA--loose coupling at all layers.

Finally, all SOA management products can perform digital signature verification and XML encryption/ decryption in both proxies and agents. The differences among vendors become more apparent when considering authentication and authorization. Although the X.509 Certificate Token profile of WS-Security 1.1 is supported unilaterally, other authentication profiles, such as UserName, SAML and Kerberos, are not so universally available. All products in this space provide mechanisms for extracting "custom" authentication credentials from within XML messages, so ostensibly, one could support any of these profiles or use nonstandard, custom credentials instead.

Lori MacVittie is a Network Computing senior technology editor working in our Green Bay, Wis., labs. She has been a software developer, a network administrator, and a member of the technical architecture team for a global transportation and logistics organization. Write to her at [email protected].

Synopsis

Find our complete comparative product review and report card at NWCREPORTS.COM.

We invited makers of SOA management suites to send their products to our NWC Inc. business applications lab, in Green Bay, Wis., for review. To make our cut, entries had to support WSDL 1.1 and SOAP 1.2 and provide auditing functionality and schema validation.

Participating Vendors:

Actional, SOA Software

Testing Scenario:

We asked our SOA management suites to manage 50 Web services artifacts, half of them SOAP 1.2 with attachments. Many required authentication and authorization, and just to make things interesting, some arrived with intentionally invalid credentials. A percentage of all requests also violated message-size limitations or carried messages that violated the schema. We required the products to authenticate administrators and end users through integration with a Windows AD 2000 instance and conducted performance testing to determine the hit on our network.

Scoring Criteria:

» Management: 40 percent Rated policy life-cycle management, systems agents, user management and audit trails provided.

» Monitoring: 40 percent Half of this grade was based on metrics, both historical and real-time. It also rated real-time monitoring and logging/data capture capabilities.

» Standards Support: 15 percent WSDL 1.1 and SOAP 1.2 were required. Others were considered nice to have.

» Price: 5 percent Grading was based on the need to manage seven separate application server platforms and included the price of each product's centralized management server.

Results:

We were pleased with both products; depending on what you want from your SOA management suite, one should suit your needs. If centralized management and control are paramount, and you don't mind a lack of policy versioning, SOA Software will easily fit the bill. If business-oriented monitoring and broader support for application server platforms is your thing and you can handle some extra configuration time, Actional is for you.

Actional's strengths are in presentation of performance metrics, policy management, and SLA enforcement. SOA Software excelled in centralized policy management, SLA enforcement and its ability to graphically represent service interrelationships. SOA Software finished a hair ahead of Actional, but we didn't award an Editor's Choice because we had only two entries--and that's fine by us. Both of these products are ready for prime time.

E-Poll Results