Review: XML Gateways

As Web services become more pervasive, attackers are taking aim. Forum Vulcon, a subscription service offering notification of XML vulnerabilities using a Web services interface, is tracking more than 100 vulnerabilities. This may sound minuscule compared with the thousands of known attacks threatening Web applications and back-end servers, but the danger is that a successful XML-based attack can act as a master key, exposing any number of those application vulnerabilities. Because SOAP (Simple Object Access Protocol) messages carry instructions in protocols that are interpreted as functions to be executed on servers, application-server-specific attacks can be transported inside the XML and passed to the application server, where they wreak havoc.

Because XML is a self-defining format, parser and content-based vulnerabilities are ever evolving and nearly impossible to predict. Therefore, a well-rounded XML security device not only must act as a firewall in the conventional sense--granting or denying access based primarily on Layer 4 information, such as IP address and port--it also must secure back-end services against threats specific to XML and against Web and database servers. This requires XML security to move away from a packet-processing paradigm to the realm of payload analysis.

To see how well the market is meeting this challenge, we asked 11 XML security vendors to participate in rigorous firewall tests in our Green Bay, Wis., NWC Inc. business applications lab. DataPower, Reactivity and Sarvega agreed. Layer 7 initially accepted, but product availability problems led it to pull out. Check Point Software Technologies agreed, then declined. Forum Systems cited scheduling conflicts, while Digital Evolution and Vordel both said they don't consider their products firewalls. Xtradyne was acquired last year by PrismTech and is still positioning its product. Oblix partners with Forum Systems to provide XML security, and after Forum Systems declined, it didn't make sense to include Oblix, which was then acquired by Oracle in March. Actional, which merged with security firm Westbridge last year, declined based on the review focus.

At first, we were frustrated with the low turnout, but the reasons cited point to an industrywide problem: What are these products, and what should they be called? See "The Name Game,", for our take.

Although most conventional firewalls can provide user-based authentication and authorization to services, they're rarely set up to do so; rather, these products control generalized access to services, and their packet-processing mechanisms are not data-aware. XML firewalls, however, must be data-aware to keep unwanted content and users from accessing potentially sensitive services. Although XML over HTTP and even SOAP can be controlled using conventional authentication means, HTTP Basic Auth, for example, SOAP and Web services cognoscenti prefer to use Web services-specific mechanisms, such as WS-Security 1.0, which require authentication and authorization mechanisms to reach into the payload and extract credentials.

For our test scenario, we used NWC Inc.'s Web services deployment, served by IBM WebSphere 6.0 and providing SOAP interfaces to order-entry and tracking functionality. After capturing both requests and responses from all operations, we served them up on our Spirent WebReflector to remove any application bottlenecks. We throttled client traffic back to no more than 2,000 concurrent users, a reasonable number--on the high end for most Web services infrastructures but realistic for an enterprise Web services application. The types of attacks we ran are detailed in "How We Tested XML Firewalls,".

We cut to the chase of what really matters by awarding 50 percent of our score to the products' protection capabilities--simply, how well they stopped attacks. We divided the protection category into four basic areas:

• Message integrity: We tested a device's ability to encrypt sensitive data and verify signatures on incoming requests. XML Encryption and XML-DigitalSignature are the primary mechanisms of ensuring message integrity in the XML/Web services world. This category also rates the devices' ability to use SSL at the transport layer.

• DoS protection: We examined a product's ability to shield the device and protected back-end services against denial-of-service attacks that use oversized documents, XML-parser exploits and other such mechanisms.

• Content-based defenses: These features stop attacks aimed at back-end services, including SQL injection and external entity attacks.

• Authentication and authorization: This category includes support of Web services-specific protocols, such as WS-Security, to verify users for explicit operations.

We also took into consideration operational and functional management, the security of administrative mechanisms, performance and accuracy under load, and price. Finally, we quizzed each vendor on its methodology for updating devices in response to newly discovered vulnerabilities.

XML Threat Defense

All the products we tested let us secure our XML services against a variety of attacks, but they differ in default settings and control levels. Sarvega's XML Guardian Gateway configures bidirectional schema validation upon policy creation by default. This approach prevents a number of XML attacks without requiring additional configuration--for example, attacks using alternate character sets and SQL injection into non-string-based fields can be stopped by validating the incoming request against the expected schema. Products from both DataPower and Reactivity support schema validation, but it must be added to your policy. Schema validation certainly won't prevent all content-based attacks--loosely written schemas undermine this security mechanism, which is best used to prevent malformed XML from reaching back-end services.

SQL injection attacks are likely the most damaging in terms of information gleaned by attackers and opportunity to destroy data. Out of the box, only Sarvega XML Guardian stopped our SQL attack. We used the other two products' failure here as an opportunity to evaluate DataPower's and Reactivity's methods of updating their devices in the face of "new" vulnerabilities. DataPower's XS40 is based wholly on XSL (Extensible Stylesheet Language); we addressed a new content vulnerability by retrieving and uploading the relevant XSL file. After uploading a new SQL injection pattern file to the XS40, our SQL injection attack no longer succeeded. These products are not proactive in retrieving and updating content filters, as one might expect. Unlike antivirus packages that automatically update filters on a scheduled basis, the products we tested required us to determine whether a new vulnerability had been discovered and then update the content filters manually.

Reactivity, like Sarvega, uses content filtering as its main mechanism for adding protection against new vulnerabilities. Unlike DataPower and Sarvega, however, Reactivity digitally signs new content-filtering files and verifies the signature once a file has been uploaded to its device, before applying it to the system. As with all the participants, we created custom content filters to quickly add protection or functionality. We wrote a new content filter so that Reactivity's 2400 Series XML Security Gateway, which uses regular expression matching, could recognize our "UNION SELECT" SQL attack, then retested. The 2400 properly stopped the probing attack.

All three products tested support standard encryption (XML-ENC) and signatures (XML-DSIG) equally well. To configure the devices, we had to modify our policies to include encryption and signature verification; we accomplished this with ease across all entries. Sarvega and DataPower went further than Reactivity in terms of their products' ability to limit message sizes and incoming XML structure, such as number and depth of nodes, and elements within any given XML document. Controlling both is important in preventing parsing attacks. Only Reactivity's 2400 XML Security Gateway let us place size limitations on the messages it sent to our back-end servers, but it supports this limitation only for non-SOAP messages. We limited message size by manually editing the configuration file and restarting services, but such restrictions were applied to all messages rather than on an operation basis, which is how we'd like to see such filtering accomplished in XML security devices.

Vendors in many product areas have embraced the flexible, open-source plug-in Eclipse framework as the basis for managing devices and applications. The XML security arena is no exception. Both Sarvega and DataPower provide Eclipse management tools for their devices, with varying degrees of success. We preferred Sarvega's CommandCenter, the principal method of device management, over its unappealing and limited Web-based administrative console. In contrast, DataPower's Eclipse management tool is effective and as powerful as its Web-based console, which continues to improve with each release and is comparable in functionality and ease of use to Reactivity's Web console.

No device gave us much operational configuration capability from the Web console. We accomplished Layer 2/3 management using terminal services or SSH (Secure Shell) in the case of DataPower and Reactivity, and by means of an LCD control panel for Sarvega's XML Guardian. DataPower and Sarvega provide operational statistics for CPU, processes and memory utilization from their management consoles, but Reactivity offers these juicy details only from the terminal, and we needed to use conventional Linux tools, such as top, from an SSH session to delve into its device.

Functional management was rich and detailed in all three products' main admin consoles, with DataPower and Reactivity providing the most intuitive and easy-to-navigate interfaces. We easily achieved message pipeline configuration--the steps within a policy that detail what actions should be taken on a message and in what order--within Reactivity's XML Security Gateway, but DataPower's configuration is confusing. For example, we were never quite certain whether we were configuring a request or response in the XS40's administrative GUI, which caused us a few fits. Sarvega's configuration was made more difficult by the hierarchical nature of Eclipse, which is essentially a file-system-based editor.

XPath is still the primary method of manipulating XML files, and we found varying degrees of support across the products. XML Guardian's excellent XPath editor at first did not perform as advertised, but Sarvega provided a patch that quickly restored it to working order. DataPower's tool also let us easily configure those features requiring XPath, such as encrypting specific elements within an XML document. Reactivity's offering, however, lacks an easy mechanism for generating XPaths.

Stress Tests

After each successful policy implementation, we ran a series of performance tests that included valid and malicious traffic. The XS40 maintained its accuracy even under heavy load, though we detected some heavy breathing in CPU utilization and an increase in latency when the device was configured to perform content filtering and authentication, which requires additional parsing and transformation. We weren't surprised that all three competitors performed schema validation, signature validation and encryption without adding latency. It was only when we piled on content filtering, IP blacklisting and authentication that DataPower and Reactivity began to bog down. Sarvega didn't bat an eye in any configuration--it never added a single millisecond of latency.

After rigorous testing in which all three devices proved capable of stopping the attacks we threw at them, even under heavy load, DataPower's XS40 edged out Sarvega for the top spot in our review. Sarvega XML Guardian's less intuitive functional-management paradigm and the company's decision to turn off account access during a dictionary password attack rather than block the offending IP address kept it from overtaking DataPower. Reactivity was close on the heels of its rivals, but was hindered by lower performance numbers. We wouldn't hesitate to recommend any of these products.

Performance, flexibility and integration are the key attributes of our Editor's Choice. The XS40's architecture is built on XSL, and it quickly adapted to just about any threat we threw at it. This generation of the XS40 is a 1U, four-port Gigabit Ethernet appliance with a separate management port and serial-console access. Hiding beneath the covers is DataPower's proprietary XG3 XML acceleration technology.

Since our last look at the XS40 (see ID# 1601sp2), DataPower has enhanced its Web administrative console with a new firewall wizard and control panel. We liked the device's fine-grained, domain- and role-based management scheme that let us permit or deny varying levels of access based on attributes such as object type, all the way down to specifically named objects like "Firewall 1" or "NWC Firewall." This level of control is offered for object management and is meant to secure XML policies, not the messages passing through the device.

The XS40 let us set policies on a per-firewall basis, which boiled down to per port. We also could create complex policies on a single firewall that emulate a per-operation or document type policy, comparable to the policy-configuration options offered by Sarvega and Reactivity.

We configured our initial scenario--asking the device to perform bidirectional schema validation, content filtering and limited authentication--in spite of a minor glitch in the new XML Firewall wizard that caused it to ignore our attempt to modify the endpoint destination on our back-end server. This capability, along with DataPower's rewrite rule, let us obfuscate service names. Sarvega's and Reactivity's products offer this capability as well, but Reactivity's method is much more elegant. DataPower fixed the glitch with a patch, and the wizard behaved as expected. Subsequent configurations were simple matters of modifying the existing policy by adding signature verification to one request, encryption of the response on another and requiring authentication by means of WS-Security headers in another. As with all the products we tested, the XS40 can encrypt an entire response or a single element within an XML document and perform transformation of XML through the application of XSLT.

We added IP ACLs (access-control lists) with ease on the XS40, though they can be configured on a per-firewall basis only, similar to Sarvega's implementation. Reactivity doesn't support IP ACLs for blacklisting, but does allow explicit IP ACLs that restrict access to SOAP operations--the specific function or method being executed on the application server--to specified ranges.

DataPower XS40 XML Security Gateway 3.1, $65,000. DataPower, (617) 864-0455. www.datapower.com

Sarvega's product is focused on performance and content protection, and supports almost complete operational management capabilities through its Eclipse CommandCenter 1.6.

The XML Guardian is a 2U device that sports four Gigabit Ethernet ports and offers both front-panel and serial-console configuration. However, it doesn't have a separate management port; this caused us problems when we decided to enable SSL, because management is through HTTPS and runs on Port 443. We could move the management interface to a different port or change the ports for the services (the latter was much simpler during our tests). We much preferred DataPower's and Reactivity's configuration setups, which placed SSL-secured Web management on alternate ports by default.

Configuring the XML Guardian for message-size limitations was almost overwhelming in terms of the number of options available. Unlike Reactivity, Sarvega provides extremely fine-grained control of XML structure. Even DataPower, which offers a good number of options, can't match Sarvega in this area. From message size to depth of elements, size of elements to number of children, nearly every aspect of an XML document can be restricted on a per-operation basis. Although most of these restrictions can be defined by the schema, they're rarely used, and when they are, they often aren't detailed enough to prevent parsing attacks. We were pleased with being able to limit message size on a per-operation basis, because this value can vary from operation to operation.

There are two factors unique to Sarvega: its decision to enable schema validation by default and the fact that it does not serve up WSDL. The company told us it's decided to wait for WS-Policy before providing this functionality. In contrast, DataPower serves up WSDL because it's essentially a proxy, while Reactivity lets you create an aggregate WSDL based on user rights--a feature we hope other vendors will implement. Although we generally applaud and encourage standards-based implementations, as long as the resulting WSDL is WS-I Basic Profile compliant, we're not that concerned about proprietary methods of generation.

But our biggest complaint with the XML Guardian was the need to restart the device when we deployed a new configuration. Minor changes to existing configurations don't require restarts, but major changes do, and we had to wait "some time" (Sarvega's phrasing) for the device to resume. During testing, "some time" lasted two to three minutes, during which time managed services were unavailable.

XML Guardian's performance was on par with DataPower XS40's and in some ways beat the competition. For example, the XML Guardian added no latency in any scenario, while both rivals added latency in at least three different test scenarios. Its XESOS 5.0.2 kept up with the XS40's custom silicon throughout our tests. During performance tests we could keep an eye on CPU utilization through the dashboard option on the XML Guardian's Web console. A wealth of other operational and functional statistics are available for near-time graphing. Reactivity offers historical functional statistics, including performance metrics for both clients and back-end servers, while DataPower provides operational and functional statistics for only a few categories--notably HTTP transactions, memory and CPU utilization--and doesn't do so in near-time.

Sarvega XML Guardian Gateway 5.0, $55,000. Sarvega, (866) 727-8342, (630) 627-3131. www.sarvega.com

This product offers the most flexible authentication and authorization of the devices we tested, in addition to a dynamic DoS protection scheme and a highly navigable administration console.

The 2400 is a 1U appliance running Linux with dual 3.06-GHz Xeon processors and dual Gigabit Ethernet NICs, and 2 GB of RAM. It comes equipped with an nCipher HSM (Hardware Security Module) for SSL acceleration and a Tarari RAX processor for XML acceleration.

Only Reactivity provides devicewide defaults that we could easily override at the policy level. These defaults can significantly reduce the amount of time needed to configure policies and help satisfy corporate security policies by enforcing required levels of security. Although DataPower told us this functionality could be achieved on the XS40 through its new domain-based administration model, we didn't relish spending time and energy to implement what Reactivity provides out of the box. Not only could we set defaults on content filtering and override them at the policy level on the 2400, we could choose specific SQL injection protection according to the type of database: For Web services interacting with SQL Server 2000, we enabled the content filter specifically designed to protect SQL Server 2000. For Oracle, another content filter is provided by Reactivity out of the box.

The 2400's authentication and authorization options were equally impressive and showcased Reactivity's easy-to-use, albeit somewhat cluttered, Web console. We configured multiple methods of authentication and authorization on a per-operation basis and further specified multiple methods for any operation. Our only complaint revolves around LDAP configuration, which requires a Ph.D.--or at least an intimate knowledge of LDAP filters and regular expressions--to set up. We much preferred the simple configuration offered by DataPower and Sarvega, which let us specify that the user name and password extracted from the WS-Security header should be validated against our NWC Inc. AD 2000 server, but LDAP implementations are a pain across the board. And in Reactivity's case, with complexity we did get power--Reactivity's implementation is exceedingly flexible and dynamic, and with the right knowledge we could limit access based on any directory attribute. One nit: We'd like to see a basic configuration option that validates against LDAP. All the products require a user name and password with which to bind to the directory.

In our performance tests, Reactivity's appliance matched DataPower's and Sarvega's in accuracy--the 2400 did not allow a single invalid or malicious request to reach our back-end servers--but it did introduce some latency. Reactivity's engineers told us the device is optimized for the WAN, and our testing on a fully Gigabit network did not take advantage of these optimizations. Our imposed limit of 2,000 concurrent users led to Reactivity's lower performance numbers. When we removed the limits and reran the tests, all three devices showed an increase in the number of messages processed per second as the number of concurrent users increased, with most tests showing an average of 1,100 to 1,300 messages per second.

Reactivity XML Security Gateway and Manager 2400 Series, starts at $65,000. Reactivity, (866) 889-3485, (650) 551-7800. www.reactivity.com

Lori MacVittie is a Network Computing senior technology editor working in our Green Bay, Wis., labs. She has been a software developer, a network administrator and a member of the technical architecture team for a global transportation and logistics organization. Write to her at [email protected].

Web services are growing in importance, and the best way to protect them is by using a dedicated firewall. For this portion of our Firewall Blowout, we tested a trio of XML firewalls in our Green Bay, Wis., NWC Inc. business applications lab. We liked all three products: Although DataPower's adaptable XS40 XML Security Gateway took our Editor's Choice award based on its strong performance, fine-grained management and ability to integrate into our NWC Inc. infrastructure, the runners-up are worthy of mention. Sarvega XML Guardian Gateway is a strong second thanks to attractive pricing and blazing performance--the device didn't add one millisecond of latency. Reactivity's ultramanageable XML Security Gateway rounds out the pack.

The XML security space is still experiencing growth even as mergers and acquisitions work to narrow the field. Two years ago, many of the products on the market today were labeled XML firewalls. However, we chose to call them security gateways (see "Enemy at the Gateway," ID# 1421f1) because they were expected to do more than conventional firewalls.

The industry latched on to that, and we saw many products renamed to include the term security gateway. So when we sent out an invitation to participate in an XML firewall review, we should have known we'd have problems with vendors that didn't want to be limited by such a name. ForumSystems, for example, launched a separate product line, the XWall, to address differences in expectations. Ironically, considering it's a no-show, the XWall is likely the only product in this market to truly represent the conventional definition of a firewall, as it does nothing more than protect against XML threats, leaving triple-A duties to its big brother, the Forum Sentry.

In addition, several players have broken off and "invented" the XML VPN, which is little more than an XML security gateway with a special focus on managing a point-to-point secure connection between two partners.

The result is confusion. There are XML firewalls, XML security gateways, Web services security gateways and XML VPNs. But what appears to be fragmentation is not--with the exception of the XWall, the product name is simply a vendor mechanism for claiming a niche in the larger XML security market. All these devices, regardless of name, provide the same basic functionality in terms of securing Web services; the differences are limited to the amount of attention paid to management, auditing, interoperability and standards support.

We're certain the renaming craze is nothing more than positioning as vendors vie for higher visibility and uniqueness in a market that is rapidly growing in terms of dollar value and mind share. As we were writing this article, Digital Evolution changed its corporate name to SOA Software, further muddying the waters by embracing the hype surrounding the SOA (service-oriented architecture) moniker. We expect to see more product name changes in the coming months to take advantage of the buzz SOA is receiving.

With apologies to Shakespeare, we'll simply say an XML firewall by any other name is still in the business of securing XML and Web services. Functionality and features, not semantics, will determine market leadership.

Our goal in testing XML security functionality was to ascertain how thoroughly these devices defend against XML-based threats and how well they perform when configured to handle typical XML and SOAP (Simple Object Access Protocol) tasks.

We used our NWC Inc. Web services as the base for both valid and invalid XML messages (all DOC/LIT encoding), then tampered with some of them to simulate multiple types of XML attacks. Responses were all served by a Spirent Reflector 2500, simulating a Web services-enabled application server.

We initially configured each product with simple bidirectional schema validation, then set up policies on a per-operation basis to handle authentication, encryption and signature verification. We limited client traffic to no more than 2,000 concurrent users, which is higher than what most Web services setups are likely to handle, except for very large enterprises--say, Amazon or Google. Client traffic for our base test (Base Configuration #1) was simulated by a Spirent Avalanche 2500 and was split as shown in the table (opposite)..

A second set of tests (Base Configuration #2) included 20 percent signed content (purchase-order submissions) and 10 percent encrypted traffic (invoices), with the balance the same as our original test. All purchase-order submissions in the first set of tests were configured to authenticate users via WS-Security 1.0. We removed authentication in the second test set and replaced it with signed content, with the assumption that a valid signature equals a valid user. Keys were 1,024 bits in length for both signature and encryption tests.

We examined each product's environmental security by running Nmap to determine what services, if any, were left open by default. We also tampered with URIs and tried our best to gain access through the products' Web administrative consoles.

A gauntlet of tests served to evaluate overall and individual threat defense on each product, as well as the performance of each device while doing specific XML security functions. The devices also were required to perform schema validation on both request and response. Our threat-defense tests comprised:

• Authentication against LDAP: Devices were configured to authenticate users against our NWC Inc. Active Directory 2000 repository by extracting the user name and password from a WS-Security 1.0 header and validating it against LDAP. Seventy percent of requests were valid, 30 percent were invalid.

• Response encryption: Devices were configured to encrypt sensitive data in response to SOAP requests. We ran two sets of encryption tests under different configurations: configuration 1, encryption of the entire SOAP body, 4 KB in size; configuration 2, encryption of a single element type within the SOAP document, 4 KB in size, six elements to be encrypted per document.

• Signature verification: Devices were configured to verify a signed SOAP request. We ran one set of signature-verification tests with varying data sizes: 30 percent, 4 KB; 40 percent, 7 KB; 30 percent, 11 KB.

• Large response size: We also configured devices to perform schema validation against a 130-KB XML response.

All our performance results are based on these tests and the limitation of 2,000 concurrent users. Just for the fun of it, we did run tests without this user limitation, resulting in substantially higher messages per second on all products, but most enterprises won't be handling upward of 20,000 concurrent users anytime soon, making these results poor measuring sticks for most implementations.

All Network Computing product reviews are conducted by current or former IT professionals in our Real-World Labs® or partner labs, according to our own test criteria. Vendor involvement is limited to assistance in configuration and troubleshooting. Network Computing schedules reviews based solely on our editorial judgment of reader needs, and we conduct tests and publish results without vendor influence.

Welcome to NETWORK COMPUTING's Interactive Report Card, v2. To launch it, click on the Interactive Report Card ® icon above. The program components take a few moments to load.

Once launched, enter your own product feature weights and click the Recalc button. The Interactive Report Card ® will re-sort (and re-grade!) the products based on the new category weights you entered.