Z|NET Development

Now that I have narrowed by server down to using JBoss Netty, it’s time to define protocols. This involves two things. First, we need to select the actual protocol itself and second, we need to define how to construct the handlers. Let’s start by defining the handlers. Netty is built upon the principle of handlers, which are more or less the same idea as filters in web applications. This allows you to define functional handlers that perform specific tasks such as compression, security, and business logic. In this manner, one handler does not need to concern itself with another handlers and handlers can be added/removed without affecting the system. However, order and dependency does matter in certain cases. For example, if you have a handler that depends on a translation of bytes into a POJO, then the handler performing the translation must be in the pipeline and occur prior to the application logic. Generally, a semi-complex, yet real world application, for a protocol stack would consist of:

• Encryption Handlers via SSL processing, handshakes, etc. These handlers take the data stream, decrypt it, and pass the decrypted bytes down the stream
• Compression Handlers via GZip processing for instance. These handlers take the data stream, uncompress it, and pass the uncompressed bytes down the stream
• Transport Protocol Handlers for converting transport protocol data in protocol messages such as HTTP. Subsequent handlers use the protocol-specific messages (ie: HTTP message) for interacting with the protoocol data
• Security Handler for performing security in terms of authentication and authorization checks. These handlers read a header or portions of the data stream and validate the data as being both a valid, authentication user as well as being permissible or authorized or view/modify the associated data
• Application Protocol Handlers for controlling an actual underlying application/business protocol. These differs from the transport protocols in that they specify the actual business data and commands to perform on the business data. Often times, these handlers parse out a given business command, convert to a business message or POJO, and invoke a business delegate object or handler to process the data
• Application/Business Handlers for processing business logic related to a specific protocol message.

This type of stack allows easy extensibility and customization. Future handlers can be added in as needed and future application messages can be added in through custom delegates. It also allows the business logic code to only contain business logic without concern for security, compression, etc. The business logic deals solely with business objects and the business protocol. Further, because of this abstraction and because of the abstraction of the transport layers in Netty, supporting other transport protocols such as UDP, serial, USB, etc is an easy task.

The one issue with this type of model is ensuring scalability. As the number of layers is increased, performance often degrades. As such, there is a balance that must be tested, analyzed, and compared to determine the best performance for the best extensibility. Further, the layering code should be built to reduce the number of data operations that could result in data being allocated. In other words, the zero copy buffers in Netty should be used as much as possible to ensure that data flows freely layer to layer without extensive copying between layers. In certain cases, this is not possible such as encryption/compression where the entire stream needs to be rewritten. However, for cases such as parsing authentication headers, session headers, data headers, etc, the channel buffers can be used to read the data, mark the stream, and then pass the remainder of the data down the stream without requiring copy operations. In general, a best practice is only used the handlers you absolutely need. For example, if your application does not deal with secure data, do not require expensive SSL processing. If your application contains short messages or binary data, do not require compression filters that would have very little gain. In a scalable application, every CPU cycle counts to supporting more and more simultaneous connections. By reducing the number of layers, you reduce the number of CPU cycles required to process a complete transaction. Thus, you increase your traffic flow. Overall, keep things simple yet abstract away common functionality into specific handlers.

The second issue after defining the protocol stack in terms of handlers and the pipeline is selecting the actual transport protocol. Transport protocols are used to transport the actual data. Transport protocols come in two flavors: layer 5 protocols and layer 7 protocols. Layer 5 protocols are protocols such as TCP or UDP which are both part of standard IP messaging. These protocols are directly supported by Netty, Java, and NIO through sockets and socket connectors. Layer 7 protocols are application protocols that are used to transfer data. HTTP is a common example of a transport protocol in that it is used to transfer data such as HTML, plain text (ie: CSV), and XML, which are examples of application or business specific protocols. In other words, a layer 5 protocol transports a layer 7 protocol which transports application protocol data. Netty supports some layer 7 protocols out of the box, such as HTTP. Others require proprietary handlers. The question is what is the best methodology for building scalable servers both for layer 7 transport protocols and application protocols.

In general, there are 2 main types of transport protocols: HTTP-based and text/binary-based. There are three main types of application protocol data: text (ie: delimiter based such as CSV), binary, and XML. We will compare and analyze each of these for performance and features.

First, let’s look at the general pros/cons for transport protocols.

HTTP: Pros

• Global standard
• Firewall-proof in being supported by almost all providers and IT admins via HTTP proxies
• Built in library support for both client and server via Netty
• Easy to build and data independence (must be text, not binary, hover)
• Built in authentication handling
• Built in SSL handling

HTTP: Cons

• Stateless protocol requires session handling techniques for tracking users
• Connection latency between requests unless keep-alive is used (not supported by all proxy servers)
• Overhead data can be longer than data itself via HTTP headers (results in slower performance)
• Text-only (XML, delimited, HTML, etc)…binary data must be encoded (ie: base64) which is expensive

Sockets: Pros

• Stateful connection requiring only single connection
• Lightweight data by requiring no transport specific headers resulting in improved performance
• Binary support for serializing data or sending direct integer bytes resulting in improved efficiency

Sockets: Cons

• Not typically supported behind firewalls without a SOCKS proxy which is not standard in most IT environment
• Requires proprietary client/server code including SSL, authentication handling, session handshaking, etc resulting in longer development time

Now, lets look at the pros/cons for the data protocols.

XML: Pros

• XML is extensible by nature and allow changes to occur generally without issue
• Supported natively by many libraries, including Java and C
• Human readable

XML: Cons

• Very heavy by requiring open/close tags resulting in performance degradation
• SAX/DOM parsing has become fast, but is still very expensive compared to binary protocols
• Local testing shows 4-7x less throughput using XML and DOM handling

Binary: Pros

• Small and efficient by using data in its simplest and smallest form
• Ability to quickly analyze data such as headers without reading in the entire stream
• Very performant by being able to directly read in data without parsing from one form to another (ie: int is already an int)

Binary: Cons

• Extensibility and versioning require some extra work to ensure support
• No automatic means for parsing data and requires proprietary protocol definition and handling on both client and server
• Non human-readable so harder to test (ie: cannot simply telnet and verify)
• Endianess can become an issue between systems when writing/reading bytes

Text: Pros

• Smaller than XML, but can be heavier than binary (ie: 6 chars to represent a 4-byte number)
• Human readable
• Can be efficient by using custom short messages and/or short delimiters
• Local testing shows performance on par with binary testing

Text: Cons

• Version/extensibility is difficult
• No automatic APIs for reading data and requires proprietary systems
• Requires delimiters and requiring ensuring data in messages properly encodes or escapes non-intentional delimiters

In setting up some quick samples in Netty comparing the various protocols,  we can conclude that binary or text data over socket transports is the most efficient for performance. Despite XML being an industry buzz word and having powerful extensibility built-in, for a true scalable system, binary provides a real benefit. From the performance data, we can see that we can process several more clients a second using binary/text. However, that does not imply that that is the best or only protocol stack to use. We also need to look into the other pros/cons. The biggest issue for socket transports is that it is not automatically supported behind firewalls. As such, a socket-only transport layer can result in loss traffic and loss audience. Rather, the best methodology is to use both sockets and HTTP. As Netty supports protocol layers/handlers, you can easily have an HTTP stack that converts the data into data protocol messages before handling off to the business logic. Thus, you can easily employ both technologies. When sockets fail to connect due to firewall issues, it can automatically failover to HTTP via standard text or base64 encoded binary data. The client-side performance will decrease, but should manage to keep up with short, precise protocol messages.

The end result in Netty results in two protocol stacks:

Socket Layer Stack

• TCP NIO Selector [built-in]
• SSL Handler [built-in]
• Compression Handler
• Authentication Handler
• Protocol Handler (text/binary)
• Data/Business Handler

HTTP Layer Stack

• TCP NIO Selector [built-in]
• SSL Handler [built-in]
• Compression Handler
• HTTP Handler [built-in]
• HTTP Authentication Handler
• HTTP Protocol Handler (text/base64)
• Data/Business Handler

Most of the code and logic lives in the data/business handlers and the HTTP-based handlers are simple handlers to write. Thus, we have abstracted out the code and made a flexible, maintainable, yet highly efficient system.

This concludes my series on building scalable NIO servers. As I build out my code, I will publish interesting metrics as I gather them.

Tags: Netty, NIO, performance, Protocols, Scalable

This entry was posted on Monday, April 20th, 2009 at 10:38 pm and is filed under Java, Software, Standards, Usability. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.