Netty: Custom Pipeline Factory

April 23, 2009

As promised in prior articles, I will demonstrate how to create a custom pipeline factory in Netty that provides some custom features including:

  • Auto-discovery of annotated handlers (including pluggable annotation provider)
  • Auto-provisioning of stateful and stateless handlers
  • Auto-pooling of stateful handlers (including pluggable pooling provider)

The purpose of this exercise was to initially demonstrate how to create poolable handlers to conserve memory such that it would increase throughput and decrease GC time.  However, my initial tests show very little savings, unfortunately.  After pondering on this, I realized that almost all handlers are stateless singletons.  Only frame decoders are really stateful and those handlers are generally small in terms of memory use.  Thus, the amount of garbage provided by a stateful handler is generally small.  Further, the pooling only helps on incoming connections as the pipeline is only created on initial connection.  So, it would only be beneficial in applications that have several short-lived connections.  The real memory use is primarily in the socket byte buffers, socket connections, etc, which is tied to the VM and not as easily controlled.

Anyways, I ended up putting some work in showing how you can go crazy with pipeline factories.  Note that the code is not well tested, yet, and is meant more as documentational than operational.  I provide no direct support/responsibility, although I can provide help/feedback if anyone is interested in actually using it.  I plan to continue implementing and using this in my applications, so as I fine tune it, I will release updates. That aside, let’s dig in.

First off, I tend to see a handler as providing one of three states or roles.  They are either stateless singletons that provide all work within their callbacks and maintain no state, stateful handlers that require state to be maintained across requests, or custom handlers that require more functionality or mixed functionality between stateful and stateless (for example, a singleton that maintains state using synchronization and locks).  These are provided in my ChannelPipelineState enumeration.  You specify the state using the state attribute on the custom ChannelPipelineCoverage annotation.  This annotation replaces the existing ChannelPipelineCoverage by providing the ability to specify the name, dependencies, and alternative factory if necessary.  The CustomPipelineFactory searches added classes for the annotation and uses it to properly load the handler into the pipeline.  The factory also supports auto-discovery of annotated handlers by searching existing classes in the classpath.

Second, in order to conserve memory and slightly improve performance, you can use pooling for stateful handlers.  This allows handlers to be re-used after their associated connection is closed.  The ChannelHandlerPooling annotation provides the necessary configuration to setup each handler pool.  The pooling specifies the min/max/initial boundaries and an alternative factory for creating the pool.  By default, the pipeline factory attempts to use commons-pool if the library is in the classpath.

So, how does all of this work?  Let’s work through an example and outline the steps taken by the pipeline factory.  To start we need to decide if we want to use auto-discovery or not.  Initially, I thought auto-discovery would be a good idea.  However, I have since stepped back to realize that a single place to see the explicit order and configuration of the pipeline is a good thing.  The pipeline is so essential to the application and how it functions that trying to figure out how a bunch of annotated classes actually tie together is difficult to follow.  My preferred approach is to annotate the classes still, but disable auto-discovery and only use the annotations to provide state and pooling information.  The ultimate approach is your decision, however.

The following is an example handler configuration using annotations:

@ChannelHandlerPooling(min=10, initial=10, max=1000)
@ChannelPipelineCoverage(name="encoder", state=ChannelPipelineState.STATELESS)
public class MyHandler extends SimpleChannelHandler
    // handler callbacks

To create the pipeline factory and add this handler to it, we can use auto-discovery or explicit code as shown below.

// create bootstrap
ServerBootstrap bootstrap = new ServerBootstrap(...);
// METHOD 1: Auto-Discovery
ChannelPipelineFactory factory = new CustomPipelineFactory();
// METHOD 2: Explicit
ChannelPipelineFactory factory = new CustomPipelineFactory();

Method 1 is as simple as merely setting the pipeline factory on the bootstrap. The factory will search the classpath for annotated handlers and add to the pipeline as necessary. The beforeXXX and afterXXX options on the ChannelPipelineCoverage annotation provides the relationships and dependencies that are used to properly order the pipeline.

Method 2 is also very simple, yet requires explicit configuration and ordering. Even still, we can easily create pooled handlers by just annotating the handler appropriately. The one piece of functionality that does not yet exist in this example library is the ability to specify the configuration in the addXXX method. This would be useful for configuring pre-existing handlers such as StringEncoder and StringDecoder. Currently, you would need to create a custom class with the proper annotations and then extend the appropriate built-in class. The class does not require a body or overriding any methods, however.

Note that the pipeline factory orders explicitly added handlers first and adds auto-discovered annotations last. This is important when a handler does not have any relationships. Relationships, on the other hand, are always preserved and properly ordered regardless of how they are added.

The ChannelPipelineFactory uses a list of ChannelHandlerInfo instances that specify the handler information and the associated factory. The factory is an instance of ChannelHandlerFactory. Every handler has a factory and the factory is responsible for returning an instance of an actual handler. For example, handlers annotated as STATEFUL are implicitly assigned a StatefulChannelHandlerFactory. This factory uses reflection to create a new instance based on the associated handler class. STATELESS handlers use a StatelessChannelHandlerFactory that creates a singleton and returns the same instance everytime. PoolableChannelHandlerFactory uses a pooling provider to borrow and return an object from a respective pool. Handlers may also be annotated with a custom factory to provide custom functionality. These handlers use the CUSTOM annotated state.

When the getPipeline method is invoked on the pipeline factory, the factory iterates its list of handlers and asks each factory for the associated handler via makeChannelHandler. The handler is then added to the custom pipeline. A custom pipeline is used rather than the default pipeline in order to know when a particular connection is closed. This allows the pipeline to return its handlers (ie: to return a handler to the pool). The custom pipeline overrides the attach method and adds a custom close listener to the channel to be notified when the connection is closed. It then informs the pipeline factory that the pipeline has completed, which in turn invokes returnChannelHandler on each ChannelHandlerFactory.

This make and return functionality was provided for pooling. When a socket is connected a new pipeline is created and pooled handlers are borrowed from the pool. This will either result in a new or existing instance being returned depending on the state of the pool. When the socket is disconnected and the pipeline and its handlers no longer needed, then the handlers are returned to their factory and released from the pool. Subsequent connections can then re-use those handlers conserving memory.

As mentioned before, this pooling system is entirely pluggable. The pipeline factory uses a ChannelHandlerPoolProvider implementation to create a ChannelHandlerPool for a given handler. The PoolableChannelHandlerFactory uses that pool to borrow and return handlers. The ChannelHandlerPool is just a front to an implementation of a specific pool. For example, commons-pool is provided as an implementation example by CommonsChannelHandlerPoolProvider creating a CommonsChannelHandlerPool which in turn creates a StackObjectPool. The StackObjectPool is the actually underlying pool from which handlers are borrowed and returned.

The auto-discovery is also pluggable. By default, there is an example implementation using Scannotation. The pipeline factory uses an AnnotationProcessorProvider to search for a list of annotated classes. The scannotation implementation uses javassist to dynamically lookup annotations in the classpath. There is a setter method that can be used to set the appropriate implementation.

This code is just an example of how you can control handlers through a custom pipeline factory. For me personally, I plan to continue using the pooling implementation for stateful handlers just to be extra efficient despite the minimal impact. Rather than use auto-discovery though, I plan to explicitly add and order the handlers. I will pass the class as an argument though to allow the pipeline factory to control the pooling and instantiation based on the annotated configuration. For example:

CustomPipelineFactory factory = new CustomPipelineFactory();

Some final future thoughts of where I can see this maturing is by first adding the ability to specify the configuration in the method when you cannot annotate a class (ie: because it is built in). For example:

ChannelHandlerConfig config = new ChannelHandlerConfig(STATEFUL);
config.setPooling(new ChannelHandlerPooling(10, 10, 1000);
factory.addLast(MyHandler.class, config);

Second, it would be nice to group handlers into buckets such as framing, encoding, decoding, converters, application, etc. Then, you can annotate a handler into a group rather than explicitly specify its dependencies. Typically, most handlers in a single group can be ordered in any order, but the ordering of the groups themselves matter. For example, framers should occur first, followed by encoders/decoders, followed by application handlers. The groups can allow handlers to join specific groups and be ordered accordingly. This would make third party handlers with annotations easier and make auto-discovery more appealing. If and when I do add any such functionality, I will update this post.

For more information, leave a comment and I will get in touch.

3 Responses to “Netty: Custom Pipeline Factory”

  1. Hi Nicholas,

    I would’ve tried to contact you directly, but the only link I found was through LinkedIn (not sure if that went through).

    Basically I’m looking for some help with Netty, which you seem to know very well. If it’s at all possible, I’d really appreciate an educational session or two on some of the finer points I’m missing. As much as I’d like to read their documentation, I’m finding it pretty weak right now.

    Please let me know if this would be possible. I do realize your time is worth money, so I’m sure we can work something out. Please go ahead and contact me at the email associated to this comment.
    I look forward to hearing from you.

  2. Many thanks to you, this article is better than the official, it’s great !!

  3. Hello Nicholas,

    I realise you wrote this in 2009. I tried some of it, and judging by the comments, people agree that your article is very good, so I would like to know, if you would be updating it, to the current Netty version, as some of the code does not seem to be agreeing any more?

    Thanks for your time.