As a continuation of evaluating an NIO server for my iPhone game, I started with looking at pure performance. First, the following links provide already existant benchmarks:
I took my own samples and quickly ran some basic tests as well to just get a really rough idea. Note that these metrics should be taken with a large grain of salt as they do not attempt to optimize any of the test libraries and the clients are all run in traditional threads on non-server class machines alongside the actual servers. Thus, these tests are quickly CPU bound on the client. The idea is to just get a very rough estimate to compare alongside the above tests.
That point aside, my test basically creates a echo server and echo client. It then starts up X simultaneous threads that push a simple ‘testing’ string back and forth to the server as fast as it can. The end result is the time it takes to send a message and receive the echo in milliseconds.
Netty: ~2.75 ms / transaction
Grizzly: ~2.52 ms / transaction
Mina: ~3.45 ms / transaction
xSocket: ~4.15 ms / transaction
Netty: ~5.23 ms / transaction
Grizzly: ~5.51 ms / transaction
Mina: ~7.19 ms / transaction
xSocket: ~8.23 ms / transaction
Basically, Netty and Grizzly are very similar in their results. Mina and xSocket are considerably worse. As such, we will look into memory usage overall and over time in my next post as we continue to drill down to selecting an ideal library.