Event driven application architecture
If you’ve been following Hackernews or my favorite, the Highscalability blog, you can’t have missed the trend towards event driven server applications which are inherently different from the request-per-thread based design of traditional web applications.
Event driven applications evolve around asynchronous, non-blocking APIs and use few threads, typically around the number of CPU cores as opposed to the hundreds of threads used by traditional applications.
The Servlet 3 API introduces such an asynchronous interface at the processing layer where servlets are invoked asynchronously and are expected to release the invoking thread if they are blocking (remember cooperative multitasking?).
On the network front, Tomcat’s NIO connector implements the asynchronous paradigm by also using very few threads.
Taken to the extremes, one could write servlets that serve resources from the filesystem in an asynchronous way, thus not blocking threads while waiting on a harddisk or a congested network or asynchronously invoking remote web services.
The problem: JDBC is blocking
However there is an important component that always is blocking: JDBC and database operations. Despite all efforts undertaken for an asynchronous application architecture, eventually a thread will be stuck waiting for database operations. Since databases scale only in clusters (read: in parallel), you also need multiple threads to exploit that parallelism which forces one eventually to maintain a large pool of threads (incidentally equal in size to the amount of connections to the database) to dispatch database operations.
A solution for Mysql: async-mysql-connector
The “obvious” solution to this problem is to directly connect via TCP/IP to the database, implement its protocol and handle any operations to the database as an asynchronous network operation. Fortunately the async-mysql-connector does this already by achieving high performance, supporting many database features at a low resource cost.
Comparing performance – test setup
In order to get a feeling of the expected benefits, I created on a small server a database with a single table and 1 million rows which however fits completely into memory.
A test was then conducted from a different computer over a 100 Mbit LAN comparing results of the blocking mysql connector with results from the async connector by reading over 3 minutes random entries from the table.
Both tests tax the database server to its maximum, thus the test analysis will focus on client load and throughput.
The blocking control test performs 1700 reads/sec with 1 thread and achieves a maximum throughput of 4900 reads/sec with 6 threads.
The asynchronous test also performs 1700 reads/sec with 1 thread and 1 connection, reaches an impressive 4900 reads/sec with 1 thread and 3 connections and absolutely outperforms the blocking connector with 3 threads and 3 connections at 5400 reads/sec which is also quite surprising, because in both cases the database server was as already stated taxed to the maximum.
Conclusion
If you are willing to adopt asynchronous programming and are not scared by callbacks and the giving up on the notion of a linear programme execution then the async mysql connector tears down the one remaining obstacle towards a 100% blocking-free, lean, asynchronous server architecture providing excellent performance.
George's Techblog 
Very interesting indeed! Thanks for the info 🙂
LikeLike
I see that you need to manage the Multiplexer by calling select on it within a tight loop otherwise nothing will happen. How should this work in say a servlet 3 based async web app? Could you provide an example async servlet that makes a query against a fictional database and return the results to the caller? Would there have to be a separate thread responsible for running the while loop that calls select on the multiplexer?
LikeLike
I think that is what mysql-async-connector's author envisioned it to be. The application would start the asynchronous connection (for instance in the intialize() servlet method) and then drive the selector in an ExecutorService. There's an example on how to scale this to multiple threads on the wiki: (for instance in the intialize() servlet method)
LikeLike