Conceptual Design of Threading Package

There are basically three efforts that have influenced the Liberty threading package. The overall concept is that the execution framework we deliver should be able to support a pipelined execution strategy, a fork/join programming model, and employ work-stealing within the scheduler.

Intel Threading Building Blocks

Intel TBB is a C++ template library that provides a set of concurrent containers, a task scheduler, and infrastructure to support the creation of scalable, parallel applications.

The scheduler implemented by TBB employs a work stealing model on top of a managed thread pool. The thread pool is generally sized such that there is one thread per hardware thread and is expected to run largely compute-bound work.

TBB has first class support for pipline execution models as well as a fork/join decomposition model. TBB also provides direct access to the scheduler to implement custom policies.

For more information see the Intel Threading Building Blocks web site.

Staged Event Driven Architecture

The Staged Event Driven Architecture (SEDA) describes a server architecture where the flow of work through a system as a pipeline of work stages connected by queues. Each stage hosts an event handler that processes a stream of similar events.

Within a stage, a controller works to throttle the amount of work coming in to avoid overloading the system. Another controller attempts to adjust the execution characteristics of the stage to provide optimal throughput for the stage.

While not fully implemented, the threading package is structured such that all work stages (backed by a shared or dedicated ExecutorService) can be controlled and monitored. The characteristics of the stages will be adjusted at runtime in an attempt to provide the best overall throughput for the system.

The Liberty Event Engine is capable of associating event topics with a named work stage. The idea is that mapping mainline event topics to stages will allow us to implement a SEDA-style architecture if we choose to. If, instead of a SEDA-style implementation we prefer to process all events on the thread that fires them, we can map events to a single stage. If we're looking for mostly single threaded behavior but with load balancing, we can use work stealing to process independent events when the system has free cycles.

JSR 166y

The java.util.concurrent packages introduced with Java 5 have generated follow on efforts known as JSR 166x, JSR 166y, and JSR 166z.

JSR 166y, planned for Java 7, introduces implementations that support fork-join style parallel programming. This style of programming works to decompose the processing of a data set into smaller chunks that can be scheduled to run in parallel. By implementing a work-stealing style ExecutorService in Liberty, it should be possible to either reuse our implementation or replace it with what comes out with the JDK.

For more information, please see the JSR 166y web site.