DataStax Java Driver Producer/Consumer PoC

This repository contains a Java concurrency proof of concept that extends the DataStax Java Driver with a producer/consumer API for CQL query results.

The project was created for a programming assignment. Its goal is to let a consumer request a bounded number of Cassandra rows at a time while results are fetched asynchronously and delivered without blocking the caller. See the original assignment for the full task specification.

What was implemented

• ProducerConsumerSession, a CqlSession extension that returns a Producer<Row> for a CQL string or statement.
• CustomRequestProcessor, which delegates CQL execution to DataStax's asynchronous request processor and adapts the resulting AsyncResultSet.
• Demand-controlled production through produce(n): a producer emits no more rows than its consumer requested.
• Asynchronous result-page fetching using CompletionStage and ForkJoinPool.
• A non-blocking drain-loop pattern coordinated by atomic work-in-progress and demand counters.
• Cancellation and explicit completion/error callbacks.
• Composable map, filter, and reduce producer transformations.
• Unit tests for concurrency behavior and transformations, plus Cassandra-backed integration tests for single-page, multi-page, partial, and failed queries.

This is an extension of the native DataStax Cassandra driver API. It is not a JDBC driver.

Architecture

ProducerConsumerSession
        |
        v
CustomRequestProcessor --> CqlRequestAsyncProcessor
        |                          |
        v                          v
  Producer<Row> <----------- AsyncResultSet pages
        |
        +--> registered Consumer<Row>
        +--> map / filter / reduce

ProducerImpl tracks outstanding demand with an AtomicLong. Calls to produce(n), completion of the initial query, and completion of subsequent page fetches all schedule the same drain path. An AtomicInteger work-in-progress counter serializes drain activity without using a blocking lock. When the current AsyncResultSet page is exhausted, the producer requests the next page asynchronously. Calling cancel() prevents further rows from being delivered.

The transformation producers preserve the same contract:

• map transforms each delivered item.
• filter requests replacement items when an upstream item is rejected, preserving downstream demand.
• reduce consumes the upstream sequence and emits the accumulated result after completion.

Main API

The key interfaces are under src/main/java/br/unb/oss/driver/api/producer:

• ProducerConsumerSession creates producers for CQL strings and statements.
• Producer<T> supports consumer registration, bounded production, cancellation, and transformations.
• Consumer<T> receives items and terminal completion or error notifications.
• ProducerConsumerSessionBuilder creates a session with the custom request-processor registry.

A producer must have exactly one registered consumer. After registration, call produce(n) with a positive value to request up to n items. Requests may be made incrementally. Calling produce or cancel before registration is an error.

Requirements

• JDK 8
• Maven 3
• DataStax Java Driver 4.1.0 (resolved by Maven)

The Cassandra-backed integration tests use the driver's CCM test infrastructure and therefore need the Cassandra/CCM prerequisites expected by that infrastructure. The unit tests use mocks and do not require a running Cassandra cluster.

Build and test

Compile and run the unit tests:

mvn test

Run the complete Maven lifecycle, including integration tests:

mvn verify

The Maven build also applies source formatting and checks compilation with Error Prone. Integration test sources are located in src/it/java; unit tests are in src/test/java.

Project status

This repository is an assignment-scale proof of concept, not a production-ready client library. The code demonstrates custom DataStax request processing, asynchronous pagination, bounded demand, concurrent state coordination, cancellation, and producer composition.