Skip to content
64 changes: 64 additions & 0 deletions docs/worker-health-adapter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Queue health snapshot adapter

`QueueHealthAdapter@cbq` reads queue backlog through an explicit connection, or combines worker and queue measurements through an explicit worker pool. It does not start workers, reserve jobs, publish metrics or depend on Sentry. Each call resolves the current connection or worker pool and its provider.

```cfml
moduleSettings.sentry.automaticCollectors = {
// Start Sentry's periodic sampler.
enabled: true,
// One provider-neutral list for queue and worker measurements.
queues: [ {
// Fixed logical metric label; independent of cbq worker-pool and queue identifiers.
name: "mail",
// Explicit provider-owned implementation. Other queue systems use their own adapters.
adapter: "QueueHealthAdapter@cbq",
// Select worker activity plus backlog from this cbq worker pool.
options: {
// Its connection chooses the provider; its queue selection chooses the queues.
workerPool: "outbound"
}
} ]
};
```

Sentry also needs its existing valid DSN and `enableMetrics:true`. Its `queues` collector handles worker activity and backlog together; separate worker/database collector definitions are not needed. All measurements use the configured logical queue label. `options.queue` can override the selected queue(s); otherwise the worker pool's string/array/wildcard queue selection applies.

For backlog without local workers, select a connection instead:

```cfml
// Choose queue backlog without collecting shared worker activity.
moduleSettings.sentry.automaticCollectors.queues = [ {
// Fixed logical metric label.
name: "mail",
// Provider-owned adapter; no worker pool is required.
adapter: "QueueHealthAdapter@cbq",
// Explicit source and optional queue selection.
options: {
// Registered cbq connection; its provider owns storage and database configuration.
connection: "database",
// Optional queue name, array or wildcard. Omit to use the connection's default queue.
queue: "mail"
}
} ];
```

Configure exactly one of `options.workerPool` or `options.connection`. Missing, empty, nonsimple, ambiguous or legacy `options.pool` selection raises `cbq.Telemetry.InvalidQueueHealthSelection`; no source is inferred from a metric label. Missing registered connections/worker pools and provider sampling failures retain their original errors.

The adapter follows `sample(context)` structurally without importing Sentry's interface. It can be used directly without Sentry:

```cfml
snapshot = wirebox.getInstance( "QueueHealthAdapter@cbq" )
.sample( { options: { workerPool: "outbound" } } );
```

Worker-pool selection returns `configuredWorkers` and supported `activeExecutions`. If the connection's provider exposes `getQueueHealth(queue)`, either selection includes supported `depth`, `oldestAgeSeconds`, `runnable`, `delayed`, `reserved`, `staleReservations`, `retries` and `exhausted` fields. Connection-only snapshots contain no worker fields. Providers own their snapshot logic; cbq's database provider uses its configured table, query options and datasource. Database counts exclude completed/failed jobs; unclaimed work is runnable or delayed, claimed work is reserved, and expired claims are stale. Queue selections are bound query parameters; `*` selects all queues. No database configuration or cbq schema knowledge is required in Sentry.

Missing capabilities stay absent. A connection-only provider without queue health support returns an empty snapshot, not zero backlog or worker activity. An unstarted executor does not imply zero activity; supported idle executors can report zero. Queue/database sampling errors propagate to the caller's failure diagnostics, rather than pretending a backlog is zero. There are no fabricated worker heartbeats, job IDs, payloads or raw exceptions in successful snapshots.

`WorkerHealthAdapter@cbq` remains an alias, and `samplePool(workerPool)` remains available for earlier manual worker integrations. The adapter must be present in the installed cbq release before selecting its mapping in Sentry. Neither package acquires a dependency on the other.

Earlier local verification of the worker-only implementation and the companion Sentry suite is recorded in the PR. The unified queue changes add combined/backlog-only snapshots, provider-owned table/datasource selection, multiple queues and wildcard/parameter binding coverage. Current local and hosted results are recorded on the draft PRs.

Worker measurements always describe the entire worker pool, even when `options.queue` narrows the backlog selection. Prefer one combined collector per worker pool; use connection-only snapshots for extra per-queue breakdowns so shared worker activity is not counted repeatedly.

Before the explicit connection/worker-pool selector correction, the unified implementation passed 9/9 adapter specs and 137/137 companion Sentry specs on each prepared Lucee 5, BoxLang and Adobe 2023 runtime, including eleven real PostgreSQL/cbq cases. MySQL integration specs exercise provider queue snapshots in cbq's hosted suite. Current selector coverage, hosted results and release prerequisites are recorded on the draft PRs. Formatting and whitespace checks pass; installed application modules were not modified.
13 changes: 13 additions & 0 deletions models/Providers/DBProvider.cfc
Original file line number Diff line number Diff line change
Expand Up @@ -697,6 +697,19 @@ component accessors="true" extends="AbstractQueueProvider" {
return records;
}

/** Read the configured database queue without reserving jobs or starting workers. */
public struct function getQueueHealth( required any queue ) {
var queryFactory = function() {
return newQuery();
};
var snapshotAdapter = new cbq.models.Telemetry.DatabaseQueueSnapshot(
queryFactory = queryFactory,
table = variables.tableName,
queryOptions = variables.defaultQueryOptions
);
return snapshotAdapter.snapshot( arguments.queue );
}

public QueryBuilder function newQuery() provider="QueryBuilder@qb" {
}

Expand Down
60 changes: 60 additions & 0 deletions models/Telemetry/DatabaseQueueSnapshot.cfc
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
/** Read-only cbq database queue snapshot using the provider's availability and lease fields. */
component {

function init(
required function queryFactory,
string table = "cbq_jobs",
struct queryOptions = {}
) {
variables.queryFactory = arguments.queryFactory;
variables.table = arguments.table;
variables.queryOptions = duplicate( arguments.queryOptions );
return this;
}
function snapshot( required any queue ) {
var builder = variables.queryFactory();
var grammar = builder.getGrammar();
var available = grammar.wrapValue( "availableDate" );
var reserved = grammar.wrapValue( "reservedDate" );
var owner = grammar.wrapValue( "reservedBy" );
var created = grammar.wrapValue( "createdDate" );
var unclaimed = reserved & " IS NULL AND " & owner & " IS NULL";
var claimed = "(" & reserved & " IS NOT NULL OR " & owner & " IS NOT NULL)";
var now = int( createObject( "java", "java.lang.System" ).currentTimeMillis() / 1000 );
var queues = isArray( arguments.queue ) ? arguments.queue : [ arguments.queue ];
if ( !queues.len() ) throw( type = "cbq.Telemetry.InvalidQueueSelection" );
for ( var queueName in queues ) {
if ( isNull( queueName ) || !isSimpleValue( queueName ) )
throw( type = "cbq.Telemetry.InvalidQueueSelection" );
}
builder.from( variables.table );
if ( !arrayFind( queues, "*" ) ) builder.whereIn( "queue", queues );
var rows = builder
.whereNull( "completedDate" )
.whereNull( "failedDate" )
.selectRaw(
"COALESCE(SUM(CASE WHEN " & unclaimed & " AND " & available & " <= ? THEN 1 ELSE 0 END),0) AS " & grammar.wrapValue( "runnable" ),
[ now ]
)
.selectRaw(
"COALESCE(SUM(CASE WHEN " & unclaimed & " AND " & available & " > ? THEN 1 ELSE 0 END),0) AS " & grammar.wrapValue( "delayed" ),
[ now ]
)
.selectRaw( "COALESCE(SUM(CASE WHEN " & claimed & " THEN 1 ELSE 0 END),0) AS " & grammar.wrapValue( "reserved" ) )
.selectRaw(
"COALESCE(SUM(CASE WHEN " & claimed & " AND " & available & " <= ? THEN 1 ELSE 0 END),0) AS " & grammar.wrapValue( "staleReservations" ),
[ now ]
)
.selectRaw( "MIN(CASE WHEN " & unclaimed & " THEN " & created & " ELSE NULL END) AS " & grammar.wrapValue( "oldestCreatedAt" ) )
.get( options = variables.queryOptions );
if ( !isArray( rows ) || rows.len() != 1 ) throw( type = "cbq.Telemetry.InvalidQueueHealthResult" );
var snapshot = rows[ 1 ];
snapshot[ "depth" ] = snapshot.runnable + snapshot.delayed;
snapshot[ "oldestAgeSeconds" ] = snapshot.depth > 0 && snapshot.keyExists( "oldestCreatedAt" ) && !isNull(
snapshot.oldestCreatedAt
) && isNumeric( snapshot.oldestCreatedAt ) ? max( 0, now - snapshot.oldestCreatedAt ) : 0;
snapshot.delete( "oldestCreatedAt" );
return snapshot;
}

}
85 changes: 85 additions & 0 deletions models/Telemetry/QueueHealthAdapter.cfc
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
/** Provider-owned queue snapshots. Uses sample(context) without requiring Sentry. */
component {

property name="config" inject="Config@cbq";

struct function sample( required struct context ) {
if ( !arguments.context.keyExists( "options" ) || !isStruct( arguments.context.options ) ) {
throw(
type = "cbq.Telemetry.InvalidQueueHealthSelection",
message = "Configure exactly one of options.workerPool or options.connection."
);
}
var options = arguments.context.options;
var hasWorkerPool = options.keyExists( "workerPool" );
var hasConnection = options.keyExists( "connection" );
if ( options.keyExists( "pool" ) || hasWorkerPool == hasConnection ) {
throw(
type = "cbq.Telemetry.InvalidQueueHealthSelection",
message = "Configure exactly one of options.workerPool or options.connection; options.pool is not supported."
);
}
var source = hasWorkerPool ? options.workerPool : options.connection;
if ( isNull( source ) || !isSimpleValue( source ) || !len( trim( source ) ) ) {
throw(
type = "cbq.Telemetry.InvalidQueueHealthSelection",
message = "The workerPool or connection selector must be a nonempty name."
);
}
var snapshot = {};
var connection = {};
if ( hasWorkerPool ) {
var pool = variables.config.getWorkerPool( source );
snapshot = samplePool( pool );
if ( !structKeyExists( pool, "getConnection" ) ) return snapshot;
connection = pool.getConnection();
} else {
connection = variables.config.getConnection( source );
}
var provider = connection.getProvider();
if ( !structKeyExists( provider, "getQueueHealth" ) ) return snapshot;
var queueSelection = options.keyExists( "queue" ) ? options.queue : (
hasWorkerPool ? pool.getQueue() : connection.getDefaultQueue()
);
var observed = provider.getQueueHealth( queueSelection );
if ( !isStruct( observed ) ) throw( type = "cbq.Telemetry.InvalidQueueHealthResult" );
for (
var field in [
"depth",
"oldestAgeSeconds",
"runnable",
"delayed",
"reserved",
"staleReservations",
"retries",
"exhausted"
]
) {
if (
observed.keyExists( field ) && !isNull( observed[ field ] ) && isNumeric( observed[ field ] ) && observed[
field
] >= 0 && observed[ field ] <= 1.7976931348623157E308
)
snapshot[ field ] = observed[ field ];
}
return snapshot;
}

struct function samplePool( required any pool ) {
var snapshot = { "configuredWorkers" : arguments.pool.getQuantity() };
try {
var stats = arguments.pool.getExecutor().getStats();
// Some executors return a fallback zero when activity cannot be measured.
if (
stats.keyExists( "features" ) && stats.features.keyExists( "taskMethods" ) && stats.features.taskMethods &&
stats.keyExists( "activeCount" )
) {
snapshot[ "activeExecutions" ] = stats.activeCount;
}
} catch ( any unavailable ) {
// An unstarted/unsupported executor does not imply zero activity or a worker heartbeat.
}
return snapshot;
}

}
5 changes: 5 additions & 0 deletions models/Telemetry/WorkerHealthAdapter.cfc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
/** Compatibility alias for integrations using the earlier worker-only adapter name. */
component extends="cbq.models.Telemetry.QueueHealthAdapter" {


}
40 changes: 40 additions & 0 deletions tests/specs/integration/Providers/DBProviderQueueHealthSpec.cfc
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
component extends="tests.resources.ModuleIntegrationSpec" appMapping="/app" {

function run() {
describe( "database provider queue health", function() {
beforeEach( function() {
variables.provider = getWireBox()
.getInstance( "DBProvider@cbq" )
.setProperties( { datasource : "cbq" } );
variables.provider
.newQuery()
.from( "cbq_jobs" )
.delete();
} );
afterEach( function() {
variables.provider
.newQuery()
.from( "cbq_jobs" )
.delete();
} );
it( "returns measured zeros for an empty queue", function() {
var snapshot = variables.provider.getQueueHealth( "mail" );
expect( snapshot.depth ).toBe( 0 );
expect( snapshot.runnable ).toBe( 0 );
expect( snapshot.reserved ).toBe( 0 );
expect( snapshot.oldestAgeSeconds ).toBe( 0 );
} );
it( "reads selected queues without reserving jobs or mixing in other queues", function() {
variables.provider.push( "mail", getWireBox().getInstance( "SendWelcomeEmailJob" ) );
variables.provider.push( "pdf", getWireBox().getInstance( "SendWelcomeEmailJob" ) );
variables.provider.push( "mail' OR 1=1 --", getWireBox().getInstance( "SendWelcomeEmailJob" ) );
expect( variables.provider.getQueueHealth( [ "mail", "pdf" ] ).depth ).toBe( 2 );
expect( variables.provider.getQueueHealth( "*" ).depth ).toBe( 3 );
expect( variables.provider.getQueueHealth( [ "*" ] ).depth ).toBe( 3 );
expect( variables.provider.getQueueHealth( "mail' OR 1=1 --" ).depth ).toBe( 1 );
expect( variables.provider.getQueueHealth( "*" ).reserved ).toBe( 0 );
} );
} );
}

}
Loading
Loading