BullMQ Workers
Use this reference when the repository already processes jobs with @nestjs/bullmq. It applies the handler rules in scaling and reliability to that package; it does not choose a queue product. Read the installed bullmq and @nestjs/bullmq versions first, because option names and defaults change between major versions.
Confirm the package before writing a processor
@nestjs/bullmq and @nestjs/bull have different consumer APIs. Do not mix them.
| Signal in the repository | Consumer shape |
|---|---|
@nestjs/bullmq and bullmq | Class with @Processor('queue') that extends WorkerHost and implements process(job); route by job.name |
@nestjs/bull and bull | @Process('name') handler methods |
import { Processor, WorkerHost } from '@nestjs/bullmq';
import { Job } from 'bullmq';
@Processor('receipts')
export class ReceiptProcessor extends WorkerHost {
constructor(private readonly sendReceipt: SendReceipt) {
super();
}
async process(job: Job<ReceiptJob>): Promise<void> {
switch (job.name) {
case 'send':
return this.sendReceipt.execute(job.data.orderId, job.data.operationKey);
default:
throw new Error(`Unknown job name ${job.name}`);
}
}
}Register processors as providers in a module that only worker processes import when web and worker capacity differ. Worker-level events use @OnWorkerEvent inside the processor; queue-level events need a QueueEventsHost class with @QueueEventsListener. During a rolling deployment an unknown job name may come from a newer producer, so decide whether it fails terminally or stays retryable for a newer worker.
Assume every job can run more than once
BullMQ provides at-least-once processing. A job runs again when:
- the processor throws and attempts remain;
- the worker stops renewing the job lock (a crash, a kill during deployment, or an event loop blocked past the stalled check, about 30 seconds by default) and the job returns to waiting;
- an operator replays a failed job.
A job that stalls more than maxStalledCount times (default 1) fails permanently even though its code never threw.
- Carry a stable operation key in the job data and claim it atomically with the protected write, following the idempotency steps in scaling and reliability.
- Do not treat a custom
jobIdor thededuplicationoption as the idempotency guard. They only block a second enqueue. A customjobIdis accepted again onceremoveOnCompleteorremoveOnFaildeletes the earlier job, so its window equals the retention window, and neither protects against redelivery of an accepted job. - Pass an idempotency key to external providers that support one; a local record does not make a remote call atomic.
Classify failures inside the processor
attemptsdefaults to 1, so a failed job is not retried unless the producer ordefaultJobOptionssets it. Withoutbackoff, a retry starts immediately.- For transient failures set bounded
attemptswithbackoff: { type: 'exponential', delay, jitter }. Exponential backoff waits2 ^ (attemptsMade - 1) * delaymilliseconds; add up the series and compare it with the business deadline. - For terminal failures (validation, authorization, missing aggregate, broken invariant) throw
UnrecoverableErrorfrombullmq. The job moves to failed without using the remaining attempts. - Throw
Errorinstances only; BullMQ's failure handling expects them. - Map causes with the taxonomy in error handling instead of inspecting messages.
Keep the lock during CPU-heavy or long work
- CPU-bound code in
process()blocks the event loop, the lock is not renewed, and the job stalls and is retried while the first run may still be executing. - Move CPU-heavy handlers to a sandboxed processor (a separate file run in a forked process) or to a dedicated worker deployment. Nest dependency injection is not available inside a sandboxed processor; the function must build what it needs.
- Total concurrency is the worker
concurrencymultiplied by replicas. Bound it by the database pool and downstream rate limits, not by CPU count alone. - Split or checkpoint jobs that can outlive the termination grace period.
Retention, payload, and Redis
- Completed and failed jobs are kept forever by default, so Redis memory grows with traffic. Set
removeOnCompleteto a small count or age andremoveOnFailto a larger one; the failed set is the inspection and replay surface. - Job data is stored in Redis as clear text. Enqueue identifiers and immutable input, not secrets, tokens, or personal data that can be reloaded.
- BullMQ needs Redis
maxmemory-policyset tonoeviction, and jobs survive a Redis restart only with persistence enabled (AOF preferred). Verify both on the deployed instance; a cache-tier Redis often has neither.
Producers and workers need different connection behavior
- Worker connections must keep
maxRetriesPerRequest: nullso blocking commands wait for Redis to return. BullMQ sets it on connections it creates and warns when a shared ioredis instance overrides it; treat that warning as a defect. - A producer on a request path should fail fast when Redis is down (for ioredis,
enableOfflineQueue: falseon the queue connection). Otherwisequeue.add()waits and requests pile up. Use a named configuration (configKey) when producers and workers in one application need different connection options. - Decide the caller-visible result when enqueueing fails. When the job must exist if and only if a database write commits, enqueue from a transactional outbox instead of from the request handler.
Shutdown
@nestjs/bullmqcloses workers inonModuleDestroyand queues inonApplicationShutdown. On a termination signal those hooks run only afterapp.enableShutdownHooks(). Without it, SIGTERM kills active jobs and they return as stalled.worker.close()waits for active jobs and has no built-in timeout in the TypeScript SDK. Set the platform grace period above the longest job plus margin; a job cut off at the deadline is recovered as stalled and runs again.- A worker-only process can start with
NestFactory.createApplicationContext(WorkerModule)and still needsenableShutdownHooks().
Verification gate
- Run the processor twice with the same job data and assert one effect and one stored result.
- Assert that a terminal failure throws
UnrecoverableErrorand a transient one throws a plainError. - With a real Redis, verify the configured attempts and backoff, kill a worker mid-job and observe the stalled job complete once on another worker, and send SIGTERM during an active job to confirm it finishes inside the grace period.
- Check
maxmemory-policy, persistence, and retention on the deployed Redis, not only locally. - Alert on waiting count, oldest waiting job age, failed count, and stalled events.
A mocked Queue or a direct call to process() proves handler logic only. It does not prove delivery, retry, stall recovery, or shutdown behavior.
This reference owns @nestjs/bullmq processor shape, job options, retention, stall, connection, and shutdown rules. Scaling and reliability owns the queue decision, idempotency design, retry budget, and backpressure; error handling owns failure classification.
See the official NestJS queues and lifecycle events documentation, and BullMQ going to production, stalled jobs, retrying failing jobs, stop retrying jobs, job ids, deduplication, auto-removal of jobs, sandboxed processors, and graceful shutdown.
Canonical source: skills/nestjs-features-performance/references/queue-workers.md. This page is generated during the documentation build.