Event-Driven Systems (BullMQ)
I design queue layers that hold up in production: BullMQ on NestJS with priority lanes, bounded retries, stalled-job detection, and real-time status over WebSockets. RealStream schedules thousands of concurrent AI inference jobs on a single Redis instance — no Kafka or extra infrastructure anywhere.
What you get
- Queue architecture design: which work is a job, which is a request, and the responsibility border
- Per-workload-class queues with priorities, timeouts, and cancellation that actually works
- Retry and backoff policy tuned to your upstream providers' real failure modes
- Stalled-job detection and automatic re-queueing, tested by killing workers on purpose
- Job status streaming to your users (Redis pub/sub → WebSocket) and operational dashboards
How it runs
Step 1 — interview every job before writing code
For each job type: is it idempotent, what is its timeout, what does one retry mean, what state does it leave behind if it dies halfway. These four answers decide the queue design.
Step 2 — separate lanes, not one big queue
Interactive and batch work get physically different queues with different concurrency and priorities, so batch renders can never starve user-facing jobs. The cheapest scaling decision there is.
Step 3 — make worker death a non-event
State lives in Redis, artifacts in object storage, stalled jobs are re-queued by a heartbeat detector. Any worker can be killed and replaced at 3 a.m. with no runbook and no lost work.
Case studies behind this service
FAQ
Should we use BullMQ or RabbitMQ for our service?
If your stack is Node.js and you already run Redis, BullMQ covers most product workloads with an idiomatic API, delayed jobs, priorities, and rate limiting. RabbitMQ earns its keep for protocol-level routing across polyglot services, and Kafka for replayable event logs. Most product teams debating this on paper are not in either regime yet — measure, then choose.
Is BullMQ production-ready at high job volumes?
Yes. RealStream has scheduled thousands of concurrent inference jobs on BullMQ backed by one Redis instance since 2026. Production-readiness comes from the policy you configure: bounded retries, per-job timeouts, stalled-job detection, and per-class queues — not from the library itself.
When is our workload too big for a single Redis-backed queue?
Signals: Redis CPU saturating, need for exactly-once semantics across services, replayable event logs, or multi-region fan-out. Those are Kafka-class problems. Until one of them appears, one managed Redis carries far more than typical product traffic implies.
Further reading
- NestJS + BullMQ at ScaleI shipped an event-driven NestJS + BullMQ pipeline to production in 60 days. What actually breaks first at scale: retry storms, lost jobs, queue starvation, and the fixes that hold.
- BullMQ vs RabbitMQBullMQ vs RabbitMQ is usually asked before the workloads are known. A decision guide from production: one Redis-backed BullMQ layer runs thousands of concurrent AI jobs, and what RabbitMQ still does better.
Who you work with
Eugene Zaiko
Eugene Zaiko is a senior full-stack engineer (5+ years, ex-PixelPlex, Syberry, Linnovate) specializing in event-driven backends, PostgreSQL at scale, and production AI systems. No account managers: you talk directly to the engineer whose fingerprints are on the code.