EnorexaEnorexa
Back to blog

Engineering

Redis Keyspace Notifications: Why Ephemeral AI Agent Locks Drop Dead

November 8, 20262 min readBy Enorexa Team

Redis keyspace notifications look like the cleanest way to reap abandoned AI agent workflows. When an agent node locks a job key with a 120-second TTL and dies during a complex multi-step reasoning loop, you expect the expiration event to fire, release the lock, and requeue the task. In high-concurrency production, that assumption quietly breaks your orchestration engine.

The Passive Eviction Trap That Freezes Queues

Redis does not guarantee exact-second key expiration delivery. Expiration happens passively when a key is accessed, or actively via background sampling of 20 random keys with TTLs per database ten times a second. When your task volume surges or memory pressure climbs, an expired worker lock can sit uncollected in memory for tens of seconds after its TTL elapses. The __keyevent@0__:expired event never emits until Redis sweeps that specific bucket. Downstream task dispatchers wait on hold, parent workflows time out with GatewayTimeoutError: 504, and your job state stays permanently trapped in limbo.

Pub/Sub Drops and the Cluster Replica Split

Keyspace notifications rely on standard Redis Pub/Sub, which is strictly fire-and-forget without acknowledgment guarantees. If your watchdog worker restarts during deployment or encounters a momentary network jitter, the expiration broadcast vanishes into the ether. Even worse, on clustered Redis topologies, keyspace events are broadcast exclusively to the specific cluster node holding that key hash slot. If your event consumers connect to an arbitrary proxy or replica without slot-aware routing, they will never receive the expiration signal. You are left polling zombie records while customer requests queue up behind phantom locks.

Active Heartbeat Leasing Beat Passive Event Listening

Production systems cannot treat key expiration as an orchestrator. Instead of passive event listening, robust AI pipelines switch to explicit cooperative heartbeats backed by Redis sorted sets (ZSET). The worker writes a timestamped heartbeat (ZADD active_leases <epoch_ms> <worker_id>) every 10 seconds. A lightweight cron sweeper queries ZRANGEBYSCORE active_leases 0 <cutoff_epoch> to actively identify dead workers, atomically clears their locks with a Lua script, and re-enqueues stalled tasks to BullMQ. At Enorexa, we build resilient operational backends and high-volume orchestration engines that bypass fragile pub-sub patterns entirely. If zombie jobs are stalling your production pipelines, talk to us to architect self-healing worker infrastructure.