This one covers operating them: why LISTEN/NOTIFY and PgBouncer are a trap, how to monitor outbox lag before it pages you, advisory locks for horizontal polling, and the concrete signals that tell you the Postgres approach has hit its ceiling.

We ran the Postgres outbox pattern for eighteen months. For most of that time it was invisible — events were inserted, polled, processed, and the processed flag flipped. No incidents. No oncall pages. The pattern genuinely earned its place.

Then we hit three problems in the same quarter: LISTEN/NOTIFY stopped working after we added PgBouncer, the outbox table grew to four million rows and the polling query got slow, and a poller deployment gap left seven hundred events sitting unprocessed for eleven minutes while we were in a planning meeting. None of these are fatal. All of them have solutions. But none of them show up in the pattern documentation.

LISTEN/NOTIFY + PgBouncer is a trap

LISTEN/NOTIFY is Postgres's real-time push mechanism. A backend calls LISTEN 'events', a trigger on the events table calls pg_notify('events', payload), and the listener receives the notification immediately without polling. It's elegant and it works — until you put a connection pooler in front of it.

PgBouncer in transaction mode (the mode that actually saves connections) multiplexes multiple clients over fewer server connections. A client issues a query, borrows a server connection for the duration of that transaction, then returns it. But LISTEN requires the server connection to stay assigned to that client permanently — the notification fires on the server connection, and if PgBouncer has moved that connection to a different client, the notification is lost. Silently.

We found this when we added PgBouncer to handle a connection spike and watched our event processing drop to zero. No errors. The poller was running, LISTEN was registered, the trigger was firing — but the notifications never arrived.

Three options, in order of preference:

// Option 1: bypass PgBouncer for the listener — maintain one direct connection
// Use your DB URL with a flag that bypasses the pooler (Supabase uses port 5432 for direct, 6543 for pooler)
// This is the cleanest fix: one dedicated long-lived connection for LISTEN, everything else goes through PgBouncer

import { Client } from 'pg'

const listenerClient = new Client({
  connectionString: process.env.DATABASE_URL_DIRECT,  // direct, not pooler port
})
await listenerClient.connect()
await listenerClient.query("LISTEN events")

listenerClient.on('notification', async (msg) => {
  if (msg.channel !== 'events') return
  const payload = JSON.parse(msg.payload ?? '{}')
  await processEvent(payload.id)
})

// Reconnect on disconnect — the listener is stateful and the lost connection means lost notifications
listenerClient.on('end', () => {
  console.error('Listener connection closed — reconnecting in 5s')
  setTimeout(() => reconnectListener(), 5000)
})

// Option 2: PgBouncer session mode — the connection stays assigned to one client
// Costs you the connection savings you added PgBouncer for.
// Only worth it if session mode solves your original connection problem.
// In pgbouncer.ini:
//   pool_mode = session

// Option 3: drop LISTEN/NOTIFY entirely, poll on a short interval
// LISTEN is an optimisation — events that arrive mid-poll-cycle get processed on next poll.
// If your SLA is seconds, not milliseconds, polling every 2s is fine.

async function startPoller(intervalMs = 2000) {
  while (true) {
    await processPendingEvents()
    await sleep(intervalMs)
  }
}

async function processPendingEvents() {
  // Pull a batch — don't process one at a time
  const events = await db.query<Event>(
    `SELECT id, type, payload FROM events
     WHERE NOT processed
     ORDER BY occurred_at
     LIMIT 100
     FOR UPDATE SKIP LOCKED`
  )
  if (events.rows.length === 0) return
  await Promise.all(events.rows.map(processEvent))
}

FOR UPDATE SKIP LOCKED is essential whether you're using LISTEN or polling. It locks the rows being processed so a second poller instance or a concurrent poll cycle doesn't double-process them. Without it, two pollers racing to process the same batch will both succeed at reading but only one should win at processing — and without the lock, neither knows about the other.

Monitoring outbox lag — the two queries you need running

Outbox lag is the delay between an event being written and it being processed. When the system is healthy, lag is under a few seconds. When something is wrong — the poller crashed, a processing function threw on every event in the batch, the DB is under load — lag grows and events stack up. You want to know before a stakeholder does.

-- Query 1: current lag — age of the oldest unprocessed event
-- If this is over your SLA, the poller is behind or stuck
SELECT
  COUNT(*)                                         AS pending_count,
  MAX(now() - occurred_at)                         AS max_lag,
  PERCENTILE_CONT(0.95) WITHIN GROUP
    (ORDER BY EXTRACT(EPOCH FROM (now() - occurred_at))) AS p95_lag_seconds
FROM events
WHERE NOT processed;

-- Query 2: processing rate — events processed per minute over last 5 minutes
-- If this drops to zero while pending_count grows, the poller is down
SELECT
  DATE_TRUNC('minute', processed_at) AS minute,
  COUNT(*)                           AS events_processed
FROM events
WHERE
  processed = true
  AND processed_at > NOW() - INTERVAL '5 minutes'
GROUP BY 1
ORDER BY 1 DESC;

Wire these into whatever monitoring you already have. In practice: a cron job every sixty seconds that runs query 1, writes the result to a metrics table or your APM, and fires an alert if max_lag exceeds your threshold. Three minutes of unprocessed lag warrants a Slack message. Ten minutes warrants a page.

// Monitoring cron — runs every 60s alongside the poller
async function checkOutboxHealth() {
  const result = await db.query<{ pending_count: string; max_lag: string }>(
    `SELECT
       COUNT(*)             AS pending_count,
       MAX(now() - occurred_at) AS max_lag
     FROM events WHERE NOT processed`
  )

  const pending  = parseInt(result.rows[0].pending_count)
  const lagMs    = parsePgInterval(result.rows[0].max_lag)  // convert '00:00:04.2' to ms

  metrics.gauge('outbox.pending_count', pending)
  metrics.gauge('outbox.max_lag_ms', lagMs)

  if (lagMs > 3 * 60 * 1000) {
    await notify.slack(`⚠️ Outbox lag: ${Math.round(lagMs / 1000)}s — ${pending} events pending`)
  }
  if (lagMs > 10 * 60 * 1000) {
    await notify.page(`Outbox lag ${Math.round(lagMs / 60_000)}m — possible poller failure`)
  }
}

Scaling the poller horizontally without double-processing

One poller is a single point of failure. Two pollers without coordination double-process events. The right primitive is Postgres advisory locks — session-level locks that are held for the duration of a connection, not a transaction, and that fail immediately rather than blocking.

// Advisory lock pattern — only one poller wins the lock per cycle
// pg_try_advisory_lock returns true if the lock was acquired, false if someone else holds it
// Use a stable integer key — I use a hash of the string 'outbox_poller'
const POLLER_LOCK_KEY = 1_073_741_824  // arbitrary stable integer

async function tryAcquirePollerLock(client: PoolClient): Promise<boolean> {
  const { rows } = await client.query<{ acquired: boolean }>(
    'SELECT pg_try_advisory_lock($1) AS acquired',
    [POLLER_LOCK_KEY]
  )
  return rows[0].acquired
}

async function pollerLoop(pool: Pool) {
  while (true) {
    const client = await pool.connect()
    try {
      const locked = await tryAcquirePollerLock(client)
      if (!locked) {
        // Another instance holds the lock — stand by, check again next cycle
        await sleep(2000)
        continue
      }

      // We have the lock — process a batch on THIS client so the lock stays held
      await processBatchWithClient(client)
    } finally {
      // Releasing the client back to the pool releases the advisory lock
      client.release()
    }

    await sleep(2000)
  }
}

This gives you active-passive failover: two poller instances run, one holds the lock and processes, the other polls for the lock every two seconds. If the active instance's pod is killed, the lock is released when the connection drops, and the standby instance acquires it within one poll cycle. Events resume processing with at most a few seconds of additional lag.

FOR UPDATE SKIP LOCKED still matters even with advisory locks — it handles the gap between lock acquisition and the batch query, and it protects against bugs in your own poller code that might accidentally run two concurrent batch queries on the same instance.

The table retention problem

Nobody tells you this in the pattern documentation: if you never delete processed events, the table grows forever. Four million rows with a partial index is still fast for polling. Forty million starts slowing vacuums. At some point the events table becomes one of your largest tables and VACUUM is struggling to reclaim space from the mass of processed rows.

-- Retention job — run nightly, delete in batches to avoid a single long transaction
-- Deleting in one statement on a large table holds a lock for too long
DO $$
DECLARE
  deleted INT;
BEGIN
  LOOP
    DELETE FROM events
    WHERE id IN (
      SELECT id FROM events
      WHERE processed = true
        AND occurred_at < NOW() - INTERVAL '30 days'
      LIMIT 10000  -- batch size: large enough to make progress, small enough not to block
      FOR UPDATE SKIP LOCKED
    );

    GET DIAGNOSTICS deleted = ROW_COUNT;
    EXIT WHEN deleted < 10000;  -- last batch was partial — we're done
    PERFORM pg_sleep(0.1);       -- brief pause between batches to release lock pressure
  END LOOP;
END $$;

-- If you need a full audit trail, archive to a cheaper table or object storage before deleting:
INSERT INTO events_archive SELECT * FROM events WHERE processed AND occurred_at < NOW() - INTERVAL '30 days';
-- Then run the batched delete above

The signals that tell you you've outgrown Postgres EDA

The Postgres outbox pattern has a ceiling. Here are the specific signals I've seen that tell a team they've hit it — not vibes, not traffic numbers, but observable symptoms.

Outbox lag under normal load exceeds five seconds. If the poller can't keep up with the write rate under normal conditions — not during a traffic spike, but on a Tuesday afternoon — the polling approach has hit its throughput limit. You can tune batch size and poll interval, but if you've already done that and lag persists, you need a different delivery mechanism.

Multiple independent consumer groups need different positions in the event stream. The outbox processed flag is a single boolean. It works when one consumer processes each event. When you add a second consumer that needs to track its own position independently — a reporting pipeline that can fall behind without affecting the main flow — you need event stream semantics that a boolean flag can't model.

Events need replay. The outbox deletes processed events. If you need to reprocess events from six months ago because a downstream service had a bug and you need to rebuild its state, the data isn't there. A proper event store with retention and consumer offsets solves this by design.

The events table is causing VACUUM contention. When autovacuum is running on the events table more than once per minute and it's showing up in pg_stat_activity regularly, the table's write pattern is putting real pressure on Postgres. At this point you're paying the operational cost of a message queue without getting its benefits.

-- Check autovacuum frequency on your events table
SELECT
  schemaname,
  relname,
  n_dead_tup,
  last_autovacuum,
  autovacuum_count,
  n_ins_since_vacuum
FROM pg_stat_user_tables
WHERE relname = 'events'
ORDER BY last_autovacuum DESC;

-- If autovacuum_count is incrementing frequently and n_dead_tup stays high
-- even after vacuums, you have a write/delete rate the table can't keep up with.

What to graduate to — and in what order

If you're hitting the ceiling, the graduation path I've seen work in practice is not 'drop Postgres and add Kafka'. It's incremental.

First: add a dedicated events database or schema. Separate the events table from your main transactional database so its VACUUM pressure doesn't affect your primary workload. This is often enough to buy another six to twelve months.

Second: evaluate managed options before self-hosted. Amazon SQS, Google Pub/Sub, and Azure Service Bus are message queues with managed retention, dead-letter queues, and consumer group semantics. They cost money and they have quirks, but they have zero operational overhead compared to a self-hosted Kafka cluster. If your team doesn't have someone who has operated Kafka at scale, a managed queue service is almost certainly the right next step.

Third: Kafka when you actually need Kafka. Kafka earns its weight when you need high-throughput ordered streams with long retention, multiple independent consumer groups, and the ability to replay from any offset. If you're at the scale where those requirements are real, you probably also have the team to operate it. If you're adding Kafka because it sounds serious, you're paying the 2am partition rebalancing tax for a problem you don't have.

The Postgres approach bought us eighteen months of not running a message broker. When we outgrew it, we moved one high-volume event type to SQS and kept the rest on Postgres. That's still the setup. It's not elegant. It's proportional.

For more blogs, visit