The Queue You Already Have

Many teams add Redis when they need a job queue. Redis is fast and widely used. It is also another service to deploy, secure, monitor, back up, and recover.

For many applications, PostgreSQL already provides what the queue needs: durable state, transactions, row locks, and query tools. PostgreSQL has supported SKIP LOCKED since 2016.

This post explains how the pattern works and where it stops being a good fit.

The problem with a simple database queue

A basic queue uses a jobs table. Workers poll for a row with a pending status, then process it.

-- Worker A
SELECT * FROM jobs WHERE status = 'pending' ORDER BY created_at LIMIT 1;
 
-- Worker B (5 ms later)
SELECT * FROM jobs WHERE status = 'pending' ORDER BY created_at LIMIT 1;

Both workers can read the same row. Both can then process it. This can charge a customer twice, send two emails, or update stock twice.

A plain SELECT does not block another read. This behavior is useful in most cases, but it does not claim work safely.

SELECT ... FOR UPDATE fixes duplicates by locking the row. However, every worker waits for the same oldest row. The queue becomes serial, even with many workers.

Redis queues avoid this with atomic list operations. PostgreSQL can avoid the same wait with SKIP LOCKED.

FOR UPDATE SKIP LOCKED

SKIP LOCKED tells PostgreSQL to skip rows that another transaction has locked. The worker selects the next unlocked row instead of waiting.

SELECT id, payload
FROM jobs
WHERE status = 'pending'
ORDER BY created_at
LIMIT 1
FOR UPDATE SKIP LOCKED;

If 50 workers run this query, one worker locks the first row. The others skip it and claim later unlocked rows. Each worker gets a different job. No worker waits for the first job to finish.

Why this is safe

PostgreSQL uses Multi-Version Concurrency Control (MVCC). Rows hold transaction data that lets PostgreSQL decide which version a query can see.

FOR UPDATE places a row lock that lasts until COMMIT or ROLLBACK. With SKIP LOCKED, PostgreSQL sees a row locked by another live transaction and continues its scan. The lock remains real, so another worker cannot claim that row.

The tradeoff is order. Under load, workers can process jobs slightly out of created_at order. Most work queues accept this tradeoff.

Claim the job in one statement

Select and mark the job as claimed in one transaction. This avoids a gap between finding work and claiming it.

WITH claimed AS (
  SELECT id
  FROM jobs
  WHERE status = 'pending'
  ORDER BY created_at
  LIMIT 1
  FOR UPDATE SKIP LOCKED
)
UPDATE jobs
SET status = 'processing', claimed_at = NOW()
FROM claimed
WHERE jobs.id = claimed.id
RETURNING jobs.*;

The selected row is locked and changed to processing before the transaction commits. Other workers cannot claim it.

What PostgreSQL provides

Crash recovery

PostgreSQL writes committed changes to its Write-Ahead Log (WAL). If a worker fails during a job, the job row still exists. A cleanup task can move stale claims back to pending.

-- reclaim jobs whose worker died holding them
UPDATE jobs
SET status = 'pending'
WHERE status = 'processing'
  AND claimed_at < NOW() - INTERVAL '5 minutes';

This is at-least-once delivery. A job can run again after a worker fails. Design job effects to be idempotent.

Transactional effects

If the job state and its business data use the same database, one transaction can update both. The job can become done only if its result is saved.

True exactly-once delivery is not possible in a distributed system. PostgreSQL can make effects exactly once when the operation is idempotent and shares the transaction.

History and retries

A job row stays available for queries. You can inspect failures, attempts, payloads, and timestamps with SQL.

SELECT * FROM jobs
WHERE status = 'failed'
  AND created_at > NOW() - INTERVAL '1 hour';

Use a run_after column to delay retries:

ALTER TABLE jobs ADD COLUMN run_after TIMESTAMPTZ DEFAULT NOW();
 
-- worker only considers jobs that are due
WHERE status = 'pending' AND run_after <= NOW()
 
-- on failure, push it into the future, further each time
UPDATE jobs
SET status = 'pending',
    attempts = attempts + 1,
    run_after = NOW() + (INTERVAL '5 seconds' * (2 ^ attempts))
WHERE id = $1;

A dead-letter queue can be a status = 'dead' value after the retry limit.

The main cost: table bloat

A queue table changes often. PostgreSQL creates a new row version for each UPDATE, then autovacuum removes the old version. A busy queue can create dead tuples faster than autovacuum removes them.

Long-running transactions make this worse. They can prevent autovacuum from removing old row versions. The queue table and its indexes can then grow and slow down.

Use these controls:

• Set more aggressive per-table autovacuum settings for the queue table.
• Delete or archive completed jobs soon.
• Partition a high-volume queue by time, then remove old partitions.
• Check pg_stat_activity for long-running transactions.

This is an operational limit, not a correctness issue. Monitor it before the queue becomes busy.

Existing queue systems

This pattern is established. Several queue systems use FOR UPDATE SKIP LOCKED:

• Solid Queue is the default Active Job backend in Rails 8.
• River is a Go and PostgreSQL queue based on this pattern.
• graphile-worker and pg-boss use the pattern in Node.js.
• que has used a PostgreSQL queue model in Ruby for years.

The pattern is a standard option, not a database trick.

When PostgreSQL is enough

PostgreSQL is often a good queue when:

• You process fewer than about 50,000 jobs each hour.
• A delay above about 50 ms is acceptable.
• Jobs already use the database.
• You need transactional job effects.
• You want fewer services to operate.

Consider Redis when you need hundreds of thousands of jobs each minute, require sub-millisecond dequeue latency, or process work that does not touch the database.

These are rough guides, not benchmarks. Hardware, payload size, job work, indexes, and polling behavior all affect throughput. Measure your workload before you choose.

Redis is faster for many queue workloads. The question is whether its speed pays for its extra operational cost.

A production example

On May 12, 2026, Shopify described a MySQL-native SKIP LOCKED design for inventory reservations. MySQL 8.0 also supports SKIP LOCKED.

Their conclusion matched this pattern: a relational database can be enough when the additional Redis system does not provide enough value for the workload.

My use case: paystable

On May 22, 2026, I made the same decision while I built paystable.

Paystable is an open-source Go daemon for Indian payment gateways. It waits before acting on a failure webhook. It then checks the gateway status with jittered exponential backoff until the status remains stable across several checks.

That process needs a queue. Workers must claim jobs without duplicates. Jobs need retries, backoff, and crash recovery.

Paystable uses one Go binary and one PostgreSQL database. Adding Redis would add a client, connection pool, service, and deployment. FOR UPDATE SKIP LOCKED, an outbox table, and PostgreSQL WAL already met the need.

The queue claim and retry code is below:

// ClaimJob atomically claims one pending job for processing.
// Returns nil if no jobs are available.
func (q *Queue) ClaimJob(ctx context.Context) (*Job, error) {
    var job Job
    err := q.db.QueryRowContext(ctx, `
        WITH claimed AS (
            SELECT id
            FROM stabilization_jobs
            WHERE status = 'pending'
              AND run_after <= NOW()
            ORDER BY created_at
            LIMIT 1
            FOR UPDATE SKIP LOCKED
        )
        UPDATE stabilization_jobs
        SET status = 'processing', claimed_at = NOW()
        FROM claimed
        WHERE stabilization_jobs.id = claimed.id
        RETURNING stabilization_jobs.*
    `).Scan(&job.ID, &job.TxnID, &job.Status, &job.Attempts,
        &job.RunAfter, &job.ClaimedAt, &job.Payload)
 
    if err == sql.ErrNoRows {
        return nil, nil
    }
    return &job, err
}
 
// Reschedule moves a failed job back to pending with exponential backoff.
func (q *Queue) Reschedule(ctx context.Context, id string, attempts int) error {
    _, err := q.db.ExecContext(ctx, `
        UPDATE stabilization_jobs
        SET status    = 'pending',
            attempts  = $2,
            run_after = NOW() + ($3 * INTERVAL '1 second')
        WHERE id = $1
    `, id, attempts, backoff(attempts))
    return err
}
 
func backoff(attempts int) int {
    // 5s -> 10s -> 20s -> 40s -> 80s -> 160s (capped)
    delay := 5 * (1 << attempts)
    if delay > 160 {
        return 160
    }
    return delay
}

No Redis client or separate queue service was needed.

Decision checklist

Before you add Redis, answer four questions:

1. What is the actual job volume?
2. Do the jobs already use the database?
3. Can the team operate another service?
4. What recovery guarantee does the queue need?

If PostgreSQL meets these needs, use the queue you already operate. Add Redis when measured scale or latency requires it.

Conclusion

Naive database queues caused duplicate work. FOR UPDATE SKIP LOCKED solves that concurrency problem without serializing workers.

PostgreSQL is not the right queue for every workload. It is a durable and simple choice for many applications. Start with it when it meets the requirements. Add more infrastructure only when the workload proves you need it.

A blog from someone working to improve reliability guarantees for Indian payment gateways.