The Inbox & Outbox Pattern

Picture this: your e-commerce service just confirmed an order. It writes to the database and fires off a message to the notification service. Then - crash. Did the message go out? Was the order saved? You genuinely don’t know. This is the dual-write problem, and it quietly corrupts distributed systems every day.

The Transactional Outbox Pattern (and its sibling, the Inbox Pattern) elegantly solves this. No two-phase commit. No Saga complexity for simple cases. Just one atomic write and a reliable relay. Let’s dissect it completely.

The Problem: Dual-Write Catastrophe

In any event-driven microservice architecture, you almost always need to do two things at once:

1. Persist state to a database
2. Publish an event/message to a message broker (Kafka, RabbitMQ, SQS…)

These are two separate resources. And two separate writes can fail independently. This creates a consistency nightmare.

The Outbox Pattern - How It Works

The key idea: instead of publishing directly to a message broker, you write the event to a special outbox table in the same database transaction as your business data. A separate relay process then reads from this table and publishes to the broker.

1. The Order Service writes to orders AND inserts a row into outbox - both in one database transaction. If the transaction fails, neither write persists.
2. A Relay Process periodically queries the outbox for unsent messages (where sent = false).
3. The relay publishes each event to the message broker.
4. On successful publish, the relay marks the outbox row as sent = true (or deletes it).

The Outbox Table Schema

CREATE TABLE outbox (
id          UUID          PRIMARY KEY DEFAULT gen_random_uuid(),
event_type  VARCHAR(100)  NOT NULL,           -- 'ORDER_CONFIRMED'
aggregate_id VARCHAR(100) NOT NULL,           -- entity the event relates to
payload     JSONB         NOT NULL,           -- the full event body
created_at  TIMESTAMPTZ   DEFAULT NOW(),
sent        BOOLEAN       DEFAULT FALSE,      -- has relay published it?
sent_at     TIMESTAMPTZ
);

Two Relay Strategies: Polling vs. CDC

The relay process can read the outbox in two fundamentally different ways:

Polling (Simple Relay)

The relay runs a SELECT * FROM outbox WHERE sent = false LIMIT 100 query on a schedule. Dead simple to implement. Works with any relational database. The downside: there’s a delay between write and publish (equal to the poll interval), and at high throughput, polling can pressure your DB.

Change Data Capture (CDC)

Tools like Debezium tap into the database’s transaction log (PostgreSQL’s WAL, MySQL’s binlog) and stream every change in near real-time. No polling needed. Far more efficient at scale. The tradeoff is operational complexity - you need to run and manage a CDC pipeline.

The Inbox Pattern - The Consumer Side

The Outbox handles publishing reliably. But what about the consumer? Message brokers with at-least-once delivery can deliver the same message twice. If your consumer isn’t idempotent, you get duplicated side effects - double charges, double emails, double everything.

The Inbox Pattern solves exactly this: it tracks which messages have already been processed, turning at-least-once delivery into effective exactly-once processing.

1. Consumer receives a message from the broker.
2. It checks the inbox table: have I seen this msg_id before?
3. If yes - it’s a duplicate. Discard it safely.
4. If no - insert into inbox AND perform the business logic in one transaction.

CREATE TABLE inbox (
msg_id       VARCHAR(200)  PRIMARY KEY,  -- broker message ID
received_at  TIMESTAMPTZ   DEFAULT NOW(),
processed_at TIMESTAMPTZ
);

-- Inside the consumer handler (pseudocode):
BEGIN TRANSACTION;
INSERT INTO inbox (msg_id) VALUES ('abc-125')
  ON CONFLICT DO NOTHING;
IF affected_rows = 0 THEN
  ROLLBACK; -- duplicate, skip
END IF;
-- do the actual work
INSERT INTO notifications ...;
UPDATE inbox SET processed_at = NOW() WHERE msg_id = 'abc-125';
COMMIT;

The Full Picture: Outbox + Inbox Together

When you combine both patterns, you get a fully reliable messaging pipeline: the producer guarantees every event is eventually published (Outbox), and the consumer guarantees every event is processed exactly once (Inbox). This is the gold standard for event-driven microservices.

When to Use This Pattern

Gotchas & Edge Cases

Key Takeaways