The Outbox Pattern in .NET

Learn how to implement the Outbox Pattern in ASP .NET Core using EF Core, background jobs, and retry logic.

Sponsor this newsletter

Your handler saved the data. But the email never sent. The notification never fired. And you have no idea why.

You have seen this bug before.

Everything looks fine. The record is in the database. But something that was supposed to happen after the save just did not.

Maybe the API crashed mid-request. Maybe an exception swallowed the side effect. Maybe it just ran out of memory.

This is not a code bug. It is an architectural gap. And the Outbox Pattern is how you close it.

---

What is the Outbox Pattern?

The idea is simple.

Instead of firing side effects directly after a save, you write them to a database table in the same transaction as your main data.

A background job then picks them up and processes them reliably.

Without Outbox:

Save User → Send Welcome Email ← email can fail silently

With Outbox:

(Save User + Insert Outbox Message) → Background Job → Send Welcome Email

The key word is same transaction. If the save fails, the outbox message never gets inserted. If the save succeeds, the message is guaranteed to be processed eventually.

---

The Problem Without It

Here is what most developers write first:

public async Task Handle(RegisterUserCommand command, CancellationToken ct)
{
    var user = User.Create(command.Email);
    context.Users.Add(user);
    await context.SaveChangesAsync(ct);
    await emailService.SendWelcomeEmailAsync(user.Email, ct);
}

This looks fine. But there are two problems.

If SaveChangesAsync throws, the email does not send. Fine, that is expected.

But if SendWelcomeEmailAsync throws after the save, your user is created and never gets the welcome email. And you will never know unless you check manually. Now imagine that email was an OTP for account verification the user is stuck and can never finish signing up.

Multiply this across ten features. Password resets, order confirmations, notification triggers. Every single one is a silent failure waiting to happen.

---

Setting Up the Outbox Table

Start with the entity:

public class OutboxMessage
{
    public Guid Id { get; set; }
    public string Type { get; set; }
    public string Payload { get; set; }
    public DateTime CreatedAt { get; set; }
    public DateTime? ProcessedAt { get; set; }
    public string? Error { get; set; }
    public int RetryCount { get; set; }
}

Apply it via migration:

CREATE TABLE OutboxMessages (
    Id UUID PRIMARY KEY,
    Type VARCHAR(255) NOT NULL,
    Payload TEXT NOT NULL,
    CreatedAt TIMESTAMP NOT NULL,
    ProcessedAt TIMESTAMP NULL,
    Error TEXT NULL,
    RetryCount INT NOT NULL DEFAULT 0
);

-- Your background job will run "WHERE ProcessedAt IS NULL ORDER BY CreatedAt" constantly.
-- A partial index keeps that query fast even after the table has millions of processed rows.
CREATE INDEX IX_OutboxMessages_Unprocessed
ON OutboxMessages (CreatedAt)
WHERE ProcessedAt IS NULL;

That partial index only covers unprocessed rows, so it stays small and cheap to scan no matter how large the table gets.

A partial index is an index built over a subset of a table; the subset is defined by a conditional expression (called the predicate of the partial index). The index contains entries only for those table rows that satisfy the predicate. Partial indexes are a specialized feature, but there are several situations in which they are useful.

---

Writing to the Outbox

Without using the outbox our code would look like this normally :

public async Task Handle(RegisterUserCommand command, CancellationToken ct)
{
    var user = User.Create(command.Email);
    context.Users.Add(user);
    await context.SaveChangesAsync(ct);
    
   // Separate operation — no guarantee this runs
    await emailService.SendWelcomeEmailAsync(user.Email, ct);
}

Instead, write the event to the outbox table in the same transaction as your business data:

public async Task Handle(RegisterUserCommand command, CancellationToken ct)
{
    var user = User.Create(command.Email);
    var welcomeEmail = new WelcomeEmailRequested(user.Email);
    var outboxMessage = new OutboxMessage
    {
        Id = Guid.NewGuid(),
        Type = typeof(WelcomeEmailRequested).AssemblyQualifiedName!,
        Payload = JsonSerializer.Serialize(welcomeEmail),
        CreatedAt = DateTime.UtcNow
    }; 
    context.Users.Add(user);
    context.OutboxMessages.Add(outboxMessage); 
    await context.SaveChangesAsync(ct);
}

No email call. No notification call. Just save and done.

Both the user and the outbox message are written atomically. If the transaction rolls back, both roll back. Nothing is lost.

A small but important detail: store AssemblyQualifiedName, not just the type’s short name (nameof(WelcomeEmailRequested) gives you "WelcomeEmailRequested" with no namespace or assembly info). Your background job needs to turn that string back into a real Type to deserialize the payload and Type.GetType(...) can only do that reliably if the string carries the full assembly info. Skip this and your job will throw InvalidOperationException: Unknown event type the first time it runs.

---

Processing the Outbox

A background job polls the table and processes unhandled messages:

[DisallowConcurrentExecution]
internal sealed class ProcessOutboxMessagesJob(AppDbContext dbContext, IMessageBus messageBus) : IJob
{
    public async Task Execute(IJobExecutionContext context)
    {
        var cancellationToken = context.CancellationToken;

        await using var transaction = await dbContext.Database.BeginTransactionAsync(cancellationToken);

        var messages = await dbContext.OutboxMessages
            .FromSqlRaw(
                """
                SELECT "Id", "Type", "Payload", "CreatedAt", "ProcessedAt", "Error", "RetryCount"
                FROM "OutboxMessages"
                WHERE "ProcessedAt" IS NULL
                ORDER BY "CreatedAt"
                LIMIT 20
                FOR UPDATE SKIP LOCKED
                """)
            .ToListAsync(cancellationToken);

        foreach (var message in messages)
        {
            try
            {
                var eventType = Type.GetType(message.Type)
                    ?? throw new InvalidOperationException($"Unknown event type '{message.Type}'.");

                var @event = JsonSerializer.Deserialize(message.Payload, eventType)
                    ?? throw new InvalidOperationException($"Could not deserialize payload for '{message.Type}'.");

                await messageBus.InvokeAsync(@event, cancellationToken);

                message.ProcessedAt = DateTime.UtcNow;
                message.Error = null;
            }
            catch (Exception ex)
            {
                message.Error = ex.Message;
                message.RetryCount++;
            }
        }

        await dbContext.SaveChangesAsync(cancellationToken);
        await transaction.CommitAsync(cancellationToken);
    }
}

The job runs every 10 seconds. It fetches up to 20 unprocessed messages, publishes them, and marks them processed. Oldest first, always.

If you’re already using Wolverine as your mediator (see my Wolverine as Mediator in .NET piece), IMessageBus.InvokeAsync here is the exact same call you’d use to dispatch a command in your handlers the outbox job is just another caller.

---

At-Least-Once Delivery: Your Consumers Must Be Idempotent

Some points worth considering, the outbox pattern does not guarantee a message is processed exactly once. It guarantees at least once.

Think about what happens if the job publishes the message successfully, then crashes before it can set ProcessedAt and commit. The next run picks up that same message and publishes it again.

That means SendWelcomeEmailAsync or whatever handler consumes the message needs to be safe to run twice. A few practical ways to get there:

• Check whether the action already happened before doing it again (e.g., “has this user already received a welcome email?”).

• Use a natural or generated idempotency key so duplicate sends are detected and skipped downstream.

• For anything transactional (charging a card, decrementing stock), make the operation itself idempotent rather than relying on the outbox to dedupe for you.

Preventing Duplicate Processing

If two instances of your app run at the same time, both might pick up the same message.

Use FOR UPDATE SKIP LOCKED to prevent that:

SELECT * FROM "OutboxMessages"
WHERE "ProcessedAt" IS NULL
ORDER BY "CreatedAt"
LIMIT 20
FOR UPDATE SKIP LOCKED

Now only one instance processes each message. The other skips it and moves on.

Note this solves concurrent duplicate processing across instances it does not replace the idempotency work above, which covers crash-and-retry duplicates on a single instance.

---

Keeping the Outbox Table Healthy

Nobody thinks about this until the table has ten million rows and the background job starts timing out.

Two habits keep it under control:

1. Cap retries. I have added a column named RetryCount in our entity use it for retries count. After a handful of failed attempts (e.g. 3 attempts), stop retrying automatically and flag the message for manual review instead of hammering a dependency that’s clearly down.

2. Archive or delete processed rows. Once a message is processed and you’re past the window where you’d ever need to debug it, delete it (or move it to a cold-storage table):

DELETE FROM "OutboxMessages"
WHERE "ProcessedAt" IS NOT NULL
AND "ProcessedAt" < NOW() - INTERVAL '7 days';

Run this as its own scheduled job, separate from the processor. A table that only ever grows will eventually make even the partial index expensive to maintain.

---

👉 Find complete running demo code at GitHub Issue #77

Summary

Here is everything covered in this issue:

• The problem: saving data and calling side effects separately creates a silent failure gap.

• The fix: write your messages to a database table in the same transaction as your business data, then process them asynchronously.

• OutboxMessage entity stores the event type (as an assembly-qualified name), serialized payload, and processed timestamp.

• Write the outbox message directly in your handler alongside your business data in one SaveChangesAsync call.

• A background job polls every 10 seconds, deserializes each message, publishes it, and marks it processed.

• FOR UPDATE SKIP LOCKED prevents duplicate processing across multiple app instances.

• Outbox delivery is at-least-once your consumers must be idempotent.

• A partial index and a cleanup job keep the table fast as it grows.

One table. One background job. Zero lost events.

There are 3 ways I can help you:

1. Enhance your .NET skills by subscribing to my YouTube Channel

2. Promote yourself to 10,000+ subscribers by sponsoring this newsletter

3. Have a software idea? Let’s turn it into a real product, Work With Me