Service Bus queues, topics, subscriptions, and dead-letter queues

Overview

Azure Service Bus is a fully managed enterprise message broker. It is used when applications need durable asynchronous communication, load leveling, competing consumers, publish-subscribe routing, delayed processing, dead-letter handling, transactions, sessions, duplicate detection, or reliable integration between independently deployed systems.

The core entities are:

• A queue receives messages from producers and delivers each message to one consumer.
• A topic receives messages from producers and fan-outs copies to subscriptions.
• A subscription behaves like a virtual queue under a topic.
• A dead-letter queue stores messages that could not be delivered or processed successfully.

For interviews, Service Bus questions usually test whether the candidate understands brokered messaging rather than just "send a message and read it." Strong answers cover temporal decoupling, at least once delivery, peek-lock settlement, duplicate handling, max delivery count, DLQ operations, topic filters, subscription isolation, sessions, message size limits, and idempotent consumers.

Core Concepts

Service Bus as a Broker

Service Bus accepts messages from producers and stores them durably until consumers receive them. This gives the system:

• Temporal decoupling between producer and consumer availability.
• Load leveling during traffic spikes.
• Competing-consumer scaling.
• Reliable retries when consumers fail.
• Loose coupling between applications.
• Broker-level routing through topics and subscriptions.

Use Service Bus for commands, work items, and high-value business messages where the sender expects some consumer to perform work.

Queue

A queue is a point-to-point messaging entity. Producers send messages to the queue, and consumers compete for messages. Each message is processed by only one consumer instance.

Common queue scenarios:

• Process an order.
• Generate a report.
• Send an email.
• Run an image conversion.
• Sync a record with another system.
• Buffer a burst of API requests.

Queues are a good default when there is one logical type of work and a pool of workers can process items independently.

Competing Consumers

Multiple consumers can read from the same queue. Service Bus locks each message for one consumer while it is being processed. This lets a team scale workers horizontally without designing custom coordination.

Benefits:

• More workers increase throughput.
• Failed workers do not permanently own messages.
• Consumers process at their own rate.
• The queue buffers spikes.

Risks:

• Processing order is not guaranteed across multiple consumers unless sessions are used.
• Consumers must be idempotent.
• Downstream services can still be overloaded if worker count is too high.

Topic

A topic is a publish-subscribe entity. Producers send messages to the topic. Service Bus copies matching messages into one or more subscriptions.

Use a topic when multiple independent consumers need to react to the same business message. For example:

• Billing records the order.
• Fulfillment reserves inventory.
• Analytics updates projections.
• Notifications sends a confirmation.

The publisher sends one message and does not need to know every consuming system.

Subscription

A subscription is a durable receiver under a topic. It behaves like a queue for one subscriber or one subscriber group.

Each subscription has:

• Its own message backlog.
• Its own lock and delivery count behavior.
• Its own dead-letter queue.
• Optional filters and actions.
• Its own scaling and monitoring surface.

One slow subscription does not directly block another subscription, although all of them still share the namespace and topic capacity.

Subscription Filters

Subscription filters decide which messages are copied to a subscription. Filters can use message system properties and application properties.

Example message:

var message = new ServiceBusMessage(BinaryData.FromObjectAsJson(order))
{
    MessageId = $"order-{order.Id}-created",
    Subject = "OrderCreated",
    ContentType = "application/json"
};

message.ApplicationProperties["region"] = order.Region;
message.ApplicationProperties["priority"] = order.Priority;

A subscription might receive only OrderCreated messages for a region. Keep filters simple. Service Bus is a broker, not a business-rules engine.

Queue Versus Topic

Use a queue when one logical consumer should process each message. Use a topic when multiple independent consumers need their own copy.

A topic subscription can be processed by competing consumers, so topics do not prevent horizontal scaling.

Receive Modes

Service Bus supports two receive modes:

• Receive and delete.
• Peek-lock.

Receive and delete removes the message as soon as Service Bus sends it to the consumer. It is fast but can lose messages if the consumer crashes before processing.

Peek-lock is the usual mode for reliable processing. Service Bus locks the message, the consumer processes it, then explicitly settles it.

Peek-Lock Settlement

In peek-lock mode, the consumer decides what happens next:

• Complete: processing succeeded; remove the message.
• Abandon: release the lock and make the message available again.
• DeadLetter: move the message to the DLQ because retrying will not help.
• Defer: hide the message until it is requested later by sequence number.

Example:

await using var client = new ServiceBusClient(
    fullyQualifiedNamespace,
    new DefaultAzureCredential());

ServiceBusReceiver receiver = client.CreateReceiver("orders");
ServiceBusReceivedMessage message = await receiver.ReceiveMessageAsync();

try
{
    await ProcessOrderAsync(message, cancellationToken);
    await receiver.CompleteMessageAsync(message, cancellationToken);
}
catch (ValidationException ex)
{
    await receiver.DeadLetterMessageAsync(
        message,
        deadLetterReason: "InvalidPayload",
        deadLetterErrorDescription: ex.Message,
        cancellationToken: cancellationToken);
}
catch
{
    await receiver.AbandonMessageAsync(message, cancellationToken);
    throw;
}

Lock Duration and Renewal

A lock is temporary. If processing takes longer than the lock duration and the lock is not renewed, the message becomes visible again and can be delivered to another consumer.

Good practice:

• Set the lock duration longer than normal processing time.
• Use automatic lock renewal for long-running handlers.
• Keep handlers bounded and observable.
• Break very long work into smaller messages.
• Make processing idempotent because lock loss can cause redelivery.

At Least Once Delivery

Service Bus reliable processing is at least once. A message can be delivered more than once if:

• The consumer crashes after doing work but before completing the message.
• The lock expires.
• Settlement fails because of a transient network issue.
• The sender retries after an uncertain send outcome.

Consumers must be idempotent. The broker can help with duplicate detection at send time, but it cannot make every consumer side effect exactly once.

Sessions and Ordering

Service Bus sessions group related messages by SessionId. A consumer locks a session and processes its messages as a group.

Use sessions when:

• Messages for the same aggregate must be processed in order.
• A workflow needs session state.
• Only one consumer should process a related stream at a time.

Avoid sessions when messages are independent and maximum parallelism matters. A hot session can become a bottleneck.

Dead-Letter Queue

Every queue and subscription has a dead-letter subqueue. It is used for messages that cannot be delivered or cannot be processed.

Service Bus can dead-letter messages when:

• The message exceeds maximum delivery count.
• The message expires and dead-lettering on expiration is enabled.
• The message violates entity rules.
• Auto-forwarding fails.
• A session-enabled entity receives a message without a session ID.

Applications can also explicitly dead-letter messages that are invalid or permanently unprocessable.

DLQ Is Not a Trash Can

The DLQ is an operational backlog. Messages remain there until they are inspected and completed from the DLQ.

A healthy DLQ process includes:

• Alerts on nonzero or rising DLQ counts.
• Reason codes and descriptions.
• Payload inspection with privacy controls.
• Categorization into retryable, data-fixable, and discardable cases.
• Safe resubmission tooling.
• Runbooks for common causes.

Ignoring DLQ messages hides production defects.

Reading from the DLQ

Use a DLQ receiver rather than manually constructing entity paths in application code.

var options = new ServiceBusReceiverOptions
{
    SubQueue = SubQueue.DeadLetter
};

ServiceBusReceiver receiver =
    client.CreateReceiver("orders", options);

ServiceBusReceivedMessage dlqMessage =
    await receiver.ReceiveMessageAsync(cancellationToken: cancellationToken);

For a topic subscription, create the receiver with topic name, subscription name, and the same subqueue option.

Message Design

A Service Bus message has a binary body plus broker and application properties. Good messages usually include:

• Stable MessageId.
• CorrelationId or trace context.
• Subject that names the event or command type.
• ContentType.
• Tenant or partition key when needed.
• Schema version.
• Small payload or a claim-check pointer for large payloads.

Avoid putting large documents in the message body. Store large payloads in Blob Storage and send a pointer.

Transactions

Service Bus supports transactional operations within Service Bus. This is useful for patterns such as receive from one entity and send to another atomically within the broker.

Do not assume a normal transaction across Service Bus, SQL, HTTP APIs, and external systems. For cross-resource workflows, use idempotency, outbox or inbox tables, sagas, and reconciliation.

Security

Prefer Microsoft Entra authentication and managed identities for Azure-hosted workloads. Assign least-privilege data-plane roles.

Use Shared Access Signatures only when appropriate, and scope them narrowly. Separate send and receive permissions. Do not give worker identities permission to manage namespace configuration unless they need it.

Common Mistakes

• Using receive and delete for business-critical work.
• Treating at least once as exactly once.
• Completing a message before durable side effects are committed.
• Dead-lettering transient failures too early.
• Retrying poison messages forever.
• Using one subscription for multiple independent consumers that need separate backlogs.
• Placing complex business routing in subscription filters.
• Forgetting every subscription has its own DLQ.
• Ignoring lock renewal for long processing.
• Sending large binary payloads directly through Service Bus.

Best Practices

• Use queues for work distribution and topics for fan-out.
• Use peek-lock for reliable processing.
• Make consumers idempotent.
• Set meaningful MessageId, CorrelationId, and Subject.
• Keep messages small and versioned.
• Use DLQ alerts and replay tooling.
• Set max delivery count based on failure type and recovery time.
• Use sessions only when ordering or grouped state is required.
• Monitor active messages, scheduled messages, dead-letter counts, delivery counts, lock lost errors, and processing latency.
• Use the latest Azure SDK libraries.