AMQP: Reliable Messaging with Exchanges, Queues and Acknowledgements

AMQP (Advanced Message Queuing Protocol) is an open wire protocol for reliable, asynchronous messaging through a broker. A producer hands a message to the broker and moves on; the broker stores it, routes it to one or more queues and delivers it to consumers, who acknowledge each message when the work is done. Nothing is lost if a consumer is slow, busy or down. AMQP is the protocol behind RabbitMQ and Azure Service Bus, and the backbone of many order, payment and background-job pipelines.

This post is part of the API communication protocols series.

At a glance

	AMQP
Type	Binary application-layer messaging protocol
Created	2003 by John O’Hara at JPMorgan Chase; working group with iMatix, Red Hat, Cisco and others from 2006
Versions	0-9-1 (2008, the RabbitMQ model) and 1.0 (OASIS 2012, ISO/IEC 19464 in 2014), which are different protocols
Transport	TCP on port 5672, TLS on 5671; AMQP over WebSocket exists
Payload	Opaque bytes plus properties (content type, message ID, correlation ID, headers, TTL)
Contract	None; routing keys and payload formats are conventions (AsyncAPI can document them)
Delivery	At least once with acknowledgements; at most once with auto-ack
Brokers	RabbitMQ, Azure Service Bus, Apache ActiveMQ and Qpid, Solace
Best for	Work queues, decoupling services, commands and business events that must not be lost

What AMQP is

The AMQP 0-9-1 model

RabbitMQ made the AMQP 0-9-1 model the one most developers know. It has four building blocks:

Concept	Role
Exchange	Receives messages from producers and routes them. Producers never publish to a queue directly
Binding	A rule linking an exchange to a queue, usually with a binding key or pattern
Queue	Stores messages in order until a consumer acknowledges them
Consumer	Receives messages pushed from a queue and acknowledges or rejects each one

Exchange type	Routing rule	Example
Direct	Routing key equals binding key	invoice.create goes only to the invoicing queue
Fanout	Every bound queue gets a copy; key ignored	Broadcast a cache invalidation to every instance
Topic	Pattern match on dot-separated keys: * = one word, # = zero or more words	order.* matches order.paid and order.created
Headers	Match on message header values instead of the key	Route by region=eu and priority=high

The model as UML

classDiagram
    direction LR
    class Connection {
        TCP or TLS
        heartbeat
    }
    class Channel {
        prefetch
        confirm mode
    }
    class Exchange {
        name
        type
        durable
    }
    class Binding {
        bindingKey
        arguments
    }
    class Queue {
        name
        durable
        dead letter exchange
        message TTL
    }
    class Message {
        routingKey
        properties
        body
    }
    Connection "1" *-- "*" Channel : multiplexes
    Channel --> Exchange : publish
    Exchange "1" --> "*" Binding
    Binding "*" --> "1" Queue
    Queue "1" o-- "*" Message : stores
    Channel --> Queue : consume and ack

A connection is one TCP connection; channels are lightweight virtual connections inside it, so an application can use many threads without opening many sockets.

AMQP 1.0 is different

AMQP 1.0, the OASIS and ISO standard, defines only the wire protocol between two endpoints: connections, sessions, links, transfers, flow control and settlement (acknowledgement). It does not define exchanges or bindings; each broker decides how addresses map to queues and topics. Azure Service Bus, Azure Event Hubs, ActiveMQ Artemis and Solace speak 1.0, and RabbitMQ supports 1.0 natively since version 4.0. Libraries such as Apache Qpid Proton and the Azure SDKs implement it.

Why it was developed

In the early 2000s, banks moved huge volumes of trade and payment messages through proprietary message-oriented middleware such as IBM MQSeries and TIBCO Rendezvous. Each product had its own protocol and API, licences were expensive, and connecting systems that used different products needed bridges. John O’Hara at JPMorgan Chase in London started AMQP in 2003 to create an open, interoperable wire protocol, so that any client could talk to any broker, as HTTP had done for the web.

The AMQP working group formed in 2006 with iMatix, Red Hat, Cisco, Microsoft and others. RabbitMQ, written in Erlang by Rabbit Technologies in 2007, implemented the pre-1.0 versions and became the most popular open-source broker, which is why the 0-9-1 model is still what most people mean by “AMQP”. The standardised AMQP 1.0, approved by OASIS in 2012, took a more general approach and was adopted by Microsoft for Azure Service Bus.

How it works

The animation follows one order.paid message through a topic exchange to two queues. Switch on the poison-message mode to see dead-lettering:

1. Publish with a routing key: The Order service publishes a persistent message with routing key order.paid to the orders exchange. It never names a queue.
2. Exchange receives it: With publisher confirms on, the broker acknowledges the publish once the message is safely routed and stored.
3. Bindings decide: A topic exchange matches the key against bindings: billing is bound with order.* and shipping with order.paid, so both get a copy.
4. Deliver to consumers: Each queue pushes messages to its consumers, at most prefetch (basic.qos) unacknowledged at a time, so a slow consumer is not flooded.
5. Billing acknowledges: Billing finishes and sends basic.ack. Only now does the broker delete the message from the billing queue.
6. Shipping acknowledges (ack): Shipping acks too. If a consumer had crashed before acking, the message would be redelivered to another consumer.
7. Poison message (dlq): Shipping cannot parse the message, so it rejects it with requeue=false instead of looping on it forever.
8. Dead-lettered (dlq): The queue has a dead-letter exchange configured, so the broker re-routes the rejected message to a dead-letter queue for inspection and replay.

Reliable delivery end to end

Reliability needs three things working together: publisher confirms (the broker tells the producer it has the message), durable queues with persistent messages (it survives a broker restart), and manual consumer acknowledgements (it is only removed when the work is done).

sequenceDiagram
    autonumber
    participant P as Producer
    participant X as Exchange
    participant Q as Queue
    participant C as Consumer
    P->>X: basic.publish, routing key order.paid, persistent
    X->>Q: route via binding order.*
    X-->>P: basic.ack, publisher confirm
    Q->>C: basic.deliver, delivery tag 1
    alt processing succeeds
        C-->>Q: basic.ack 1
        Note over Q: message removed
    else consumer crashes before ack
        Note over Q: connection lost, message requeued
        Q->>C: redeliver to another consumer, redelivered flag set
    else message cannot be processed
        C-->>Q: basic.nack 1, requeue false
        Note over Q: dead-lettered to the DLX
    end

A consumer in Python

import json, pika

conn = pika.BlockingConnection(pika.URLParameters("amqps://app:secret@mq.example.com:5671/%2F"))
ch = conn.channel()
ch.exchange_declare("orders", exchange_type="topic", durable=True)
ch.exchange_declare("orders.dlx", exchange_type="fanout", durable=True)
ch.queue_declare("shipping", durable=True, arguments={
    "x-queue-type": "quorum",                 # replicated, Raft-based queue
    "x-dead-letter-exchange": "orders.dlx",
})
ch.queue_bind("shipping", "orders", routing_key="order.paid")
ch.basic_qos(prefetch_count=20)               # at most 20 unacked messages per consumer

def handle(channel, method, props, body):
    try:
        create_shipment(json.loads(body))     # must be idempotent: redelivery can happen
        channel.basic_ack(method.delivery_tag)
    except ValueError:
        channel.basic_nack(method.delivery_tag, requeue=False)   # poison message: dead-letter it

ch.basic_consume("shipping", handle)
ch.start_consuming()

Messaging patterns

Pattern	How it is built	Example
Work queue (competing consumers)	One queue, many consumers; each message goes to one of them	Image resizing workers
Publish/subscribe	Fanout or topic exchange, one queue per subscriber	order.paid to billing, shipping and analytics
Routing	Direct or topic exchange with specific keys	Logs by severity: log.error, log.info
Request/reply (RPC)	reply_to and correlation_id properties, a reply queue	Synchronous-looking call over the broker
Delayed retry	TTL plus dead-letter exchanges, or a delayed-message plugin	Retry a failed payment after 5 minutes

Where it is used

• Microservices that communicate through commands and events instead of synchronous calls.
• Background jobs: emails, PDF generation, video transcoding, imports.
• Financial services: trade processing, payments, settlement messages.
• Enterprise integration on Azure Service Bus with sessions, transactions and scheduled messages.
• Task frameworks: Celery (Python) commonly uses RabbitMQ as its broker; MassTransit and NServiceBus (.NET) and Spring AMQP (Java) build on it.

Real-life examples

RabbitMQ behind Celery and microservice back ends

A typical web application uses RabbitMQ with Celery or a similar framework: the web request publishes a send_invoice task and returns immediately, and a pool of workers consumes the queue. If the workers are down for maintenance, tasks wait in a durable queue; if a worker crashes mid-task, the unacked message is redelivered. Many companies use exactly this for order pipelines: the checkout service publishes order.paid to a topic exchange, and billing, shipping, loyalty and analytics each bind their own queue, so a new consumer can be added without changing the producer.

Azure Service Bus

Azure Service Bus is Microsoft’s managed enterprise message broker, and AMQP 1.0 is its primary protocol. Applications use queues for point-to-point commands and topics with subscriptions (and SQL-like filter rules) for publish/subscribe. It adds features enterprises need on top of the protocol: sessions for ordered processing per key (for example all messages of one order), dead-letter queues on every queue and subscription, scheduled messages, duplicate detection by message ID, and transactions across several operations. Because the protocol is the open standard, clients from Java, .NET, Python, JavaScript and Go use the same wire format.

Strengths and limitations

Strengths	Limitations
Producers and consumers decoupled in time: neither waits for the other	The broker is extra infrastructure to run, monitor and scale
Durable queues, acks and confirms: messages are not lost	At-least-once delivery means consumers must be idempotent
Flexible routing with exchanges and bindings	Not built for replaying history; consumed messages are gone (unlike Kafka)
Back-pressure through prefetch and queue limits	Two incompatible protocol versions cause confusion
Dead-lettering, TTL, priorities and request/reply	Ordering is only guaranteed per queue, with a single consumer

Best practices

• Use publisher confirms, durable queues, persistent messages and manual acks for anything that matters.
• Make consumers idempotent (deduplicate by message ID) because redelivery will happen.
• Set a prefetch count so one consumer does not hoard messages.
• Configure dead-letter exchanges and alert on dead-letter queue depth; never requeue poison messages forever.
• Use quorum queues in RabbitMQ for replicated, highly available queues (classic mirrored queues are removed in RabbitMQ 4.0).
• Reuse connections, open channels per thread, and enable heartbeats.
• Keep messages small; put large payloads in object storage and send a reference.
• Monitor queue depth, consumer count, unacked messages and publish/deliver rates.

When to use AMQP, and when not

Use AMQP when work must be done reliably but not immediately, when one event has several independent consumers, when you need to absorb load spikes, or when you want services to keep working while others are down.

Prefer gRPC or REST when the caller needs an answer now, MQTT for huge numbers of small devices, and Kafka when events must be stored and replayed by many consumers.

Related posts

• API Communication Protocols: overview
• MQTT
• Webhooks
• Comparison of API protocols