Queue Topic

Origin & evolution

Message-oriented middleware was codified by Gregor Hohpe and Bobby Woolf in Enterprise Integration Patterns (2003), which described 65 patterns for moving data through messaging✉️MessagingGo-To-MarketMessaging framework and key messagesView reference → systems. Their distinction between a point-to-point channel (one message, one consumer) and a publish-subscribe channel (one message, many subscribers) still names the two shapes most systems pick between.

Apache Kafka, open-sourced in 2011, reframed the topic as a durable, replayable log. A traditional queue deletes a message once it is consumed; Kafka retains messages for a configured period regardless of who has read them, so any number of consumer groups can read the same stream independently and a new consumer can replay history. That shift, from transient delivery to a retained log, is why "topic" now often implies a durable stream that many consumers can replay.

Delivery semantics are where teams get burned. Kafka guarantees at-least-once delivery by default; exactly-once is achievable with idempotent producers and transactional writes, but it is not the default path, and most systems that claim exactly-once actually run at-least-once with downstream deduplication. Designing consumers to be idempotent is the honest baseline.

How it works in practice

An e-commerce platform publishes an OrderPlaced event to a topic. Three consumer groups subscribe: fulfilment, email, and analytics, each reading the same stream at its own pace. On a flash-sale morning the order service emits 4,000 events per second while the email service can only process 600. With a synchronous call this back-pressure would stall checkout. With a retained topic, the email consumer simply lags behind, draining the backlog over the next twenty minutes while orders keep flowing. Because delivery is at-least-once, the email consumer deduplicates on order_id, so a redelivered event after a restart does not send a second confirmation.

Queue topic vs. its neighbours

• Service is a unit of deployable behaviour that publishes or consumes messages. A queue topic is the channel between services; it holds no business logic, only the contract that messages will be delivered under stated guarantees.
• Domain event📡Domain EventEngineeringAn event published when something happens in the domainView reference → is the meaning of a message ("an order was placed"), a fact about the domain. A queue topic is the transport that carries that fact; one topic may carry many event types, and the same event may be replayed long after it occurred.
• Integration pattern🔗Integration PatternEngineeringAn integration pattern (saga, event sourcing, etc.)View reference → is the broader style (messaging versus RPC, file transfer, or shared database). A queue topic is the concrete mechanism that realises the messaging style; choosing it is choosing decoupling and asynchrony over a direct call.

In the graph

In the Unified Product Graph, queue_topic📬Queue TopicEngineeringA message queue topic is a leaf in the Engineering & Platform region, whose mental model runs from bounded context🔲Bounded ContextEngineeringA DDD bounded context defining a service boundaryView reference → through service to the channels and schemas beneath it. Its defining edge is service_publishes_to_queue_topicServicepublishes toQueue Topichierarchy, a hierarchy edge that records which service owns the publishing side. That single relationship makes the decoupling legible: a reader can see every topic a service emits to, and by extension trace the asynchronous seams where back-pressure absorbs load and where delivery guarantees, in place of direct calls, hold the system together.