Memory Crypt
Message Brokers - Part 1: RabbitMQ

Photo by Gavin Allanwood on Unsplash

Message Brokers - Part 1: RabbitMQ

Hemant Singh's photo
Hemant Singh
·

8 min read

Table of contents

This topic is categorized into following parts:
Message Brokers - Part 1: RabbitMQ
Message Brokers - Part 2: Apache Kafka
Message Brokers - Part 3: IBM MQ

Message Brokers - An Overview

Message brokers are a type of middleware used in computer systems to facilitate communication and data exchange between different software applications or components. They play a crucial role in decoupling the sender and receiver of messages, allowing them to communicate asynchronously and independently. Message brokers help manage the complexity of inter-application communication by providing a central hub through which messages can be sent, received, and routed.

Here's how a message broker generally works:

  1. Publishers: Applications or systems that generate messages are known as publishers. They send messages to the message broker without having to know who or what will receive those messages.

  2. Brokers: The message broker is a central intermediary that receives messages from publishers and then routes or delivers those messages to the appropriate recipients, known as subscribers.

  3. Subscribers: These are the applications or systems that are interested in receiving and processing specific types of messages. Subscribers connect to the message broker and specify the types of messages they want to receive.

Key features and benefits of message brokers include:

  • Asynchronous Communication: Message brokers enable asynchronous communication, where applications can send and receive messages independently of each other. This is especially useful in scenarios where the sender and receiver are not directly connected or where the receiver might not be available all the time.

  • Decoupling: By using a message broker, publishers and subscribers are decoupled from each other. Publishers do not need to know who the subscribers are, and vice versa. This allows for more flexible and maintainable systems.

  • Scalability: Message brokers can handle high volumes of messages and distribute them efficiently to subscribers. This is crucial in scenarios with a large number of applications or when dealing with spikes in message traffic.

  • Message Persistence: Many message brokers offer features to store messages temporarily or permanently, ensuring that messages are not lost in case of system failures or downtime.

  • Routing and Filtering: Message brokers often provide mechanisms for routing messages to specific subscribers based on predefined criteria or topics. Subscribers can express interest in specific message types, allowing for fine-grained control over which messages they receive.

Some popular message broker systems include Apache Kafka, RabbitMQ, ActiveMQ, AWS SQS (Simple Queue Service), and more. These systems may vary in terms of their features, performance, scalability, and supported messaging patterns, so the choice of a message broker depends on the specific requirements of the application or system.

RabbitMQ is a widely used open-source message broker that enables efficient communication between different components of distributed systems. It implements the Advanced Message Queuing Protocol (AMQP) to facilitate reliable messaging between producers and consumers. RabbitMQ acts as an intermediary, ensuring that messages are safely transferred between various applications, services, and microservices.

In this article, we will delve into the core concepts of RabbitMQ and provide multiple examples in C# to showcase its usage.

RabbitMQ - Architecture and Core Concepts

As per the official RabbitMQ page - messages are published to exchanges, which are often compared to post offices or mailboxes. Exchanges then distribute message copies to queues using rules called bindings. Then the broker either deliver messages to consumers subscribed to queues, or consumers fetch/pull messages from queues on demand.

Source: rabbitmq.com/tutorials/amqp-concepts.html

1. Queues

Queues in RabbitMQ store messages sent by producers. Consumers retrieve and process these messages. Queues act as buffers, enabling asynchronous communication between different components.

2. Exchanges

Exchanges are responsible for routing messages from producers to queues. They determine how messages should be distributed based on routing keys, message attributes, and binding rules.

3. Bindings

Bindings connect queues to exchanges and define the routing rules that direct messages to their appropriate destination queues.

4. Routing Keys

Routing keys are attributes attached to messages that exchanges use to determine which queues should receive the message.

5. Consumers

Consumers are applications or components that subscribe to queues and retrieve messages for processing.

6. Publishers

Publishers are applications or components that send messages to exchanges, which then route them to the appropriate queues.

7. Broker

The broker is the core messaging server in RabbitMQ. It receives messages from producers, routes them through exchanges, and delivers them to consumers via queues. The broker also manages virtual hosts, connections, and users.

8. Virtual Hosts

A Virtual Host or vhost is a logical grouping and isolation mechanism within a RabbitMQ broker. Each virtual host is essentially a separate instance of the RabbitMQ server that operates independently from other virtual hosts. Virtual hosts provide a way to isolate messaging resources within a RabbitMQ instance. Each virtual host has its own exchanges, queues, and permissions.

9. Channels

Channels are lightweight communication channels within a connection. Multiple channels can be created within a single connection, allowing applications to use parallelism for efficient message processing.

10. Acknowledgements

Acknowledgements (acks) are used to ensure message reliability. Consumers send an ack to the broker after successfully processing a message. If a message is not acked, RabbitMQ assumes it needs to be reprocessed.

Setting up RabbitMQ on your local machine

Setting up RabbitMQ on your machine involves several steps, including installing the RabbitMQ server, configuring it, and starting the server. Here's a step-by-step guide to help you set up RabbitMQ on your machine:

Note: These instructions are based on a generic installation process. Depending on your operating system and specific requirements, some steps may vary slightly.

1. Install Erlang: RabbitMQ is built using Erlang, so you'll need to install Erlang first. Visit the official Erlang website (erlang.org/downloads) to download and install the appropriate version for your operating system.

2. Install RabbitMQ Server: After installing Erlang, you can proceed to install RabbitMQ itself.

  • Visit the official RabbitMQ website (rabbitmq.com/download.html) to download the RabbitMQ installer for your operating system.

  • Follow the installation instructions for your specific platform.

3. Start RabbitMQ Server: After installation, you'll need to start the RabbitMQ server.

  • On Windows: Look for the RabbitMQ Server application in your Start menu and launch it.

  • On Linux/Mac: Open a terminal and navigate to the RabbitMQ installation directory. Use the command rabbitmq-server to start the server.

4. Access RabbitMQ Management Console: RabbitMQ comes with a web-based management console that allows you to monitor and manage your RabbitMQ instance.

  • Open a web browser and navigate to http://localhost:15672/.

  • Log in using the default credentials: Username: guest, Password: guest.

5. Create a User and Set Permissions: For security reasons, it's recommended to create a new user and give it appropriate permissions.

  • Log in to the RabbitMQ Management Console.

  • Go to the "Admin" tab.

  • Under the "Add a user" section, create a new user with a username and password.

  • After creating the user, you can assign it to specific virtual hosts and configure permissions as needed.

6. Install a RabbitMQ Client Library: If you plan to interact with RabbitMQ programmatically, you'll need a client library for your programming language. For example, you can use the RabbitMQ.Client library for .NET/C#.

  • Install the library using NuGet: Install-Package RabbitMQ.Client.

7. Start Using RabbitMQ: With RabbitMQ up and running, you can start producing and consuming messages using your chosen programming language and client library.

Getting started with RabbitMQ in .NET

To begin using RabbitMQ in C#, you need the RabbitMQ.Client NuGet package. You can install it using the following command:

dotnet add package RabbitMQ.Client

Sending and Receiving messages using queues

Let us start with a basic example of sending and receiving messages using queues. Here, the producer sends a message to the "my_queue" queue, and the consumer retrieves and processes the message from the same queue.

Producer (Sender)

using RabbitMQ.Client;

class Producer
{
    static void Main()
    {
        var factory = new ConnectionFactory() { Hostname = "localhost" };
        using var connection = factory.CreateConnection();
        using var channel = connection.CreateModel();

        channel.QueueDeclare("my_queue", false, false, false, null);

        string messge = "Hello, RabbitMQ!";
        var body = Encoding.UTF8.GetBytes(message);

        channel.BasicPublish("", "my_queue", null, body);
        Console.WriteLine("Sent: {0}", message);
    }

}

Consumer (Receiver)

using RabbitMQ.Client;
using RabbitMQ.Client.Events;

class Consumer
{
    static void Main()
    {
        var factory = new ConnectionFactory() { HostName = "localhost" };
        using var connection = factory.GetConnection();
        using var channel = connection.CreateModel();

        channel.QueueDeclare("my_queue", false, false, false, null);

        var conumer = new EventingBasicConsumer(channel);
        consumer.Received += (sender, e) =>
        {
            var body = e.Body.ToArray();
            var message = Encoding.UTF8.GetString(body);
            Console.WriteLine("Received: {0}",message);
        };

        channel.BasicConsume("my_queue", true, consumer);
        Console.WriteLine("Waiting for messages...");
        Console.ReadLine();
    }
}

Sending and Receiving messages using Publish/Subscribe pattern

The publish/subscribe pattern allows a single message to be broadcasted to multiple consumers. In RabbitMQ, this is achieved using fanout exchanges.

In this example, the publisher sends messages to the "logs" exchange, and the subscriber(s) listen to messages from this exchange.

Publisher

using RabbitMQ.Client;

class Publisher
{
    static void Main()
    {
        var factory = new ConnectionFactory() { Hostname = "localhost" };
        using var connection = factory.CreateConnection();
        using var channel = connection.CreateModel();

        var exchangeName= "logs";
        channel.ExchangeDeclare(exchangeName, ExchangeType.Fanout);

        string message = "Log message!";
        var body = Encoding.UTF8.GetBytes(message);

        channel.BasicPublish(exchangeName, "", null, body);
        Console.WriteLine("Sent: {0}", message);
    }
}

Subscriber:

using RabbitMQ.Client;
using RabbitMQ.Client.Events;

class Subscriber
{
    static void Main()
    {
        var factory = new ConnectionFactory() { HostName = "localhost" };
        using var connection = factory.CreateConnection();
        using var channel = connection.CreateModel();

        var exchangeName= "logs";
        channel.ExchangeDeclare(exchangeName, ExchangeType.Fanout);
        var queueName = channel.QueueDeclare().QueueName;
        channel.QueueBind(queueName, exchangeName, "");

        var consumer = new EventingBasicConsumer(channel);
        consumer.Received += (sender, e) =>
        {
            var body = e.Body.ToArray();
            var message = Encoding.UTF8.GetString(body);
            Console.WriteLine("Received: {0}", message);
        };

        channel.BasicConsume(queueName, true, consumer);
        Consol.WriteLine("Waiting for messages...");
        Console.WriteLine();
    }
}
rabbitmq.NETC#queuemessaging