Create an ActiveMQ client using Python

In modern organizations, applications are broken down into services that can be independently developed, deployed and maintained. These services run as autonomous processes and communicate with each other through APIs. The service owner can choose a programming language that is most effective to solve the specific problem at hand, irrespective of what language other services are built in.

This also makes scaling easier, because organizations can choose to scale the components that are being used to their full capacity, instead of scaling the whole application. Such an architecture is called the microservice architecture.

Microservice Architecture

That said, not all services need to run synchronously. For a customer-facing web application, the web server only needs to talk to a handful of services, if at all, before sending a response back to the user. Majority of the services that make up a microservice architecture can bear some latency without compromising the overall performance of the application. So, these services can be invoked asynchronously by the calling service.

Message brokers are a key component of a microservice architecture, that allows for such asynchronous communication between services.

What Are Message Brokers?

The message broker can implement the message buffer in two ways:

  1. Queues, or
  2. Topics

Both are places where a producer or publisher sends messages to for further relay. The difference lies in who receives these messages.

The Queue Architecture:

Source: https://aws.amazon.com/message-queue/

Queues can be used by multiple clients to send and receive messages, however each message is processed only once by sending it to exactly one client. In the queue architecture, the client sending the message is known as a ‘Producer’ and the client receiving the message is known as a ‘Consumer’.

The Topic Architecture:

Source: https://aws.amazon.com/pub-sub-messaging/

In the Topic architecture, multiple clients can receive the same message. The clients who wish to receive the messages, inform the message broker, that they would like to ‘subscribe’ to a given ‘topic’ configured on the broker. Whenever a client sends a message on the topic, the message broker will broadcast the message to all clients who have subscribed to this topic.

A client sending the messages in the topic architecture is known as a ‘Publisher’ and the receiving client is known as a ‘Subscriber’.

Below are some of the most popular open source message brokers:

  1. Apache ActiveMQ
  2. RabbitMQ
  3. Apache Kafka
  4. Kestrel

Let’s take a closer look at ActiveMQ in this article.

ActiveMQ

In this article, we will see how easy it is to implement a Python client using ActiveMQ message broker.

If you would like to code along, ensure that you have Python installed on your system. You can download ActiveMQ and install it on your native system or an Ubuntu virtual machine, using the instructions available here.

Ensure that your ActiveMQ installation is up and running and that you can access its web console through https://localhost:8161/.

Create Queue and Topic

  1. Click on the ‘Queues’ link from the top navbar
  2. On the Queues page, enter name of the queue you want to create in the textbox labeled ‘Queue Name’

Follow the same steps for Topic by clicking on the ‘Topics’ link on the navbar:

Now that your test queue and topic is created, you can go ahead to create the clients for interacting with them.

Create Python Clients

pip install stomp.py 

We will create two Python clients, one to demonstrate the Queue architecture and another for the Topic architecture.

Let’s start with the queue architecture first. The below code implements both, a producer and a consumer, for a queue. I have included comments inline for explaining what code does.

import time 
import sys
import stomp
# Create a Listener class inheriting the stomp.ConnectionListener class
# stomp.ConnectionListener class definition can be found here:
# https://github.com/jasonrbriggs/stomp.py/blob/2435108cfc3eb4bd6477653b071e85acd6a2f211/stomp/listener.py
class Listener(stomp.ConnectionListener):
# Override the methods on_error and on_message provides by the
# parent class
def on_error(self, headers, message):
print('received an error "%s"' % message)
# Print out the message received def on_message(self, headers, message):
print('received a message "%s"' % message)
# Decare hosts as an array of tuples containing the ActiveMQ server # IP address or hostname and the port number
hosts = [('localhost', 61616)]
# Create a connection object by passing the hosts as an argument
conn = stomp.Connection(host_and_ports=hosts)
# Tell the connection object to listen for messages using the
# Listener class we created above
conn.set_listener('', Listener())
# Initiate the connection with the credentials of the ActiveMQ server
conn.start()
conn.connect('admin', 'admin', wait=True)
# Register a consumer with ActiveMQ. This tells ActiveMQ to send all # messages received on the queue 'queue-1' to this listenerconn.subscribe(destination='/queue/queue-1', id=1, ack='auto')# Act as a message producer and send a message the queue queue-1
# The actual message to be sent is picked up from the command line arguments
# Say, if you want to send the message "Hello Queue", you will run
# the code aspython queue.py "Hello Queue"
conn.send(body=' '.join(sys.argv[1:]), destination='/queue/queue-1')# When this message is received by the listener, it will be handled
# by the on_message method we defined above.
# Wait for things to clean up and close the connectiontime.sleep(2)
conn.disconnect()

In the example code above, we used a single file to demonstrate both the producer and consumer, but you can refactor this code to separate out these components.

Let us try and do the same for the Topic Architecture. I am only going to add comments for code which is unique to the topic architecture:

import time 
import sys
import stomp
class Listener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
hosts = [('localhost', 61616)]
conn = stomp.Connection(host_and_ports=hosts)
conn.set_listener('', Listener())
conn.start()
conn.connect('admin', 'admin', wait=True)
# Register a subscriber with ActiveMQ. This tells ActiveMQ to send
# all messages received on the topic 'topic-1' to this listener
conn.subscribe(destination='/topic/topic-1', ack='auto') # Act as a message publisher and send a message the queue queue-1conn.send(body=' '.join(sys.argv[1:]), destination='/topic/topic-1')
time.sleep(2)
conn.disconnect()

Wasn’t that easy. The stomp.py library makes it trivial to spin up ActiveMQ clients. You can create as many queues and topics as you would like on the ActiveMQ server and communicate with them through various clients.

So, if you are thinking about breaking your monolith application into a microservice architecture, do consider a message broker like ActiveMQ in your architecture design for achieving better performance, reliability, scalability and simplified decoupling.

Security Analyst aka Triager @HackerOne. Curious. Minimalist.