Create an ActiveMQ client using Python
In modern organizations, applications are broken down into services that can be independently developed, deployed and maintained. These services run as autonomous processes and communicate with each other through APIs. The service owner can choose a programming language that is most effective to solve the specific problem at hand, irrespective of what language other services are built in.
This also makes scaling easier, because organizations can choose to scale the components that are being used to their full capacity, instead of scaling the whole application. Such an architecture is called the microservice architecture.
Microservice Architecture
The real power of a microservice architecture comes from the relationships among its services rather than from the individual services themselves. So, it is important these services communicate efficiently for the success of a microservice architecture.
That said, not all services need to run synchronously. For a customer-facing web application, the web server only needs to talk to a handful of services, if at all, before sending a response back to the user. Majority of the services that make up a microservice architecture can bear some latency without compromising the overall performance of the application. So, these services can be invoked asynchronously by the calling service.
Message brokers are a key component of a microservice architecture, that allows for such asynchronous communication between services.
What Are Message Brokers?
Message Brokers provide efficient communication and coordination between services in a microservice architecture. A message broker provides a lean buffer which stores messages that are supposed to passed around to various services. These messages can be requests, replies, errors, logs, or just any piece of information. The sender of a message is known as a producer or publisher, while the receiver is known as a consumer or subscriber.
The message broker can implement the message buffer in two ways:
- Queues, or
- Topics
Both are places where a producer or publisher sends messages to for further relay. The difference lies in who receives these messages.
The Queue Architecture:
Queues can be used by multiple clients to send and receive messages, however each message is processed only once by sending it to exactly one client. In the queue architecture, the client sending the message is known as a ‘Producer’ and the client receiving the message is known as a ‘Consumer’.
The Topic Architecture:
In the Topic architecture, multiple clients can receive the same message. The clients who wish to receive the messages, inform the message broker, that they would like to ‘subscribe’ to a given ‘topic’ configured on the broker. Whenever a client sends a message on the topic, the message broker will broadcast the message to all clients who have subscribed to this topic.
A client sending the messages in the topic architecture is known as a ‘Publisher’ and the receiving client is known as a ‘Subscriber’.
Below are some of the most popular open source message brokers:
Let’s take a closer look at ActiveMQ in this article.
ActiveMQ
ActiveMQ message broker is an implementation of the Java Messaging Service (JMS). It also supports a variety of Cross Language Clients and Protocols from Java, C, C++, C#, Ruby, Perl, Python and PHP. This cross-language compatibility is the key feature of ActiveMQ that makes it an ideal message broker in a microservice architecture.
In this article, we will see how easy it is to implement a Python client using ActiveMQ message broker.
If you would like to code along, ensure that you have Python installed on your system. You can download ActiveMQ and install it on your native system or an Ubuntu virtual machine, using the instructions available here.
Ensure that your ActiveMQ installation is up and running and that you can access its web console through https://localhost:8161/.
Create Queue and Topic
Using the web console, create a test Queue and a test Topic for your clients:
- Click on the ‘Queues’ link from the top navbar
- On the Queues page, enter name of the queue you want to create in the textbox labeled ‘Queue Name’
Follow the same steps for Topic by clicking on the ‘Topics’ link on the navbar:
Now that your test queue and topic is created, you can go ahead to create the clients for interacting with them.
Create Python Clients
ActiveMQ provides support for Python clients using the STOMP protocol. So, your Python clients need to use the stomp.py library. To install this library, run the following command
pip install stomp.py We will create two Python clients, one to demonstrate the Queue architecture and another for the Topic architecture.
Let’s start with the queue architecture first. The below code implements both, a producer and a consumer, for a queue. I have included comments inline for explaining what code does.
import time
import sys
import stomp# Create a Listener class inheriting the stomp.ConnectionListener class
# stomp.ConnectionListener class definition can be found here:
# https://github.com/jasonrbriggs/stomp.py/blob/2435108cfc3eb4bd6477653b071e85acd6a2f211/stomp/listener.pyclass Listener(stomp.ConnectionListener):
# Override the methods on_error and on_message provides by the
# parent class
def on_error(self, headers, message):
print('received an error "%s"' % message)# Print out the message received def on_message(self, headers, message):
print('received a message "%s"' % message)# Decare hosts as an array of tuples containing the ActiveMQ server # IP address or hostname and the port number
hosts = [('localhost', 61616)]# Create a connection object by passing the hosts as an argument
conn = stomp.Connection(host_and_ports=hosts)# Tell the connection object to listen for messages using the
# Listener class we created above
conn.set_listener('', Listener()) # Initiate the connection with the credentials of the ActiveMQ server
conn.start()
conn.connect('admin', 'admin', wait=True)# Register a consumer with ActiveMQ. This tells ActiveMQ to send all # messages received on the queue 'queue-1' to this listenerconn.subscribe(destination='/queue/queue-1', id=1, ack='auto')# Act as a message producer and send a message the queue queue-1
# The actual message to be sent is picked up from the command line arguments# Say, if you want to send the message "Hello Queue", you will run
# the code aspython queue.py "Hello Queue"conn.send(body=' '.join(sys.argv[1:]), destination='/queue/queue-1')# When this message is received by the listener, it will be handled
# by the on_message method we defined above.# Wait for things to clean up and close the connectiontime.sleep(2)
conn.disconnect()
In the example code above, we used a single file to demonstrate both the producer and consumer, but you can refactor this code to separate out these components.
Let us try and do the same for the Topic Architecture. I am only going to add comments for code which is unique to the topic architecture:
import time
import sys
import stompclass Listener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message) def on_message(self, headers, message):
print('received a message "%s"' % message)hosts = [('localhost', 61616)]
conn = stomp.Connection(host_and_ports=hosts)
conn.set_listener('', Listener())
conn.start()
conn.connect('admin', 'admin', wait=True)# Register a subscriber with ActiveMQ. This tells ActiveMQ to send
# all messages received on the topic 'topic-1' to this listenerconn.subscribe(destination='/topic/topic-1', ack='auto') # Act as a message publisher and send a message the queue queue-1conn.send(body=' '.join(sys.argv[1:]), destination='/topic/topic-1')
time.sleep(2)
conn.disconnect()
Wasn’t that easy. The stomp.py library makes it trivial to spin up ActiveMQ clients. You can create as many queues and topics as you would like on the ActiveMQ server and communicate with them through various clients.
So, if you are thinking about breaking your monolith application into a microservice architecture, do consider a message broker like ActiveMQ in your architecture design for achieving better performance, reliability, scalability and simplified decoupling.
