RabbitMQ way of using parallel work?


#1

Hello, first thank you very much for providing such excellent tutorial and clean architecture design. I am learning the micro services with message broker and would be more than interested in how to use message broker to improve the micro services performance in real world cases.

In your Episode-3 where you introduced the publisher and subscribe with command handler way for delivering message between micro services, I research the code and find that it seems to use the “TOPIC” type of messaging with rabbitmq which would be a “single task message queue”, I was wondering if you could better introduce why you choose “topic” message type in this project, and I am more curious if it’s possible to do parallel work/services with rabbitmq.

For example, you may have three instances of Discounts Services running at the same time, meanwhile there would be 10k message coming at busy traffic time, what I wish to happen is to let rabbitmq queue distributes the 10k messages to your three instances of services continuously so we could decrease the workload of one service instance to its 1/3 size.

I am curious if such scenario could be implemented using rabbitmq with .net web api micro services in your projects? In addition, for the docker section, could we setup and start multiply instances of services at same time? So far I cannot think about any ideal solution by using the “topic”/“fanout”/“direct” queue of rabbitmq. However, I think if it’s possible to implement such scenarios will be another good benefit of using micro services architecture.

Sorry about my above complicated information, I really appreciate your tutorial and guide which brought me into a fantastic new world!


#2

I’m also curious about this, it seems like if I have multiple instances of a service subscribed to an event, all of them will pick it up and try to process it? Wouldn’t that be a problem?


#3

If I am not mistaken, RabbitMQ being an implementation of the AMQP protocol requires each ‘consumer’ of the queue send an acknowledge message back to the queue when they pickup a message. This confirmation indicates to the queue that this message has been picked up and handled, and therefore will be removed from the queue. This means that even if you have multiple instances, only one will get the chance to deal a particular message.


#4

I see, does that mean it’s a queue, which allows multiple subscribers to pick unique messages like kafka consumer group?


#5

I am unfortunately not familiar with Kafka but I am pretty sure that in RabbitMq, a message can be consumed only once, and after it is consumed, it is removed from the queue and isn’t accessible anymore.