Welcome back to my blog! Finally it’s time, for me to share you about this piece of amazing technology. For new timers here, you can look at all my previous posts here. Without further ado, let’s start digging about Kafka!

hand heart
Couples’ heart, on hand

Look at the picture, do you think that is a picture of heart or not? Real heart, maybe no, but at least it is heart shaped. That is also what Kafka is to microservices. It is not the real heart of each microservices, but at least, it creates heart shaped hands for each microservices.

It doesn’t make sense, does it? LOL

I think, first, I need to tell you what it really is. Basically, Kafka is a distributed messaging platform. The message here means a way to communicate between services (we probably have many of them) in a simple asynchronous way. Kafka have topics that can be written or read by everyone connected to it (topic, just think it like an article topic).

There are two actor categories connected to Kafka based on its activity to the topic.

  1. Publisher. This is the one who write the message to the topic.
  2. Consumer. The one who read the message from the topic.
kafka
some other illustration

Take a look on the picture above. The girl in the left, is basically the publisher who write (in this case, speak). And the boys in the right are the consumer. By the way, the speaker is Kafka. LOL.

For your information, neither the number of publisher nor the consumer is limited to only one. The picture above shown one producer, and two consumers. To Kafka, everyone can publish, and everyone can consume.

Isn’t it just a copycat for other messaging platform like RabbitMQ and the like?

By the design, Kafka is really different with the others. It is constructed in a logging platform fashion, as opposed with the other publish-subscribe platform like RabbitMQ and other MQs.

topic
Kafka in a nutshell

Look at the picture. This is the basic Kafka topic looks like. It stores everything you write into the log (with a default retention time for each message, which is 7 days). And you can consume all the message from the earliest available in the topic. While the old Message Queue system like RabbitMQ will delete its message after someone consume it.

Now, what it can provide as the heart of microservices?

Let’s take a look at the image taken from Kafka official site.

kafka ecosystem
Lots of things here

Basically, Kafka becomes the centre of all things here. Some services or apps produces messages to it and some consumes. It is actually decoupling each apps from each other, increasing the testability of each apps. Means you don’t need to call another app when testing one app only.

The code will be simplified since you don’t have to think about how other app behaves. Good simple code means high maintainability. Good clarity means easier bug fixing. 

A simple use case

Let’s say you want to blast emails for a new blog post to every single users registered in your website. Which one do you want?

  1. Send it synchronously using the resource of your app to your 100k subscribers.
  2. Send a message to Kafka, and create a consumer who gets the email info of your 100k subscribers asynchronously and send the emails.

The second one is definitely better, since there is a risk, that your app can spike its resource, even worse go down. While the second one will not disturb your app at all.

Do it the Kafka way!

Please share it if you like the post!


Leave a Reply

Your email address will not be published. Required fields are marked *