This Kafka Summit, like the ones before it, was fantastic. Tons of content to choose from and an excellent virtual venue to use to consume that content. I highly recommend watching many of the sessions on demand. I am working through more content myself, but best to share what I have uncovered so far before I wait until I see everything I want to see.
A link to demoing the application leveraging Synchronous Commands Over Apache Kafka. The demo application is using Subscribe approach as mentioned in the presentation. 1 NGINX Load Balancer and 2 Request Handlers. The Vue application generates the QUIZ_ID (which is used as the key) so NGINX will send requests to both request handlers, and 1 will say “me” and handle the request and one will say “not me” and return a 406 to the load-balancer to indicate that it is not the instance that has the partition that the given QUIZ_ID will end up on.
You have built an event-driven system leveraging Apache Kafka. Now you face the challenge of integrating traditional synchronous request-response capabilities, such as user interaction, through an HTTP web service.
There are various techniques, each with advantages and disadvantages. This talk discusses multiple options on how to do a request-response over Kafka — showcasing producers and consumers using single and multiple topics, and more advanced considerations using the interactive queries of ksqlDB and Kafka Streams.
Introduction Goal: build a multi-protocol Apache Kafka Clusters for SSL Client Authentication for all clients while leveraging PLAINTEXT for inter broker communication.
There are many tutorials and articles on setting up Apache Kafka Clusters with different security options. However, I have not seen a mix-protocol cluster with SSL for encryption and client authentication and PLAINTEXT for broker communication; so I decided to build one.
To build a multi-protocol Apache Kafka Clusters to allow for SSL Client Authentication with PLAINTEXT for inter broker communication, I needed to generate both broker and client SSL certificates. There were many interesting things I learned in this process and wanted to share them.