What I learned about SSL Certificates when building a Secured Kafka Cluster

To build a multi-protocol Apache Kafka Clusters to allow for SSL Client Authentication with PLAINTEXT for inter broker communication, I needed to generate both broker and client SSL certificates. There were many interesting things I learned in this process and wanted to share them.

An upcoming article will be using these certificates for setting up my Secured Apache Kafka Cluster.


I build certificates to explore the security available in various systems; in this particular scenario, it is to explore a mixed protocol Kafka Cluster. Please seek advice from your security teams when creating certificates.


If you only need to create certificates as part of an enterprise setup, consider actively supported projects such as Confluent’s Ansible Playbooks.

There are excellent articles on SSL Certificates, including Confluent’s documentation and an excellent tutorial. The documentation and tutorial focuses on SASL client-side authentication (SASL_SSL protocol) and I uncovered some unique challenges with client-side SSL authentication.

My experience in certificate generation is based on using OpenSSL. There are other means of generating certificates, so if another means is used for certificate creation, some of these highlights may not apply to you.

My full scripts are in the repository Kafka SSL Cluster.


Your certificate subject must start with /, and should contain at minimum a CN element, as in /CN=${hostname}.

For broker certificates, have the CN matches the hostname, and then leverage Subject Alternate Names for ensuring the certificate can be used with and without domain name successfully.

There are no wildcard options with the standard Apache Kafka Authorizer, kafka.security.authorizer.AclAuthorizer. So if you a complete subject of common name (CN), organization unit (OU), organization (OU), locality (L), state (S), and country (C) the full name is what will be used as the User in ACLs.


Not This: User:CN=broker-1

JKS key and keystore passwords should be the same

When creating Java key-stores (.jks files) for your JVMs, make sure the keystore password is the same as the key password. This is due to limitations of the SunX509 KeyManagerFactory. Java Secure Socket Extension (JSSE) Reference Guide.

All keys in the KeyStore must be protected by the same password

I have found a few other documented cases of this with other systems, the most recent is with BitBucket.


Be sure the certificates have the proper x509 extensions.

Basic Contraints extension for CA certificate


If you setup SSL certificates for client authentication, the CA certificate needs the following x509 extension indicating it is indeed a CA certificate. Without this extension, it will not authenticate client certificates.

For additional information, see x509v3 config.

Subject Alternate Name Extension for Brokers

subjectAltName=DNS:${i}, DNS:${i}.${DOMAIN}

When it comes to SSL Certificates for encryption, it is important that it can have extensions to handle being used with and without domain names.

Extended Key Usage Extension (for all)


The public key a certificate had to be noted that it can be used for server and client authentication. For my cluster, I configured all certificates so they can be used for both server and client authentication. I added both to all certificates.

extensions are not copied from request to certificate

By default, extensions added to a request is not propagated to the certificate. In order for extensions to be copied, you would need copy_extensions = copy added to your openssl.cfg. I was unsuccessful in getting this to work in MacOS so I updated my scripts to explicitly add extensions when certificate requests are signed.

It is a good reminder if you create certificate requests for another group to sign, be sure to indicate extensions you need in your certificates.

Check Status after every call to OpenSSL

If you write scripts, make sure you check return status codes after each command and fail on error. I have spent many hours troubleshooting script errors only to find out that something failed much earlier in the process.

After each command, add the following. Scripts are finicky, especially with loops. So double-check the error handling works as well. This is the type of check I do after every openssl and every other command.

[ $? -eq 1 ] && echo "failure" && exit

Intermediate Certificates

Most enterprises do not sign machine or applications certificates with their top-level CA certificate; they use an intermediate certificate. Using an intermediate certificate requires a certificate chain when creating the trust-store. Personally, I use intermediate certificates, so it is a reminder on how to properly chain them when creating the pkcs12 file.

The openSSL configuration file

The configuration file of OpenSSL, openssl.cnf can vary greatly between Unix systems and even between Linux distributions.

For every command of openssl that uses openssl.cfg provide a configuration file; otherwise, a default file will be used and it varies greatly between OS distributions. To avoid confusing on how OpenSSL executes your request, I explicitly provide the openssl.cfg to use. I keep this file with my scripts. Furthermore, if you are scripting your certificate process, leverage inline files to provide a custom configuration file to each specific command execution; it is then self-documenting too.

An example of inlining a file concatenation.

-config <(cat ./openssl.cnf <(printf "\n[ext]\nbasicConstraints=CA:TRUE,pathlen:1"))

update 5/2021: it has come to my attention that due to intermediate certificate pathlen of 1 is needed. I am not sure why this didn’t cause me issues in my initial implementation; additional investigation is needed.

Check your OpenSSL documentation

OpenSSL varies between Unix variants and even between Linux distributions; please read the manual pages for your specific version and release of openssl. Also, there could be a bug or limitation with the instance of openssl you are using. For example, I was not successful leveraging copy_extensions on MacOS.


What I have documented here were the major pieces I had to figure out when building certificates for client authentication for my Kafka Cluster. It wouldn’t surprise me if there are other things I could uncover, especially limiting extensions only between the broker certificates and client certificates.

If you want a glimpse to the scripts that I will use in part 2 of this blog where I configure my Secured (Mix-Protocol) Kafka Cluster, you find them all at Kafka SSL Cluster.