The Kafka Client's SSL Config, and the Absence-Gating Trick
The Kafka client needs four SSL settings, but only when security.protocol is set. Leave it unset and it stays plain. One 'if' is the whole gap between prod and laptop.
To connect to Kafka, an app uses a Kafka client library (the same core library sits under the Go and Python clients). To turn on encryption (mutual TLS), it needs four settings:
security.protocol=SSL
ssl.ca.location=<root_ca.pem> # verify the Kafka server
ssl.certificate.location=<certificate.pem> # my identity
ssl.key.location=<private_key.pem> # proves the identity is mine(There’s a fifth, ssl.key.password, only if the private key is itself password-protected.)
The neat part is how this keeps local development simple. Call it absence-gating: add the four encryption settings only when security.protocol is set. Leave it unset and the library quietly stays plain and ignores the encryption settings, so the local path keeps working with no extra config at all.
One “if” is the entire difference between “encrypted in production” and “plain on my laptop”. No separate build, and no error just because the certificate settings are not there.
In this migration: the producer code builds the client config and appends the four encryption settings behind a single “if security protocol is set” check, so the local config file (no encryption, Kafka switched off) is left completely alone. That satisfied the “local dev starts with no SSL config” requirement. One language-specific gotcha worth noting: in Go the config object is a named type, not a plain map, so a test helper has to use that exact type or the code won’t compile.
Let absence be the config. “No protocol set” is a valid, working state, not a missing one.
References
Related: the four settings point at the three certificate files, and only apply to the encrypted 9093 door. Back to the overview: two SSLs behind one migration ticket.