I have developed one RD Kafka Client using LIBRDKAFKA C Library. LIBRDKAFKA C library has exposed set_config APIs to set configuration values for Consumer and Producer Handles. There are around hundreds of configs available to be set, it became very tedious to set_congig for each value. Importantly its not efficient way of setting configs in code itself as here we don't have flexibility to tune configurations values according to different platform as it require re-built of RD Kafka Clients.
11 Answer
The librdkafka configuration interface is designed with this in mind, providing a simple string key, value API for 95% of the configuration properties (the remaining are C callbacks, etc, which need proper types).
Here's an example on how to implement a configuration file reader (with key=value\n syntax) that allows any standard librdkafka configuration to be set:
rd_kafka_conf_t *conf = rd_kafka_conf_new(); FILE *fp; char buf[512]; int line = 0; fp = fopen(conf_path, "r"); if (!fp) ;// add error checking.. while (fgets(buf, sizeof(buf)-1, fp)) { char *t; char *b = buf; rd_kafka_conf_res_t res; char *name, *val; char errstr[512]; line++; if ((t = strchr(b, '\n'))) *t = '\0'; if (*b == '#' || !*b) continue; if (!(t = strchr(b, '='))) fprintf(stderr, "%s:%i: expected name=value format\n", conf_path, line); name = b; *t = '\0'; val = t+1; res = rd_kafka_conf_set(conf, name, val, errstr, sizeof(errstr)); if (res != RD_KAFKA_CONF_OK) fprintf(stderr, "%s:%i: %s\n", conf_path, line, errstr); } fclose(fp); ... // Create client rd_kafka_t *rk; rk = rd_kafka_new(... , conf, ..); ncG1vNJzZmirpJawrLvVnqmfpJ%2Bse6S7zGiorp2jqbawutJoa29qY21%2FeH6OoaawZaSkerOxwJ1knKeem7aowdGaq6KnnmLDorjUnqpmnqKkum6tjJ%2BgpZ1dnrtuvsNmopqem5Z6pLjInqWt