Posted in Devops, Information Technology

Kafka Cheat Sheet

CLI Commands for Kafka Topics

List existing topics

/usr/bin/kafka-topics --zookeeper <zookeeper host:port> --list

Create a new topic

/usr/bin/kafka-topics --create --zookeeper <zookeeper host:port>/kafka --replication-factor 2 --partitions 4 --topic <name>

Try to read from a topic

/usr/bin/kafka-console-consumer --zookeeper  <zookeeper host:port> --topic <name>

Try to write to a topic on one or more brokers

echo "Test data" | /usr/bin/kafka-console-producer --broker-list <broker1 host:port>,<broker2 host:port> --topic <name>

Checking Kafka Consumers

Show consumer group configuration

/usr/bin/kafka-consumer-groups --zookeeper <zookeeper host:port> --describe --group <group name>

To read messages read/written including lag per consumer in a consumer group

/usr/bin/kafka-consumer-offset-checker --group <group name> --topic <name> --zookeeper <zookeeper host:port>

Syslog to Kafka

Monitoring Tools

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s