AWS msk kafka tutorial | Access IAM authentication

Published: 01 January 1970
on channel: MaxcoTec Learning

3,357

AWS msk kafka tutorial. In this video, we are going to demostrate how to access an IAM auth enabled MSK kafka cluster.

Get the scripts here: https://github.com/maxcotec/aws-IAM-a...

#maxcoteclearning #aws #awsmsk #awsiam #kafka

timeline;
00:00 intro
00:25 outline
00:48 opening
00:54 setup kafka cluster
01:46 Create IAM policies and Users
02:43 Create machine where to test access kafka
04:12 Prepare machine
06:08 export IAM user
06:56 create kafka topics
08:42 spin kafka producer
09:41 spin kafka consumer
11:21 test end-to-end
11:39 outro

Common Issues/Errors:
1. @07:57 -
Error: Uncaught Exception in thread 'kafka-admin-client-thread java.lang.OutOfMemoryError: Javaheap space'
Reason: Caused because we didn't provided client.properties files with kafka-topics.sh script. provide the following option when running kafka-topics.sh script; `--command-config ./kafka_2.12-3.4.1/client.properties`

2. @09:17
Error: Bootstrap-broker (id: -1 rack: null) disconnected
Reason: Again caused due to not providing client-properties file. provide the following option when running kafka-console-producer.sh script; `--producer.config kafka_2.12-3.4.1/client.properties`

3. @10:48
Error: Not authorised to access group
Reason: because we have restricted consumer IAM user to access group named `reader-group`, thats why we have to specifically provide consumer group name with kafka-console-consumer.sh script. `--consumer.config kafka_2.12-3.4.1/client.properties --group reader-group`

Watch video AWS msk kafka tutorial | Access IAM authentication online, duration hours minute second in high quality that is uploaded to the channel MaxcoTec Learning 01 January 1970. Share the link to the video on social media so that your subscribers and friends will also watch this video. This video clip has been viewed 3,357 times and liked it 30 visitors.

11,720

318