kafka connect Creation of database history topic failed, please create the topic manually / TimeoutException 에러

kafka-connect_1    | [2019-03-18 08:23:37,859] ERROR WorkerSourceTask{id=inventory-connector-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask)

kafka-connect_1    | org.apache.kafka.connect.errors.ConnectException: Creation of database history topic failed, please create the topic manually

kafka-connect_1    | at io.debezium.relational.history.KafkaDatabaseHistory.initializeStorage(KafkaDatabaseHistory.java:348)

kafka-connect_1    | at io.debezium.connector.mysql.MySqlSchema.intializeHistoryStorage(MySqlSchema.java:266)

kafka-connect_1    | at io.debezium.connector.mysql.MySqlTaskContext.initializeHistoryStorage(MySqlTaskContext.java:196)

kafka-connect_1    | at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:137)

kafka-connect_1    | at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:47)

kafka-connect_1    | at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:198)

kafka-connect_1    | at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)

kafka-connect_1    | at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)

kafka-connect_1    | at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

kafka-connect_1    | at java.util.concurrent.FutureTask.run(FutureTask.java:266)

kafka-connect_1    | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

kafka-connect_1    | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

kafka-connect_1    | at java.lang.Thread.run(Thread.java:748)

kafka-connect_1    | Caused by: java.util.concurrent.TimeoutException

kafka-connect_1    | at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:108)

kafka-connect_1    | at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:274)

kafka-connect_1    | at io.debezium.relational.history.KafkaDatabaseHistory.getKafkaBrokerConfig(KafkaDatabaseHistory.java:353)

kafka-connect_1    | at io.debezium.relational.history.KafkaDatabaseHistory.initializeStorage(KafkaDatabaseHistory.java:337)

kafka-connect_1    | ... 12 more

원인은 kafka connect 포트 이슈이다.

docker compose를 사용할 때. 

kafka를 9092로 띄우면 docker 이슈가 발생한다. 따라서 저 에러가 나면 해당 포트(9092)에 데몬 이슈가 있으니 아래 링크를 참고해서 kafka:29092로 변경하고 관련해서 kafka_connect 컴포넌트에서 kafka:29092에 연결하도록 변경한다. 

https://rmoff.net/2018/08/02/kafka-listeners-explained/