Skip to content

build-a-streaming-pipeline's ksqldb and schema-registry meet 2 starter issue. #254

Open
@Michael2008S

Description

@Michael2008S

Hi confluentors,

I meet 2 starter issues when using docker-compose up -d to run the build-a-streaming-pipeline demo in the current master branch.

image

ksqldb  | [2023-01-30 01:54:56,634] INFO KsqlConfig values: 
ksqldb  |       ksql.access.validator.enable = auto
ksqldb  |       ksql.authorization.cache.expiry.time.secs = 30
ksqldb  |       ksql.authorization.cache.max.entries = 10000
ksqldb  |       ksql.cast.strings.preserve.nulls = true
ksqldb  |       ksql.connect.url = http://kafka-connect-01:8083
ksqldb  |       ksql.connect.worker.config = 
ksqldb  |       ksql.create.or.replace.enabled = true
ksqldb  |       ksql.error.classifier.regex = 
ksqldb  |       ksql.extension.dir = ext
ksqldb  |       ksql.hidden.topics = [^_.*]
ksqldb  |       ksql.insert.into.values.enabled = true
ksqldb  |       ksql.internal.topic.min.insync.replicas = 1
ksqldb  |       ksql.internal.topic.replicas = 1
ksqldb  |       ksql.lambdas.enabled = true
ksqldb  |       ksql.metastore.backup.location = 
ksqldb  |       ksql.metrics.extension = null
ksqldb  |       ksql.metrics.tags.custom = 
ksqldb  |       ksql.nested.error.set.null = true
ksqldb  |       ksql.output.topic.name.prefix = 
ksqldb  |       ksql.persistence.default.format.key = KAFKA
ksqldb  |       ksql.persistence.default.format.value = null
ksqldb  |       ksql.persistence.wrap.single.values = null
ksqldb  |       ksql.persistent.prefix = query_
ksqldb  |       ksql.properties.overrides.denylist = []
ksqldb  |       ksql.pull.queries.enable = true
ksqldb  |       ksql.query.error.max.queue.size = 10
ksqldb  |       ksql.query.persistent.active.limit = 2147483647
ksqldb  |       ksql.query.persistent.max.bytes.buffering.total = -1
ksqldb  |       ksql.query.pull.enable.standby.reads = false
ksqldb  |       ksql.query.pull.interpreter.enabled = true
ksqldb  |       ksql.query.pull.max.allowed.offset.lag = 9223372036854775807
ksqldb  |       ksql.query.pull.max.concurrent.requests = 2147483647
ksqldb  |       ksql.query.pull.max.hourly.bandwidth.megabytes = 2147483647
ksqldb  |       ksql.query.pull.max.qps = 2147483647
ksqldb  |       ksql.query.pull.metrics.enabled = true
ksqldb  |       ksql.query.pull.table.scan.enabled = false
ksqldb  |       ksql.query.pull.thread.pool.size = 100
ksqldb  |       ksql.query.push.scalable.enabled = false
ksqldb  |       ksql.query.push.scalable.interpreter.enabled = true
ksqldb  |       ksql.query.retry.backoff.initial.ms = 15000
ksqldb  |       ksql.query.retry.backoff.max.ms = 900000
ksqldb  |       ksql.query.status.running.threshold.seconds = 300
ksqldb  |       ksql.query.transient.max.bytes.buffering.total = -1
ksqldb  |       ksql.queryanonymizer.cluster_namespace = null
ksqldb  |       ksql.queryanonymizer.logs_enabled = true
ksqldb  |       ksql.readonly.topics = [_confluent.*, __confluent.*, _schemas, __consumer_offsets, __transaction_state, connect-configs, connect-offsets, connect-status, connect-statuses]
ksqldb  |       ksql.schema.registry.url = http://schema-registry:8081
ksqldb  |       ksql.security.extension.class = null
ksqldb  |       ksql.service.id = confluent_rmoff_01
ksqldb  |       ksql.sink.window.change.log.additional.retention = 1000000
ksqldb  |       ksql.streams.shutdown.timeout.ms = 300000
ksqldb  |       ksql.suppress.buffer.size.bytes = -1
ksqldb  |       ksql.suppress.enabled = false
ksqldb  |       ksql.timestamp.throw.on.invalid = false
ksqldb  |       ksql.transient.prefix = transient_
ksqldb  |       ksql.udf.collect.metrics = false
ksqldb  |       ksql.udf.enable.security.manager = true
ksqldb  |       ksql.udfs.enabled = true
ksqldb  |       ksql.variable.substitution.enable = true
ksqldb  |       metric.reporters = []
ksqldb  |       ssl.cipher.suites = null
ksqldb  |       ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
ksqldb  |       ssl.endpoint.identification.algorithm = https
ksqldb  |       ssl.engine.factory.class = null
ksqldb  |       ssl.key.password = null
ksqldb  |       ssl.keymanager.algorithm = SunX509
ksqldb  |       ssl.keystore.certificate.chain = null
ksqldb  |       ssl.keystore.key = null
ksqldb  |       ssl.keystore.location = null
ksqldb  |       ssl.keystore.password = null
ksqldb  |       ssl.keystore.type = JKS
ksqldb  |       ssl.protocol = TLSv1.3
ksqldb  |       ssl.provider = null
ksqldb  |       ssl.secure.random.implementation = null
ksqldb  |       ssl.trustmanager.algorithm = PKIX
ksqldb  |       ssl.truststore.certificates = null
ksqldb  |       ssl.truststore.location = null
ksqldb  |       ssl.truststore.password = null
ksqldb  |       ssl.truststore.type = JKS
ksqldb  |  (io.confluent.ksql.util.KsqlConfig:376)
ksqldb  | log4j:WARN No appenders could be found for logger (org.apache.kafka.connect.json.JsonConverterConfig).
ksqldb  | log4j:WARN Please initialize the log4j system properly.
ksqldb  | log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
ksqldb  | [2023-01-30 01:55:27,987] ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain:69)
ksqldb  | java.lang.RuntimeException: Failed to get Kafka cluster information
ksqldb  |       at io.confluent.ksql.services.KafkaClusterUtil.getKafkaClusterId(KafkaClusterUtil.java:107)
ksqldb  |       at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:638)
ksqldb  |       at io.confluent.ksql.rest.server.KsqlServerMain.createExecutable(KsqlServerMain.java:155)
ksqldb  |       at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:62)
ksqldb  | Caused by: java.util.concurrent.TimeoutException
ksqldb  |       at java.base/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1886)
ksqldb  |       at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2021)
ksqldb  |       at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:180)
ksqldb  |       at io.confluent.ksql.services.KafkaClusterUtil.getKafkaClusterId(KafkaClusterUtil.java:105)
ksqldb  |       ... 3 more
schema-registry  | [2023-01-30 01:56:37,469] INFO HV000001: Hibernate Validator 6.1.7.Final (org.hibernate.validator.internal.util.Version)
schema-registry  | [2023-01-30 01:56:38,594] INFO Started o.e.j.s.ServletContextHandler@7fc7c4a{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler)
schema-registry  | [2023-01-30 01:56:38,611] INFO Started o.e.j.s.ServletContextHandler@440eaa07{/ws,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler)
schema-registry  | [2023-01-30 01:56:38,714] INFO Started NetworkTrafficServerConnector@2ce6c6ec{HTTP/1.1, (http/1.1)}{0.0.0.0:8081} (org.eclipse.jetty.server.AbstractConnector)
schema-registry  | [2023-01-30 01:56:38,714] INFO Started @57225ms (org.eclipse.jetty.server.Server)
schema-registry  | [2023-01-30 01:56:38,714] INFO Server started, listening for requests... (io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain)
schema-registry  | [2023-01-30 01:57:07,637] INFO Registering new schema: subject ratings-value, version null, id null, type null (io.confluent.kafka.schemaregistry.rest.resources.SubjectVersionsResource)
schema-registry  | [2023-01-30 01:57:07,869] INFO Wait to catch up until the offset at 1 (io.confluent.kafka.schemaregistry.storage.KafkaStore)
schema-registry  | [2023-01-30 01:57:07,870] INFO Reached offset at 1 (io.confluent.kafka.schemaregistry.storage.KafkaStore)
schema-registry  | [2023-01-30 01:57:07,898] INFO Wait to catch up until the offset at 2 (io.confluent.kafka.schemaregistry.storage.KafkaStore)
schema-registry  | [2023-01-30 01:57:07,927] INFO Reached offset at 2 (io.confluent.kafka.schemaregistry.storage.KafkaStore)
schema-registry  | [2023-01-30 01:57:08,068] INFO 172.18.0.9 - - [30/Jan/2023:01:57:07 +0000] "POST /subjects/ratings-value/versions HTTP/1.1" 200 8  623 (io.confluent.rest-utils.requests)
schema-registry  | [2023-01-30 02:02:35,266] INFO 172.18.0.1 - - [30/Jan/2023:02:02:35 +0000] "GET / HTTP/1.1" 200 2  18 (io.confluent.rest-utils.requests)
schema-registry  | [2023-01-30 02:02:35,377] ERROR Request Failed with exception  (io.confluent.rest.exceptions.DebuggableExceptionMapper)
schema-registry  | javax.ws.rs.NotFoundException: HTTP 404 Not Found
schema-registry  |      at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:252)
schema-registry  |      at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
schema-registry  |      at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
schema-registry  |      at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
schema-registry  |      at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
schema-registry  |      at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
schema-registry  |      at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
schema-registry  |      at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234)
schema-registry  |      at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
schema-registry  |      at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
schema-registry  |      at org.glassfish.jersey.servlet.ServletContainer.serviceImpl(ServletContainer.java:386)
schema-registry  |      at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:561)
schema-registry  |      at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:502)
schema-registry  |      at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:439)
schema-registry  |      at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
schema-registry  |      at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
schema-registry  |      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
schema-registry  |      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
schema-registry  |      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
schema-registry  |      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
schema-registry  |      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435)
schema-registry  |      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
schema-registry  |      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
schema-registry  |      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
schema-registry  |      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
schema-registry  |      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350)
schema-registry  |      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
schema-registry  |      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
schema-registry  |      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
schema-registry  |      at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:179)
schema-registry  |      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
schema-registry  |      at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:735)
schema-registry  |      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
schema-registry  |      at org.eclipse.jetty.server.Server.handle(Server.java:516)
schema-registry  |      at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)
schema-registry  |      at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)
schema-registry  |      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)
schema-registry  |      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
schema-registry  |      at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
schema-registry  |      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
schema-registry  |      at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
schema-registry  |      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
schema-registry  |      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
schema-registry  |      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
schema-registry  |      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
schema-registry  |      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882)
schema-registry  |      at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036)
schema-registry  |      at java.base/java.lang.Thread.run(Thread.java:829)
schema-registry  | [2023-01-30 02:02:35,385] INFO 172.18.0.1 - - [30/Jan/2023:02:02:35 +0000] "GET /favicon.ico HTTP/1.1" 404 49  16 (io.confluent.rest-utils.requests)
schema-registry  | [2023-01-31 02:14:40,493] INFO Stopped NetworkTrafficServerConnector@2ce6c6ec{HTTP/1.1, (http/1.1)}{0.0.0.0:8081} (org.eclipse.jetty.server.AbstractConnector)
schema-registry  | [2023-01-31 02:14:40,494] INFO node0 Stopped scavenging (org.eclipse.jetty.server.session)
schema-registry  | [2023-01-31 02:14:40,496] INFO Stopped o.e.j.s.ServletContextHandler@440eaa07{/ws,null,STOPPED} (org.eclipse.jetty.server.handler.ContextHandler)
schema-registry  | [2023-01-31 02:14:40,508] INFO Stopped o.e.j.s.ServletContextHandler@7fc7c4a{/,null,STOPPED} (org.eclipse.jetty.server.handler.ContextHandler)
schema-registry  | [2023-01-31 02:14:40,509] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,510] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,510] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,510] INFO Shutting down schema registry (io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry)
schema-registry  | [2023-01-31 02:14:40,511] INFO [kafka-store-reader-thread-_schemas]: Shutting down (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread)
schema-registry  | [2023-01-31 02:14:40,512] INFO [kafka-store-reader-thread-_schemas]: Stopped (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread)
schema-registry  | [2023-01-31 02:14:40,516] INFO [kafka-store-reader-thread-_schemas]: Shutdown completed (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread)
schema-registry  | [2023-01-31 02:14:40,518] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,518] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,518] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,526] INFO App info kafka.consumer for KafkaStore-reader-_schemas unregistered (org.apache.kafka.common.utils.AppInfoParser)
schema-registry  | [2023-01-31 02:14:40,526] INFO KafkaStoreReaderThread shutdown complete. (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread)
schema-registry  | [2023-01-31 02:14:40,526] INFO [Producer clientId=producer-1] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. (org.apache.kafka.clients.producer.KafkaProducer)
schema-registry  | [2023-01-31 02:14:40,528] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,529] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,529] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,529] INFO App info kafka.producer for producer-1 unregistered (org.apache.kafka.common.utils.AppInfoParser)
schema-registry  | [2023-01-31 02:14:40,529] INFO Kafka store producer shut down (io.confluent.kafka.schemaregistry.storage.KafkaStore)
schema-registry  | [2023-01-31 02:14:40,529] INFO Kafka store shut down complete (io.confluent.kafka.schemaregistry.storage.KafkaStore)
schema-registry  | [2023-01-31 02:14:40,530] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
schema-registry  | [2023-01-31 02:14:40,694] INFO Session: 0x100136f092b0007 closed (org.apache.zookeeper.ZooKeeper)
schema-registry  | [2023-01-31 02:14:40,695] INFO EventThread shut down for session: 0x100136f092b0007 (org.apache.zookeeper.ClientCnxn)
schema-registry  | ===> User
schema-registry  | uid=1000(appuser) gid=1000(appuser) groups=1000(appuser)
schema-registry  | ===> Configuring ...
schema-registry  | ===> Running preflight checks ... 
schema-registry  | ===> Check if Zookeeper is healthy ...
schema-registry  | SLF4J: Class path contains multiple SLF4J bindings.
schema-registry  | SLF4J: Found binding in [jar:file:/usr/share/java/cp-base-new/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
schema-registry  | SLF4J: Found binding in [jar:file:/usr/share/java/cp-base-new/slf4j-simple-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
schema-registry  | SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
schema-registry  | SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
schema-registry  | log4j:WARN No appenders could be found for logger (io.confluent.admin.utils.cli.ZookeeperReadyCommand).
schema-registry  | log4j:WARN Please initialize the log4j system properly.
schema-registry  | log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
schema-registry  | ===> Check if Kafka is healthy ...
schema-registry  | SLF4J: Class path contains multiple SLF4J bindings.
schema-registry  | SLF4J: Found binding in [jar:file:/usr/share/java/cp-base-new/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
schema-registry  | SLF4J: Found binding in [jar:file:/usr/share/java/cp-base-new/slf4j-simple-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
schema-registry  | SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
schema-registry  | SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
schema-registry  | log4j:WARN No appenders could be found for logger (io.confluent.admin.utils.cli.KafkaReadyCommand).
schema-registry  | log4j:WARN Please initialize the log4j system properly.
schema-registry  | log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions