kafka-connect无法找到已开发插件的类

vd2z7a6w 于 2021-06-08 发布在 Kafka

关注(0)|答案(1)|浏览(358)

我为实现sinkconnector的kafka connect创建了一个插件，我使用gradle jar task将其打包到一个jar中：

jar {
  archiveName='name.jar'
}

我将它复制到kafka集群的一个文件夹中，并设置classpath=where-my-jar。然后我执行kafka脚本来启动独立连接，它给我一个错误，说找不到我的类：

[2017-07-25 05:15:52,084] WARN could not get type for name mypackage.SplunkSinkConnector from any class loader (org.reflections.Reflections:384)
org.reflections.ReflectionsException: could not get type for name mypackage.SplunkSinkConnector
        at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:378)
        at org.reflections.ReflectionUtils.forNames(ReflectionUtils.java:397)
        at org.reflections.Reflections.getSubTypesOf(Reflections.java:367)
        at org.apache.kafka.connect.runtime.PluginDiscovery.connectorPlugins(PluginDiscovery.java:76)
        at org.apache.kafka.connect.runtime.PluginDiscovery.scanClasspathForPlugins(PluginDiscovery.java:70)
        at org.apache.kafka.connect.runtime.AbstractHerder$1.run(AbstractHerder.java:354)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: mypackage.SplunkSinkConnector
        at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:376)
        ... 6 more
[2017-07-25 05:15:52,419] INFO Reflections took 1586 ms to scan 62 urls, producing 3002 keys and 15379 values  (org.reflections.Reflections:229)
[2017-07-25 05:15:52,420] WARN could not get type for name mypackage.SplunkSinkConnector from any class loader (org.reflections.Reflections:384)
org.reflections.ReflectionsException: could not get type for name mypackage.SplunkSinkConnector
        at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:378)
        at org.reflections.ReflectionUtils.forNames(ReflectionUtils.java:397)
        at org.reflections.Reflections.getSubTypesOf(Reflections.java:367)
        at org.apache.kafka.connect.runtime.ConnectorFactory.getConnectorClass(ConnectorFactory.java:69)
        at org.apache.kafka.connect.runtime.ConnectorFactory.newConnector(ConnectorFactory.java:38)
        at org.apache.kafka.connect.runtime.AbstractHerder.getConnector(AbstractHerder.java:334)
        at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:233)
        at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:159)
        at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:93)
Caused by: java.lang.ClassNotFoundException: mypackage.SplunkSinkConnector
        at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:376)
        ... 8 more

你知道他为什么不拿我的jar吗？
谢谢您

编辑：kakfa connect版本10.2.1，根据脚本，classpath的计算公式为：classpath=“$classpath”：“$kafka\u home/libs/*”

Java apache-kafka apache-kafka-connect

来源：https://stackoverflow.com/questions/45299578/kafka-connect-cant-find-class-of-developed-plugin

1条答案

按热度按时间

你能检查一下你的.jar文件以确保类在那里吗。将scala用作jvm shell：

// Or Maybe try $CLASSPATH with the full classpath that you are using.
scala -classpath path-to-jar.jar
// If class is not loaded, this will trigger an error.
classOf[mypackage.SplunkSinkConnector]

仅供参考，我正是这样做的，使用一个自定义的.jar插件，通过classpath环境变量加载kafka connect。
更新：这是我的 build.gradle 我的插件的文件。在我看来，这是用一些简单的依赖关系构建java.jar的最简单方法。我用玻璃做jar gradle jar 它将在 ./build/libs/(project-name).jar :

apply plugin: 'java'
apply plugin: 'idea'

sourceCompatibility = 1.8

repositories {
    mavenLocal()
    mavenCentral()
    maven { url "http://packages.confluent.io/maven/" }
}

configurations {
    all*.exclude group: 'org.slf4j', module: 'slf4j-log4j12'
    all*.exclude group: 'log4j'
}

dependencies {
    compile 'io.confluent:kafka-connect-storage-partitioner:3.2.1'
    compile 'org.apache.kafka:connect-api:0.10.2.1'

    testCompile 'junit:junit:4.+'
}

idea {
    project {
        languageLevel = '1.8'
    }
}

赞(0）回复(0）举报 2021-06-08

相关问题

热门标签

Java query python Node 开发语言 request Util 数据库 Table 后端算法 Logger Message Element Parser

最新问答

xxl-job 安全组扫描到执行器端口服务存在信息泄露漏洞
回答(1) 发布于 6个月前
xxl-job 不能和nacos兼容？
回答(3) 发布于 6个月前
xxl-job 任务执行完后无法结束，日志一直转圈
回答(3) 发布于 6个月前
xxl-job-admin页面上查看调度日志样式问题
回答(1) 发布于 6个月前
xxl-job 参数512字符限制能否去掉
回答(1) 发布于 6个月前