python版本的wordcount在flink上失败

ia2d9nvy  于 2021-06-24  发布在  Flink
关注(0)|答案(3)|浏览(429)

在centos(centos8.0.1905(64位))上,我尝试在flink(1.9)上运行python(3.6.8)版本的wordcount程序,如下所述。我得到的错误如下。同样的环境也适用于java版本的wordcount程序。我错过了什么?非常感谢您的帮助。错误

$ ./bin/flink run -py examples/python/table/batch/word_count.py
Starting execution of program
org.apache.flink.client.program.OptimizerPlanEnvironment$ProgramAbortException
    at org.apache.flink.client.python.PythonDriver.main(PythonDriver.java:83)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576)
    at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
    at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
    at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
    at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
    at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
    at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
    at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
    at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
    at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)
wixjitnu

wixjitnu1#

在运行测试之前,您应该使用pip构建源代码并安装pyflink,可以在下面找到详细信息:https://ci.apache.org/projects/flink/flink-docs-release-1.9/flinkdev/building.html#build-皮Flink。

polhcujo

polhcujo2#

我认为你提供的信息没有显示根本原因。一般来说,我们有两种方法来运行flink1.9中的示例,一种是前面提供的答案1,即从源代码安装pyflink,另一种是下载二进制软件包。具体如下:
检查python版本,最好使用python2.7.6+作为flink1.9。
下载curl-ohttp://mirrors.gigenet.com/apache/flink/flink-1.9.1/flink-1.9.1-bin-scala_2.11.tgz 焦油-zxvf flink-1.9.1-bin-scala\u 2.11.tgz
config(可选)vi flink-1.9.1/conf/flink-conf.yaml->rest.port:4000
启动集群cd flink-1.9.1;bin/start-cluster.sh文件
运行word count example./bin/flink run-py examples/python/table/batch/word\u count.py
更多详情请参见:https://enjoyment.cool/2020/01/19/three-min-series-run-the-example-of-wordcount-in-pyflink-1-9/#more
希望对你有帮助!

zynd9foi

zynd9foi3#

感谢孙金成的鼓励和支持
我把$flink\u home/opt/python/\uumacosx移到了上面一级。程序运行时没有任何错误。因为我是在linux上运行的,所以我不需要macosx。我会删除它

相关问题