我试图在spark集群上使用spark submit运行python模块。该包具有某些依赖项，这些依赖项已压缩到virtualenv中。我正在使用下面的命令来运行它。

export PYSPARK_PYTHON=./environment/bin/python

spark-submit --deploy-mode cluster --master yarn --conf spark.yarn.maxAttempts=1 --conf spark.yarn.appMasterEnv.PYSPARK_PYTHON=${PYSPARK_PYTHON} --conf spark.executorEnv.PYSPARK_PYTHON=${PYSPARK_PYTHON} --jars hdfs:///jars/deequ-1.2.2-spark-3.0.jar --archives hdfs:///jars/chitrakoot/envs/dependencies.zip#environment --files hdfs:///jars/chitrakoot/configs/validation_config.json --py-files hdfs:///jars/chitrakoot-0.0.0-py3.7.egg hdfs:///jars/chitrakoot/driver.py --config_file_path hdfs:///jars/chitrakoot/configs/validation_config.json

python版本：3.7 spark版本：3.0.1
运行此操作时，我遇到以下错误。

LogType:stdout
LogLastModifiedTime:Tue Jul 06 02:31:43 +0530 2021
LogLength:84
LogContents:
./environment/bin/python: 1: ./environment/bin/python: Syntax error: "(" unexpected

LogType:stderr
21/07/06 02:31:40 ERROR ApplicationMaster: Uncaught exception:
org.apache.spark.SparkException: Exception thrown in awaitResult:
        at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:302)
        at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:500)
        at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:264)
        at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:890)
        at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:889)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
        at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:889)
        at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
Caused by: org.apache.spark.SparkUserAppException: User application exited with 2
        at org.apache.spark.deploy.PythonRunner$.main(PythonRunner.scala:103)
        at org.apache.spark.deploy.PythonRunner.main(PythonRunner.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:728)

提前谢谢！