我的代码spark2.4和python-3声明如下,带有spark上下文,但总是失败,出现以下错误。
是什么,我都错过了申报。
错误: java.lang.IllegalStateException
:用户未初始化spark上下文!。
# lib imported
from pyspark.sql import SparkSession
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job
from awsglue.utils import getResolvedOptions
import pyspark.sql.functions as F
from datetime import datetime
from pyspark.sql.utils import AnalysisException
main():
{
# Initialize start of the Glue job
sc = SparkContext()
glueContext = GlueContext(sc.getOrCreate())
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args['JOB_NAME'],args)
}
暂无答案!
目前还没有任何答案,快来回答吧!