如何从这个Dataframe中删除tu,因为我无法将数据加载到我的配置单元表中,该配置单元表具有基于id的分区。
总是因为“u”而出现illigalargument错误
>>schema = StructType([ StructField("ability_id", StringType(), True),
StructField("bid", StringType(), True),
StructField("bidtime", StringType(), True),
StructField("bidder", StringType(), True),
StructField("bidderrate", StringType(), True),
StructField("openbid", StringType(), True),
StructField("price", StringType(), True)])`
>>df = sqlContext.createDataFrame(auction_data,schema)
>>df.registerTempTable("auction")
>>first_line = sqlContext.sql("select * from auction where auctionid=8211480551").collect()
>>for i in first_line:
>> print i
>>Row(ability_id=u'8211480551', bid=u'52.99', bidtime=u'1.201505', bidder=u'hanna1104', bidderrate=u'94', openbid=u'49.99', price=u'311.6')
>>Row(ability_id=u'8211480551', bid=u'50.99', bidtime=u'1.203843', bidder=u'wrufai1', bidderrate=u'90', openbid=u'49.99', price=u'311.6')`enter code here`
sqlContext.sql(""" INSERT INTO TABLE dev_core_t1.PINO_KLANT_3 partition (abillity_id) SELECT bid,bidtime,bidder,bidderrate,openbid,price from temp """)
1条答案
按热度按时间idv4meu81#
此问题已解决,似乎任何小于或等于2.0.0.x.x.x的spark版本都不起作用。
它只适用于spark版本2.1.x.x.x或更高版本。