如何在pysparkDataframe中将一列中的字典列表拆分为两列？

ldioqlga 于 2021-07-13 发布在 Spark

关注(0)|答案(1)|浏览(455)

我想将上面sparkDataframe的filteredaddress列拆分为两个新列，分别是flag和address：

customer_id|pincode|filteredaddress|                                                              Flag| Address
1000045801 |121005 |[{'flag':'0', 'address':'House number 172, Parvatiya Colony Part-2 , N.I.T'}]
1000045801 |121005 |[{'flag':'1', 'address':'House number 172, Parvatiya Colony Part-2 , N.I.T'}]
1000045801 |121005 |[{'flag':'1', 'address':'House number 172, Parvatiya Colony Part-2 , N.I.T'}]

谁能告诉我怎么做吗？

python apache-spark pyspark apache-spark-sql

来源：https://stackoverflow.com/questions/66263556/how-to-split-list-of-dictionary-in-one-column-into-two-columns-in-pyspark-datafr

1条答案

按热度按时间

ghg1uchk1#

你可以从 filteredaddress 使用键Map列：

df2 = df.selectExpr(
    'customer_id', 'pincode',
    "filteredaddress['flag'] as flag", "filteredaddress['address'] as address"
)

访问Map值的其他方法有：

import pyspark.sql.functions as F

df.select(
    'customer_id', 'pincode',
    F.col('filteredaddress')['flag'],
    F.col('filteredaddress')['address']
)

# or, more simply

df.select(
    'customer_id', 'pincode',
    'filteredaddress.flag',
    'filteredaddress.address'
)

赞(0）回复(0）举报 2021-07-13

我来回答

如何在pysparkDataframe中将一列中的字典列表拆分为两列？

1条答案

相关问题

热门标签

最新问答