从timezonefinder()创建新的“timezone”列，在pyspark中输入经度和纬度列

7lrncoxx 于 2021-07-13 发布在 Spark

关注(0)|答案(1)|浏览(317)

我想创建一个包含等效经纬度时区的新列。已有列中的经度和纬度是timezonefinder函数的输入，即get\u timezone（）。我不停地 TypeError: an integer is required (got type Column) 谢谢。

from timezonefinder import TimezoneFinder

def get_timezone(longitude, latitude):
    tzf = TimezoneFinder()
    return tzf.timezone_at(lng=longitude, lat=latitude)

location_table = location_table.withColumn("timezone", get_timezone(location_table["location_longitude"], location_table["location_latitude"]))

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<command-253463262459944> in <module>
      8 
      9 # df = sqlContext.read.parquet(INPUT)
---> 10 location_table.withColumn("timezone", get_timezone(location_table["location_longitude"].cast(IntegerType()), location_table["location_latitude"].cast(IntegerType())))
     11 #   .write.parquet(OUTPUT)

<command-253463262459944> in get_timezone(longitude, latitude)
      3 def get_timezone(longitude, latitude):
      4     tzf = TimezoneFinder()
----> 5     return tzf.timezone_at(lng=longitude, lat=latitude)
      6 
      7 # udf_timezone = F.udf(get_timezone, StringType())

/databricks/python/lib/python3.7/site-packages/timezonefinder/timezonefinder.py in timezone_at(self, lng, lat)
    657         :return: the timezone name of the matched timezone polygon. possibly "Etc/GMT+-XX" in case of an ocean timezone.
    658         """
--> 659         lng, lat = rectify_coordinates(lng, lat)
    660 
    661         shortcut_id_x, shortcut_id_y = coord2shortcut(lng, lat)

TypeError: an integer is required (got type Column)

apache-spark pyspark user-defined-functions timezone

来源：https://stackoverflow.com/questions/66225233/create-new-timezone-column-from-timezonefinder-with-longitude-and-latitude-c

1条答案

按热度按时间

xxls0lw81#

首先需要将函数转换为自定义项：

import pyspark.sql.functions as F
from timezonefinder import TimezoneFinder

@F.udf('string')
def get_timezone(longitude, latitude):
    if longitude is None or latitude is None:
        return None
    tzf = TimezoneFinder()
    return tzf.timezone_at(lng=longitude, lat=latitude)

location_table = location_table.withColumn("timezone", get_timezone(location_table["location_longitude"], location_table["location_latitude"]))

赞(0）回复(0）举报 2021-07-13

我来回答

从timezonefinder()创建新的“timezone”列，在pyspark中输入经度和纬度列

1条答案

相关问题

热门标签

最新问答