aws glue刚刚从dynamodb读取了选定的记录

6ioyuze2  于 2021-07-14  发布在  Spark
关注(0)|答案(0)|浏览(295)

下面的代码正在读取完整的dynamo db表。有没有办法,我们可以读取选定的行(给定一组分区列的有限值)

  1. import sys
  2. from pyspark.context import SparkContext
  3. from awsglue.context import GlueContext
  4. from awsglue.job import Job
  5. from awsglue.utils import getResolvedOptions
  6. args = getResolvedOptions(sys.argv, ["JOB_NAME"])
  7. glue_context= GlueContext(SparkContext.getOrCreate())
  8. job = Job(glue_context)
  9. job.init(args["JOB_NAME"], args)
  10. dyf = glue_context.create_dynamic_frame.from_options(
  11. connection_type="dynamodb",
  12. connection_options={
  13. "dynamodb.input.tableName": "test_source",
  14. "dynamodb.throughput.read.percent": "1.0",
  15. "dynamodb.splits": "100"
  16. }
  17. )

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题