aiffort/amazon emr:专有网络/子网配置无效:需要子网:指定的示例类型m5.xlarge只能在专有网络中使用

92dk7w1h  于 2021-07-14  发布在  Spark
关注(0)|答案(0)|浏览(298)

我想创建一个通过亚马逊电子病历气流触发电子病历集群。emr集群出现在amazon emr的ui中,但出现一个错误:“专有网络/子网配置无效:需要子网:指定的示例类型m5.xlarge只能在专有网络中使用”
下面是中的代码段和配置详细信息 json 气流脚本中使用的此任务的格式。
我的问题是如何将专有网络和子网的信息(id码)合并到json中(如果可能的话)?没有明确的例子。
提示:已创建网络和ec2子网

  1. JOB_FLOW_OVERRIDES = {
  2. "Name": "sentiment_analysis",
  3. "ReleaseLabel": "emr-5.33.0",
  4. "Applications": [{"Name": "Hadoop"}, {"Name": "Spark"}], # We want our EMR cluster to have HDFS and Spark
  5. "Configurations": [
  6. {
  7. "Classification": "spark-env",
  8. "Configurations": [
  9. {
  10. "Classification": "export",
  11. "Properties": {"PYSPARK_PYTHON": "/usr/bin/python3"}, # by default EMR uses py2, change it to py3
  12. }
  13. ],
  14. }
  15. ],
  16. "Instances": {
  17. "InstanceGroups": [
  18. {
  19. "Name": "Master node",
  20. "Market": "SPOT",
  21. "InstanceRole": "MASTER",
  22. "InstanceType": "m5.xlarge",
  23. "InstanceCount": 1,
  24. },
  25. {
  26. "Name": "Core - 2",
  27. "Market": "SPOT", # Spot instances are a "use as available" instances
  28. "InstanceRole": "CORE",
  29. "InstanceType": "m5.xlarge",
  30. "InstanceCount": 2,
  31. },
  32. ],
  33. "KeepJobFlowAliveWhenNoSteps": True,
  34. "TerminationProtected": False, # this lets us programmatically terminate the cluster
  35. },
  36. "JobFlowRole": "EMR_EC2_DefaultRole",
  37. "ServiceRole": "EMR_DefaultRole",
  38. }
  1. create_emr_cluster = EmrCreateJobFlowOperator(
  2. task_id="create_emr_cluster",
  3. job_flow_overrides=JOB_FLOW_OVERRIDES,
  4. aws_conn_id="aws_default",
  5. emr_conn_id="emr_default",
  6. dag=dag,
  7. )

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题