spark上的java语言应用

llycmphe  于 2021-05-27  发布在  Spark
关注(0)|答案(0)|浏览(409)

我是spark的新手,我想用java语言在这个框架上运行一个应用程序。我尝试了以下代码:

  1. public class Alert_Arret {
  2. private static final SparkSession sparkSession = SparkSession.builder().master("local[*]").appName("Stateful Streaming Example").config("spark.sql.warehouse.dir", "file:////C:/Users/sgulati/spark-warehouse").getOrCreate();
  3. public static Properties Connection_db () {
  4. Properties connectionProperties = new Properties();
  5. connectionProperties.put("user", "xxxxx");
  6. connectionProperties.put("password", "xxxxx");
  7. connectionProperties.put("driver","com.mysql.jdbc.Driver");
  8. connectionProperties.put("url","xxxxxxxxxxxxxxxxx");
  9. return connectionProperties;
  10. }
  11. public static void GetData() {
  12. boolean checked = true;
  13. String dtable = "alerte_prog";
  14. String dtable2 = "last_tracking_tdays";
  15. Dataset<Row> df_a_prog = sparkSession.read().jdbc("jdbc:mysql://host:port/database", dtable, Connection_db());
  16. // df_a_prog.printSchema();
  17. Dataset<Row> track = sparkSession.read().jdbc("jdbc:mysql://host:port/database", dtable2, Connection_db());
  18. if (df_a_prog.select("heureDebut") != null && df_a_prog.select("heureFin") != null ) {
  19. track.withColumn("tracking_hour/minute", from_unixtime(unix_timestamp(col("tracking_time")), "HH:mm")).show() }
  20. }
  21. public static void main(String[] args) {
  22. Connection_db();
  23. GetData();
  24. }
  25. }

当我运行此代码时,不会显示任何内容,我得到以下结果:

  1. 0/05/11 14:00:31 WARN ProcfsMetricsGetter: Exception when trying to compute pagesize, as a result reporting of ProcessTree metrics is stopped
  2. # A fatal error has been detected by the Java Runtime Environment:
  3. # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000000006c40022a,pid=3376, tid=0x0000000000002e84

我使用intellij思想和spark:3.0.0版本。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题