我正在向配置单元表中写入一个Dataframe,并检查进程是否成功完成(配置单元中的数据加载进程),然后我想打印消息“数据写入进程已完成”,但如果由于某些错误(错误的表名、错误的表名、语法错误等)而失败,然后打印“数据写入过程失败”。我在写下面的代码:
import spark.implicits._
import scala.io.Source
import org.apache.spark.sql._
import scala.util.control.Exception._
import com.hortonworks.hwc.HiveWarehouseSession
import com.hortonworks.hwc.HiveWarehouseSession._
val sparksession1 = SparkSession.builder().appName("Appname")
.config("hive.exec.dynamic.partition", "true")
.config("hive.exec.dynamic.partition.mode", "nonstrict")
.config("hive.mapred.mode", "nonstrict")
.config("spark.sql.hive.hiveserver2.jdbc.url", "jdbc:hive2:xxxxxxxx")
.enableHiveSupport().getOrCreate()
val Hive2 = com.hortonworks.spark.sql.hive.llap.HiveWarehouseBuilder.session(sparksession1).build()
val someDF = Seq((1, 2019),(2, 2019),(3, 2019),(4, 2019),(5, 2019),(6, 2019))
.toDF("MONTH", "YEAR")
Hive2.setDatabase("DatabaseName")
val s= Try {
val result=someDF.write.format("com.hortonworks.spark.sql.hive.llap.HiveWarehouseConnector")
.option("database","DatabaseName").option("table","tablename").mode("append").save()
}val y = s match {
case Failure(e) => e match {
case e: Exception =>
}
case Success(s) =>
}
我收到下面的错误信息
<console>:4: error: ';' expected but 'match' found.
} match {
^
scala> case Success(_) =>
<console>:1: error: illegal start of definition
case Success(_) =>
^
scala> println("Data Wtriting Process has been Completed")
Data Wtriting Process has been Completed
scala> case Failure(_) =>
<console>:1: error: illegal start of definition
case Failure(_) =>
^
scala> println("Data Wtriting Process has been Failed")
Data Wtriting Process has been Failed
scala>
| }
<console>:2: error: eof expected but '}' found.
}
^
有人能指导我如何实现同样的目标吗。
注意:我正在使用hdp3.0并用spark/scala编写我的应用程序。
暂无答案!
目前还没有任何答案,快来回答吧!