我正在尝试使用spark connector for azure sql databases library(github)从spark获取数据并将其批量插入azure数据库。我尝试使用“activedirectorypassword”作为身份验证方法,但adal4j总是抛出一个authenticationexception异常,该异常表示抛出了javax.mail.internet.parseexception。
我正在构建一个包含所有库的胖jar,并使用spark submit运行它。
azure组织有一个自定义域。
对于用户配置选项,我尝试了:
myuser@mydomain.com
我的用户
myuser@mydomain.com@mydb.database.windows.net
myuser@mydomain.com@database.windows.net
myuser@organization.onmicrosoft.com
当我将字段留空时,它会抛出一个错误,即用户名为空。
我可以使用该用户通过azure active directory登录azure data studio。
下面是我用来加载数据的截图:
val config = Config(Map(
"url" -> "myDB.database.windows.net",
"databaseName" -> "DatabaseName",
"user" -> "myUser@myDomain.com",
"password" -> "****",
"authentication" -> "ActiveDirectoryPassword",
"encrypt" -> "true",
"trustServerCertificate" -> "false",
"hostNameInCertificate" -> "myDB.database.windows.net",
"dbTable" -> "test.testing",
"bulkCopyBatchSize" -> "2500",
"bulkCopyTableLock" -> "true",
"bulkCopyTimeout" -> "600"
))
val session = SparkSession.builder.appName("App").enableHiveSupport.getOrCreate
val sqlContext = session.sqlContext
val df = sqlContext.sql(s"SELECT * FROM myTable")
df.bulkCopyToSqlDB(config)
这里抛出的异常是:
com.microsoft.sqlserver.jdbc.SQLServerException: Failed to authenticate the user myUser@myDomain.com in Active Directory (Authentication=ActiveDirectoryPassword).
at com.microsoft.sqlserver.jdbc.SQLServerADAL4JUtils.getSqlFedAuthToken(SQLServerADAL4JUtils.java:62)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.getFedAuthToken(SQLServerConnection.java:4264)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.onFedAuthInfo(SQLServerConnection.java:4237)
......Truncated for readability......
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.util.concurrent.ExecutionException: com.microsoft.aad.adal4j.AuthenticationException: javax/mail/internet/ParseException
at com.microsoft.sqlserver.jdbc.SQLServerADAL4JUtils.getSqlFedAuthToken(SQLServerADAL4JUtils.java:60)
... 48 more
Caused by: com.microsoft.aad.adal4j.AuthenticationException: javax/mail/internet/ParseException
at com.microsoft.sqlserver.jdbc.SQLServerADAL4JUtils.getSqlFedAuthToken(SQLServerADAL4JUtils.java:53)
... 48 more
提前谢谢
暂无答案!
目前还没有任何答案,快来回答吧!