使用hbase客户端api创建一个大表

klr1opcd  于 2021-06-09  发布在  Hbase
关注(0)|答案(1)|浏览(542)

我正在使用scala中的clientapi与googlecloud大表一起工作,我试图创建一个具有单列族的表,但是我遇到了错误
下面是我写的代码:

  1. `object TestBigtable {
  2. val columnFamilyName = Bytes.toBytes("cf1")
  3. def createConnection(ProjectId: String, InstanceID: String): Connection = {
  4. BigtableConfiguration.connect(ProjectId, InstanceID)
  5. }
  6. def createTableIfNotExists(connection: Connection, name: String) = {
  7. val tableName = TableName.valueOf(name)
  8. val admin = connection.getAdmin()
  9. if (!admin.tableExists(tableName)) {
  10. val tableDescriptor = new HTableDescriptor(tableName)
  11. tableDescriptor.addFamily(
  12. new HColumnDescriptor(columnFamilyName))
  13. admin.createTable(tableDescriptor)
  14. }
  15. }
  16. def runner(projectId: String,
  17. instanceId: String,
  18. tableName: String) = {
  19. val createTableConnection = createConnection(projectId, instanceId)
  20. try {
  21. createTableIfNotExists(createTableConnection, tableName)
  22. } finally {
  23. createTableConnection.close()
  24. }
  25. }`

一旦我执行jar,就会出现以下一组错误:

  1. 18/07/25 10:36:20 INFO com.google.cloud.bigtable.grpc.BigtableSession: Bigtable options: BigtableOptions{dataHost=bigtable.googleapis.com, adminHost=bigtableadmin.googleapis.com, port=443, projectId=renault-ftt, instanceId=testfordeletion, appProfileId=, userAgent=hbase-1.4.3, credentialType=DefaultCredentials, dataChannelCount=4, retryOptions=RetryOptions{retriesEnabled=true, allowRetriesWithoutTimestamp=false, statusToRetryOn=[UNAUTHENTICATED, ABORTED, DEADLINE_EXCEEDED, UNAVAILABLE], initialBackoffMillis=5, maxElapsedBackoffMillis=60000, backoffMultiplier=2.0, streamingBufferSize=60, readPartialRowTimeoutMillis=60000, maxScanTimeoutRetries=3}, bulkOptions=BulkOptions{asyncMutatorCount=2, useBulkApi=true, bulkMaxKeyCount=125, bulkMaxRequestSize=1048576, autoflushMs=0, maxInflightRpcs=40, maxMemory=97307852, enableBulkMutationThrottling=false, bulkMutationRpcTargetMs=100}, callOptionsConfig=CallOptionsConfig{useTimeout=false, shortRpcTimeoutMs=60000, longRpcTimeoutMs=600000}, usePlaintextNegotiation=false, useCachedDataPool=false}.
  2. 18/07/25 10:36:20 INFO com.google.cloud.bigtable.grpc.io.OAuthCredentialsCache: Refreshing the OAuth token
  3. Exception in thread "grpc-default-executor-0" java.lang.IllegalAccessError: tried to access field com.google.protobuf.AbstractMessage.memoizedSize from class com.google.bigtable.admin.v2.ListTablesRequest
  4. at com.google.bigtable.admin.v2.ListTablesRequest.getSerializedSize(ListTablesRequest.java:236)
  5. at io.grpc.protobuf.lite.ProtoInputStream.available(ProtoInputStream.java:108)
  6. at io.grpc.internal.MessageFramer.getKnownLength(MessageFramer.java:204)
  7. at io.grpc.internal.MessageFramer.writePayload(MessageFramer.java:136)
  8. at io.grpc.internal.AbstractStream.writeMessage(AbstractStream.java:52)
  9. at io.grpc.internal.DelayedStream$5.run(DelayedStream.java:218)
  10. at io.grpc.internal.DelayedStream.drainPendingCalls(DelayedStream.java:132)
  11. at io.grpc.internal.DelayedStream.setStream(DelayedStream.java:101)
  12. at io.grpc.internal.DelayedClientTransport$PendingStream.createRealStream(DelayedClientTransport.java:361)
  13. at io.grpc.internal.DelayedClientTransport$PendingStream.access$300(DelayedClientTransport.java:344)
  14. at io.grpc.internal.DelayedClientTransport$5.run(DelayedClientTransport.java:302)
  15. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  16. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  17. at java.lang.Thread.run(Thread.java:748)

有人能帮我吗?

vltsax25

vltsax251#

回复所罗门的评论:
io.grpc.protobuf.lite在堆栈中。云bigtable客户端从未使用protobuf-lite进行过测试。依赖关系图会有所帮助。作为一个快速解决方案,您还可以尝试bigtable-hbase-1.x-shaded工件,而不是bigtable-hbase-1.x工件。
你使用 io.grpc.protobuf.lite 正在引发问题。据我所知, io.grpc.protobuf.lite 主要用于android客户端。
使用着色工件应该以更大的jar大小和潜在的内存占用为代价来防止依赖冲突。您可能还需要查看这些类似的问题报告以及它们是如何解决的:
https://groups.google.com/forum/#!主题/协议
https://github.com/grpc/grpc-java/issues/2300

相关问题