DataX mongodbreader may should not always use MULTIPLE mode

kt06eoxx 于 4个月前发布在 Go

关注(0)|答案(1)|浏览(124)

when mongodbreader connect to db server it use :
public MongoClient(List<ServerAddress> seeds, List<MongoCredential> credentialsList)
construction method to instance MongoClient.
this will cause client use MULTIPLE mode to make up cluster settings.
Cluster created with settings {hosts=[10.7.x.x:30016], mode=MULTIPLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500}

In my scenario，mongodb was run on k8s with REPLICA_SET mode, this will cause client keep trying to touch backend service:

Server 10.7.x.x:30016 is no longer a member of the replica set.  Removing from client view of cluster.
Canonical address mongodb-1.mongodb-headless.bangwo8-db-prod.svc.cluster.local:27017 does not match server address.  Removing 10.7.x.x:30016 from client view of cluster
Updating cluster description to  {type=REPLICA_SET, servers=[{address=mongodb-0.mongodb-headless.bangwo8-db-prod.svc.cluster.local:27017, type=UNKNOWN, state=CONNECTING}, {address=mongodb-arbiter-0.mongodb-arbiter-headless.bangwo8-db-prod.svc.cluster.local:27017, type=UNKNOWN, state=CONNECTING}, {address=mongodb-1.mongodb-headless.bangwo8-db-prod.svc.cluster.local:27017, type=UNKNOWN, state=CONNECTING}]

finally datax job will faild with java.net.UnknownHostException.

but if I change to use construction method :
public MongoClient(ServerAddress addr, List<MongoCredential> credentialsList)
the MongoClient will use SINGLE to make up cluster settings. And then I can search data normally.

DataX

来源：https://github.com/alibaba/DataX/issues/1645

1条答案

按热度按时间

sdnqo3pr1#

DataX/mongodbreader/src/main/java/com/alibaba/datax/plugin/reader/mongodbreader/util/MongoUtil.java

Line 48 in 2f5c9cf

| | returnnewMongoClient(parseServerAddress(addressList), Arrays.asList(credential)); |

change to :

List<ServerAddress> serverAddressList = parseServerAddress(addressList);
if(serverAddressList.size() == 1){
    return new MongoClient(serverAddressList.get(0), Arrays.asList(credential));
}else{
    return new MongoClient(serverAddressList, Arrays.asList(credential));
}

<dependency>
  <groupId>org.mongodb</groupId>
  <artifactId>mongo-java-driver</artifactId>
  <version>3.2.2 -> 3.12.11</version>
</dependency>

赞(0）回复(0）举报 4个月前

我来回答

DataX mongodbreader may should not always use MULTIPLE mode

1条答案

相关问题

热门标签

最新问答