mongodb迁移导致密钥重复

ybzsozfc  于 2021-07-08  发布在  Java
关注(0)|答案(2)|浏览(346)

在一个老项目中,我们正在尝试从basicspring迁移到springboot2.3.1。为此,由于我们有一个mongo数据库,我们必须从spring数据进行迁移-mongodb:1.10.18 where 这段代码是这样写的:

  1. DBCollection contextCollection = this.mongoTemplate.getCollection("productStock");
  2. BulkWriteOperation builder = contextCollection.initializeUnorderedBulkOperation();
  3. StockType stockItem = stockMessage.getStockItem();
  4. final BasicDBObject id = new BasicDBObject("storeID", stockItem.getStoreID()).append("productID", stockItem.getProductID());
  5. BulkWriteRequestBuilder bulkWriteRequestBuilder = builder.find(new BasicDBObject("_id", id));
  6. HashMap<String, Object> stock = new HashMap<>();
  7. Date currentDate = Calendar.getInstance().getTime();
  8. stock.put("value", stockItem.getValue());
  9. if (stockItem.getAssociateDate() != null) {
  10. stock.put("associateDate", stockItem.getAssociateDate());
  11. }
  12. if (stockItem.getLastAccessDateSource() != null) {
  13. stock.put("lastAccessDateSource", stockItem.getLastAccessDateSource());
  14. // check
  15. BasicDBObject ltLast = new BasicDBObject("$lt", stockItem.getLastAccessDateSource());
  16. BasicDBList dbList = new BasicDBList();
  17. dbList.add(new BasicDBObject(stockItem.getStockCategory() + ".lastAccessDateSource", ltLast));
  18. dbList.add(new BasicDBObject(stockItem.getStockCategory() + ".lastAccessDateSource", null));
  19. bulkWriteRequestBuilder = builder.find(new BasicDBObject("_id", id).append("$or", dbList));
  20. } else {
  21. stock.put("lastAccessDateSource", currentDate);
  22. }
  23. stock.put("lastUpdateDate", currentDate);
  24. BasicDBObject set = new BasicDBObject(stockItem.getStockCategory(), new Document(stock));
  25. bulkWriteRequestBuilder.upsert().updateOne(new BasicDBObject("$set", set));
  26. builder.execute();

到Spring Data -mongodb:3.0.1.release with 此更新的代码

  1. Map<String, List<StockType>> mapMultiUpdate = new HashMap<>();
  2. StockType stockItem = stockMessage.getStockItem();
  3. final Document id = new Document("storeID", stockItem.getStoreID()).append("productID", stockItem.getProductID());
  4. HashMap<String, Object> stock = new HashMap<>();
  5. Date currentDate = Calendar.getInstance().getTime();
  6. Document searchQuery = new Document("_id", id).append("$or", dbList);
  7. stock.put("value", stockItem.getValue());
  8. if (stockItem.getAssociateDate() != null) {
  9. stock.put("associateDate", stockItem.getAssociateDate());
  10. }
  11. if (stockItem.getLastAccessDateSource() != null) {
  12. stock.put("lastAccessDateSource", stockItem.getLastAccessDateSource());
  13. // check
  14. Document ltLast = new Document("$lt", stockItem.getLastAccessDateSource());
  15. List<Document> dbList = Lists.newArrayList();
  16. dbList.add(new Document(stockItem.getStockCategory() + ".lastAccessDateSource", ltLast));
  17. dbList.add(new Document(stockItem.getStockCategory() + ".lastAccessDateSource", null));
  18. } else {
  19. stock.put("lastAccessDateSource", currentDate);
  20. }
  21. //Bulk write options
  22. BulkWriteOptions bulkWriteOptions = new BulkWriteOptions();
  23. bulkWriteOptions.ordered(false);
  24. bulkWriteOptions.bypassDocumentValidation(true);
  25. MongoCollection<Document> mongoCollection = this.mongoTemplate.getCollection("productStoreStock");
  26. mongoCollection.bulkWrite(updateDocuments, bulkWriteOptions);

但是当新代码在已经存在的对象上执行时,我们会得到一个重复的键错误

  1. com.mongodb.MongoBulkWriteException: Bulk write operation error on server localhost:27017. Write errors: [BulkWriteError{index=0, code=11000, message='E11000 duplicate key error collection: test.productStoreStock index: _id_ dup key: { : { storeID: 400, productID: 100000 } }', details={}}].

我们也从mongo-java转换过来了-driver:3.6.4 to mongodb驱动程序-sync:4.0.4
编辑:
在测试阶段的第3步中,在空数据库/集合上引发此错误。步骤如下:
在特定日期用一种产品的库存开始收集
检查基值
在java中修改股票的值,但不修改日期并尝试更新它
由于mongo查询上的lt过滤器,检查值仍然是第一个
我们从来没有达到检查值,在迁移之前,这个测试一切正常

szqfcxe2

szqfcxe21#

  1. test.productStoreStock index: _id_ dup key: { : { storeID: 400, productID: 100000 } }', details={}}

看起来您正在用自己的“构造”id替换内部id字段。这不是一件好事。让mongo创建它自己的\u id,它们在不同的用法中保证是唯一的。。永远不会复制。将自己的companyid作为附加字段没有错,但是替换mongodbs自动生成的id字段是危险的。。。你已经发现了。

falq053o

falq053o2#

mongo没有更新,当找不到匹配项时,它正在升级(插入新行)。

相关问题