我尝试使用CrudRepository.saveAll()
方法保存一个从XLS文件创建的大型List
(超过100K个条目)。由于数据大小的原因,saveAll
方法需要很长时间才能运行,所以我尝试使用Apache Commons Collections ListUtils.partition
对列表进行分区,并在循环中调用saveAll
方法,以提高性能:
public interface ClaimsSegmentRepository extends CrudRepository<ClaimsSegment, String> {}
public class FileLoaderService {
@Async("asyncFileLoadExecutor")
public CompletableFuture<List<String>> loadFile(String fileName) throws Exception {
for(List<ClaimsSegment> claimsSegmentGrp : ListUtils.partition(claimsSegmentList, MAX_PARTITION_SIZE)) {
CompletableFuture.supplyAsync(() -> claimSegmentRepo.saveAll(claimsSegmentGrp))
.thenAccept(claimSegmentList -> {
//Processing child objects for Claim Segment List
});
}
}
这是抛出错误HHH000010: On release of batch it still contained JDBC statements
,所以我读了一些更多的在线和看到这个建议,我的回购类:
public interface ClaimsSegmentRepository extends CrudRepository<ClaimsSegment, String> {
@Async
<S extends ClaimsSegment> CompletableFuture<S> saveAll()
}
但是,这会导致编译错误,并抱怨返回类型不匹配:
org.springframework.beans.factory.UnsatisfiedDependencyException:
Error creating bean with name 'loadController':
Unsatisfied dependency expressed through field 'loaderSvc'; nested exception is org.springframework.beans.factory.BeanCreationException:
Error creating bean with name 'fileLoaderService' defined in file [Path\to\file\FileLoaderService.class]:
Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException:
Failed to instantiate [nom.side.xls.service.FileLoaderService]:
Constructor threw exception; nested exception is java.lang.Error: Unresolved compilation problem:
The method saveAll(Iterable<ClaimsSegment>) is ambiguous for the type ClaimsSegmentRepository
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:659) ~[spring-beans-5.3.23.jar:5.3.23]
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:639) ~[spring-beans-5.3.23.jar:5.3.23]
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119) ~[spring-beans-5.3.23.jar:5.3.23]
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399) ~[spring-beans-5.3.23.jar:5.3.23]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431) ~[spring-beans-5.3.23.jar:5.3.23]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619) ~[spring-beans-5.3.23.jar:5.3.23]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.23.jar:5.3.23]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.23.jar:5.3.23]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.23.jar:5.3.23]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.23.jar:5.3.23]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.23.jar:5.3.23]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955) ~[spring-beans-5.3.23.jar:5.3.23]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918) ~[spring-context-5.3.23.jar:5.3.23]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) ~[spring-context-5.3.23.jar:5.3.23]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:147) ~[spring-boot-2.7.4.jar:2.7.4]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:734) ~[spring-boot-2.7.4.jar:2.7.4]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408) ~[spring-boot-2.7.4.jar:2.7.4]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:308) ~[spring-boot-2.7.4.jar:2.7.4]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) ~[spring-boot-2.7.4.jar:2.7.4]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) ~[spring-boot-2.7.4.jar:2.7.4]
at nom.side.xls.ExcelFileLoaderApplication.main(ExcelFileLoaderApplication.java:10) ~[classes/:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49) ~[spring-boot-devtools-2.7.4.jar:2.7.4]
...
@Async
注解对于find
方法是有效的,但是对于save
和saveAll
就不行了。
谢谢你,
1条答案
按热度按时间ct3nt3jp1#
要使用 Boot 和SpringDataJPA进行批量插入,您只需要做两件事:
1.将选项spring.jpa.properties. hib. jdbc.batch_size设置为所需的适当值(例如:第20段)
1.请使用存储库的saveAll()方法和准备插入的实体列表。
工作示例为here。