这是我的spring batch配置:
@RequiredArgsConstructor
@Configuration
@Slf4j
@EnableBatchProcessing
public class SpringBatchConfig {
private final JobBuilderFactory jobBuilderFactory;
private final StepBuilderFactory stepBuilderFactory;
@Autowired
@Lazy
private CRMTService crmtService;
@Bean
public TaskExecutor myTaskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(10);
executor.setMaxPoolSize(15);
executor.setQueueCapacity(100);
executor.setThreadNamePrefix("Batch-");
executor.initialize();
return executor;
}
@Bean
public Step step() {
return stepBuilderFactory.get("step")
.<String, String>chunk(10)
.reader(itemReader())
.processor(itemProcessor())
.writer(itemWriter())
.taskExecutor(myTaskExecutor())
.build();
}
@Bean
public Job job() {
return jobBuilderFactory.get("job")
.start(step())
.build();
}
@Bean
@StepScope
public ItemReader<String> itemReader() {
List<String> list = crmtService.getVertrags();
return new IteratorItemReader<>(list != null ? list.iterator() : Collections.emptyIterator());
}
@Bean
@StepScope
public ItemProcessor<String, String> itemProcessor() {
return item -> {
//Logic -> This calls a method in service class. In that method I am calling an API for each string and saving the response to mongodb.
return item;
};
}
@Bean
@StepScope
public ItemWriter<String> itemWriter() {
return items -> {
//If we plan to log do anything in future
};
}
}
字符串
Jdk版本11
Sping Boot 版本2.6.3
我要在我的服务课上启动这个 Spring 批次。
当我不使用task executor时,我可以在mongodb中获取所有产品(214)[使用单线程我可以获取所有产品,但显然需要时间]。当我使用task executor时,我错过了1或2个产品(212/213)。
还有两个有趣的观察:
当spring batch被触发时,这些是我在控制台中获得的日志:
WARN [] 9758 --- [onPool-worker-1] o.s.b.c.c.a.DefaultBatchConfigurer : No datasource was provided...using a Map based JobRepository
WARN [] 9758 --- [onPool-worker-1] o.s.b.c.c.a.DefaultBatchConfigurer : No transaction manager was provided, using a ResourcelessTransactionManager
INFO [] 9758 --- [onPool-worker-1] o.s.b.c.l.support.SimpleJobLauncher : **No TaskExecutor has been set, defaulting to synchronous executor**.
型
即使我已经在步骤bean中设置了任务执行器。我还是得到了这个消息。
当spring batch真正开始时,正在运行的线程来自myTaskExecutor”,因为在控制台中我可以看到线程的名称以“Batch”开头,这是我在spring batch config class中添加的前缀。
[ Batch-1] c.t.openapi.crmt.conf.SpringBatchConfig : Processing 123
[ Batch-3] c.t.openapi.crmt.conf.SpringBatchConfig : Processing 456
[ Batch-2] c.t.openapi.crmt.conf.SpringBatchConfig : Processing 789
型
问题1:为什么即使我显式定义了任务执行器,也找不到它?
问题2:如果没有选择,那么为什么spring batch中的线程以“Batch-”开头
即使我使用这个而不是任务执行器:
@Bean
public Step step() {
return stepBuilderFactory.get("step")
.<VertragsListe, VertragsListe>chunk(10)
.reader(itemReader())
.processor(itemProcessor())
.writer(itemWriter())
.taskExecutor(new SimpleAsyncTaskExecutor())
.build();
}
型
我仍然得到“未找到任务执行者”。
1条答案
按热度按时间bfhwhh0e1#
我认为这与WriterExecutor有关,尝试使用上下文处理器调试API,对我有效