为使您的问题得到快速解决，在建立Issues前，请您先通过如下方式搜索是否有相似问题:【搜索issue关键字】【使用labels筛选】【官方文档】
如果您没有查询到相似问题，为快速解决您的提问，建立issue时请提供如下细节信息：
- 标题：简洁、精准概括您的问题，例如“Insufficient Memory xxx" ”
- 版本、环境信息：
1）PaddlePaddle版本：请提供您的PaddlePaddle版本号，例如1.1或CommitID
2）CPU：预测若用CPU，请提供CPU型号，MKL/OpenBlas/MKLDNN/等数学库使用情况
3）GPU：预测若用GPU，请提供GPU型号、CUDA和CUDNN版本号
4）系统环境：请您描述系统类型、版本，例如Mac OS 10.14，Python版本
- 训练信息
1）单机/多机，单卡/多卡
2）显存信息
3）Operator信息
- 复现信息：如为报错，请给出复现环境、复现步骤
- 问题描述：请详细描述您的问题，同步贴出报错信息、日志、可复现的代码片段
Thank you for contributing to PaddlePaddle.
Before submitting the issue, you could search issue in the github in case that there was a similar issue submitted or resolved before.
If there is no solution,please make sure that this is a training issue including the following details:
System information
-PaddlePaddle version （eg.1.1）or CommitID
-CPU: including CPUMKL/OpenBlas/MKLDNN version
-GPU: including CUDA/CUDNN version
-OS Platform (eg.Mac OS 10.14)
-Other imformation: Distriuted training/informantion of operator/
Graphics card storage
To Reproduce
Steps to reproduce the behavior
Describe your current behavior
Code to reproduce the issue
Other info / logs
3条答案
按热度按时间3phpmpom1#
2020-06-04 18:47:00,840-WARNING: consumer[9307] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,841-WARNING: consumer[9308] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,841-WARNING: consumer[9309] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,841-WARNING: consumer[9310] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,841-WARNING: consumer[9311] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,841-WARNING: consumer[9312] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,841-WARNING: consumer[9313] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,841-WARNING: consumer[9314] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,841-WARNING: 8 consumers have exited abnormally!!!
2020-06-04 18:47:00,842-WARNING: consumer[9307] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,842-WARNING: consumer[9308] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,842-WARNING: consumer[9309] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,842-WARNING: consumer[9310] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,842-WARNING: consumer[9311] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,842-WARNING: consumer[9312] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,842-WARNING: consumer[9313] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,842-WARNING: consumer[9314] exit abnormally with exitcode[-11]
2020-06-04 18:47:00,842-WARNING: 8 consumers have exited abnormally!!!
2020-06-04 18:47:00,843-WARNING: Your reader has raised an exception!
Exception in thread Thread-11:
Traceback (most recent call last):
File "/Users/qingyang/opt/anaconda3/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/Users/qingyang/opt/anaconda3/lib/python3.7/threading.py", line 870, in run
self._target(self._args,self._kwargs)
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/reader.py", line 1156, inthread_main*
six.reraise(*sys.exc_info())
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/six.py", line 703, in reraise
raise value
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/reader.py", line 1136, inthread_main
for tensors in self._tensor_reader():
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/reader.py", line 1206, intensor_reader_impl
for slots in paddle_reader():
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/data_feeder.py", line 506, inreader_creator
for item in reader():
File "/Users/qingyang/PaddleDetection/ppdet/data/reader.py", line 421, in _reader
reader.reset()
File "/Users/qingyang/PaddleDetection/ppdet/data/parallel_map.py", line 267, in reset
assert self._consumer_healthy(), "cannot start another pass of data"
AssertionError: cannot start another pass of data for some consumers exited abnormally before!!!
/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/executor.py:1070: UserWarning: The following exception is not an EOF exception.
"The following exception is not an EOF exception.")
Traceback (most recent call last):
File "tools/train.py", line 366, in
main()
File "tools/train.py", line 239, in main
outs = exe.run(compiled_train_prog, fetch_list=train_values)
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/executor.py", line 1071, in run
six.reraise(*sys.exc_info())
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/six.py", line 703, in reraise
raise value
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/executor.py", line 1066, in run
return_merged=return_merged)
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/executor.py", line 1167, in _run_impl
return_merged=return_merged)
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/executor.py", line 879, in _run_parallel
tensors = exe.run(fetch_var_names, return_merged)._move_to_list()
paddle.fluid.core_avx.EnforceNotMet:
C++ Call Stacks (More useful to developers):
0 std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > paddle::platform::GetTraceBackString<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&>(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, char const*, int)
1 paddle::operators::reader::BlockingQueue<std::__1::vector<paddle::framework::LoDTensor, std::__1::allocatorpaddle::framework::LoDTensor > >::EnforceNotKilled()
2 paddle::operators::reader::BlockingQueue<std::__1::vector<paddle::framework::LoDTensor, std::__1::allocatorpaddle::framework::LoDTensor > >::Receive(std::__1::vector<paddle::framework::LoDTensor, std::__1::allocatorpaddle::framework::LoDTensor >)
3 paddle::operators::reader::PyReader::ReadNext(std::__1::vector<paddle::framework::LoDTensor, std::__1::allocatorpaddle::framework::LoDTensor >)
4 std::__1::__packaged_task_func<std::__1::__bind<paddle::operators::reader::BufferedReader::ReadAsync(unsigned long)::$_0>, std::__1::allocator<std::__1::__bind<paddle::operators::reader::BufferedReader::ReadAsync(unsigned long)::$_0> >, unsigned long ()>::operator()()
5 std::__1::packaged_task<unsigned long ()>::operator()()
6 ThreadPool::ThreadPool(unsigned long)::'lambda'()::operator()() const
7 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_deletestd::__1::__thread_struct >, ThreadPool::ThreadPool(unsigned long)::'lambda'()> >(void*)
Python Call Stacks (More useful to users):
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/framework.py", line 2610, in append_op
attrs=kwargs.get("attrs", None))
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/reader.py", line 1078, in _init_non_iterable
attrs={'drop_last': self._drop_last})
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/reader.py", line 976, ininit
self._init_non_iterable()
File "/Users/qingyang/opt/anaconda3/lib/python3.7/site-packages/paddle/fluid/reader.py", line 608, in from_generator
iterable, return_list, drop_last)
File "/Users/qingyang/PaddleDetection/ppdet/modeling/architectures/yolo.py", line 152, in build_inputs
iterable=iterable) if use_dataloader else None
File "tools/train.py", line 112, in main
feed_vars, train_loader = model.build_inputs(**inputs_def)
File "tools/train.py", line 366, in
main()
Error Message Summary:
Error: Blocking queue is killed because the data reader raises an exception
[Hint: Expected killed_ != true, but received killed_:1 == true:1.] at (/home/teamcity/work/ef54dc8a5b211854/paddle/fluid/operators/reader/blocking_queue.h:141)
[operator < read > error]
5t7ly7z52#
数据集是官方的，配置文件也是官方的，总是报这种错误，似乎是说进程被kill掉，有没有高人指点一下
xmq68pz93#
paddle pgl deepwalk bug report
environment
MacOS Big Sur 11.0.1
Intel core i5 / 8GB RAM
conda environment
python 3.7.8 64bit
paddlepaddle==1.8.5
pgl==1.2.1
Passed CMD
Error CMD: change '--walk_len' from '5' to '10'
Python Running Environment