tensorflow Encounter Abort(core dump) in ClipOpsTest when changing shapes

rqcrx0a6  于 6个月前  发布在  其他
关注(0)|答案(6)|浏览(50)

System information

  • Ubuntu 20.04
  • python 3.7.11
  • In TensorFlow 2.5.0, 2.6.0, 2.7.0, 2.8.0

当我在TensorFlow中运行测试用例时,发现ClipOpsTest在使用巨大属性的shape时会崩溃。

Describe the current behavior

程序找到了与形状相关的错误,并返回了ValueError或MemoryError。

Contributing

  • Do you want to contribute a PR? (yes/no): no
  • Briefly describe your candidate solution(if contributing):
    Standalone code to reproduce the issue

提供一个最小必要的可重现问题的测试用例。如果可能的话,请分享一个链接到Colab/Jupyter/任何笔记本。

from tensorflow.python.framework import ops
from tensorflow.python.ops import clip_ops
from tensorflow.python.platform import test
from tensorflow.python.framework import constant_op

class ClipOpsTest(test.TestCase):

    def _testClipIndexedSlicesByNorm(self, values, indices, shape, max_norm, axes):
        values = constant_op.constant(values)
        indices = constant_op.constant(indices)
        shape = constant_op.constant(shape)
        indexed_slices = ops.IndexedSlices(values, indices, shape)
        clipped = clip_ops.clip_by_norm(indexed_slices, max_norm, axes)
        clipped = ops.convert_to_tensor(clipped)
        
    def testClipIndexedSlicesByNorm_Failed(self):
        values = [[[(- 3.0), 0.0, 0.0], [4.0, 0.0, 0.0]], [[0.0, 2.0, 0.0], [0.0, 0.0, (- 1.0)]]]
        indices = [2, 6]
        # shape = [9223372036854775807, 1, 9223372036854775807]
     
        shape = [9223372036854775807, 2, 3] 
        self._testClipIndexedSlicesByNorm(values, indices, shape, 4.0, None) # crashed

    def testClipIndexedSlicesByNorm_Pass(self):
        values = [[[(- 3.0), 0.0, 0.0], [4.0, 0.0, 0.0]], [[0.0, 2.0, 0.0], [0.0, 0.0, (- 1.0)]]]
        indices = [2, 6]

        shape = [10, 2, 3] 
        self._testClipIndexedSlicesByNorm(values, indices, shape, 4.0, None) # passed

if (__name__ == '__main__'):
    test.main()

Other info / logs 包括任何有助于诊断问题的日志或源代码。如果包括回溯信息,请包括完整的回溯。大型日志和文件应附加。

[ RUN ] ClipOpsTest.testClipIndexedSlicesByNorm

**2022-03-12 05:39:14.085939: F tensorflow/core/framework/tensor_shape.cc:404] Check failed: 0 <= new_num_elements (0 vs. -2)

Fatal Python error: Aborted
Current thread 0x00007fb591724180 (most recent call first):
File "/root/anaconda3/envs/tf2.8.0/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 12118 in unsorted_segment_sum
File "/root/anaconda3/envs/tf2.8.0/lib/python3.7/site-packages/tensorflow/python/framework/indexed_slices.py", line 448 in _indexed_slices_to_tensor
File "/root/anaconda3/envs/tf2.8.0/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1695 in convert_to_tensor
File "/root/anaconda3/envs/tf2.8.0/lib/python3.7/site-packages/tensorflow/python/profiler/trace.py", line 183 in wrapped
File "test_bug.py", line 16 in _testClipIndexedSlicesByNorm
File "test_bug.py", line 29 in testClipIndexedSlicesByNorm
File "/root/anaconda3/envs/tf2.8.0/lib/python3.7/unittest/case.py", line 628 in run
File "/root/anaconda3/envs/tf2.8.0/lib/python3.7/unittest/case.py", line 676 in **call
File "test_bug.py", line 76 in _testClipIndexedSlicesByNorm
File "test_bug.py", line 86 in testClipIndexedSlicesByNorm
File "test_bug.py", line 96 in main
File "test_bug.py", line 101 in init
File "test_bug.py", line 116 in runTests
File "test_bug.py", line 122 in init
File "test_bug.py", line 136 in suiteClass.call(self)
File "test_bug.py", line 146 in _runTestsInProcesses
File "test_bug.py", line 156 in _runTestsInSubProcesses
File "test_bug.py", line 166 in _runTestsInThreads
File "test_bug.py", line 176 in runTests
File "test_bug.py", line 186 in main
File "test_bug.py", line 204 in init
File "test_bug.py", line 216 in runTests
File "test_bug.py", line 226 in init
File "test_bug.py", line 246 in suiteClass.call(self)
File "test_bug.py", line 256 in _runTestsInProcesses
File "test_bug.py", line 266 in _runTestsInSubProcesses
File "test_bug.py", line 276 in _runTestsInThreads
File "test_bug

juud5qan

juud5qan1#

我能够重现tf v2.8的问题。谢谢!

d7v8vwbk

d7v8vwbk2#

这里有一个实际的用例吗?
我们可能在这里遇到了整数溢出。该形状的总元素数量超过了int64的最大值。

sqxo8psd

sqxo8psd3#

在大多数情况下,如果形状较小,此程序运行良好。您是否指的是此API的功能与int64最大值有关?

yftpprvb

yftpprvb4#

是的,Tensor中的元素总数必须小于 int64 的最大值。在尝试计算要添加的新元素数量时,它溢出为负数。我们不支持如此大的形状。

rm5edbpk

rm5edbpk5#

也许最好对溢出形状进行条件检查?

8zzbczxx

8zzbczxx6#

如果你喜欢,你可以为此提交一个PR。这并不仅限于clip操作符-所有Tensor都受到这个限制。

相关问题