陈明威:
if is_classify:
pyreader = fluid.layers.py_reader(
capacity=50,
shapes=[[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1,args.paragram_conut,args.paragram_max_len, 1],
[-1, 1], [-1, 1]],
dtypes=[ 'int64', 'int64', 'int64', 'int64', 'float32', 'int64', 'int64', 'int64'
],
lod_levels=[0, 0, 0, 0, 0, 1, 0, 0],
name=task_name + "_" + pyreader_name,
use_double_buffer=True)

陈明威:
这个是官方提供的代码，我就加了 [-1,args.paragram_conut,args.paragram_max_len, 1]倒数第三个

陈明威:
(src_ids, sent_ids, pos_ids, task_ids, input_mask, paragraph, labels,
qids) = fluid.layers.read_file(pyreader)

陈明威:
数据处理部分是def _pad_batch_records(self, batch_records,paragram_conut):
"""change data type to model"""
batch_token_ids = [record.token_ids for record in batch_records]
batch_text_type_ids = [record.text_type_ids for record in batch_records]
batch_position_ids = [record.position_ids for record in batch_records]
batch_labels = [record.label_id for record in batch_records]
batch_contents_ids = [record.contens_ids for record in batch_records]
if self.is_classify:
batch_labels = np.array(batch_labels).astype("int64").reshape([-1, 1])
elif self.is_regression:
batch_labels = np.array(batch_labels).astype("float32").reshape([-1, 1])

if batch_records[0].qid or batch_records[0].qid == 0:
    batch_qids = [record.qid for record in batch_records]
    batch_qids = np.array(batch_qids).astype("int64").reshape([-1, 1])
else:
    batch_qids = np.array([]).astype("int64").reshape([-1, 1])

# padding

padded_token_ids, input_mask = pad_batch_data(
    batch_token_ids, pad_idx=self.pad_id, return_input_mask=True)
padded_text_type_ids = pad_batch_data(
    batch_text_type_ids, pad_idx=self.pad_id)
padded_position_ids = pad_batch_data(
    batch_position_ids, pad_idx=self.pad_id)
padded_contents_ids = pad_batch_content_data(
    batch_contents_ids,paragram_conut=paragram_conut, pad_idx=self.pad_id)
padded_task_ids = np.ones_like(padded_token_ids, dtype="int64") * self.task_id

# print( np.max(padded_contents_ids), np.min(padded_contents_ids),'111111111111111111111111')

if padded_contents_ids.shape[0]<16:
    print(padded_contents_ids.shape,'111111111111111111111111111111111')

# print(padded_token_ids.shape)

# if np.max(padded_contents_ids)>500 or np.min(padded_contents_ids)<0:

# 

# print('11111111111111111111111111111111111111111111111')

label_tensor = fluid.LoDTensor()
label_tensor.set(padded_contents_ids, fluid.CPUPlace())
return_list = [        padded_token_ids, padded_text_type_ids, pa

陈明威:
其中padded_contents_ids = pad_batch_content_data(
batch_contents_ids,paragram_conut=paragram_conut, pad_idx=self.pad_id)

陈明威:
是我添加的，padded_contents_ids 在这个地方最大值也就476，数值类型都是int64

陈明威:
读取用的是train_pyreader.decorate_tensor_provider(train_data_generator)

陈明威:
然后最后读出来的数据竟然id变得很大

陈明威:
Exception: /paddle/paddle/fluid/operators/lookup_table_op.cu:36 Assertion id < N failed (received id: 4140473109978529412).
Exception: /paddle/paddle/fluid/operators/lookup_table_op.cu:36 Assertion id < N failed (received id: 4140473109978529412).
Exception: /paddle/paddle/fluid/operators/lookup_table_op.cu:36 Assertion id < N failed (received id: 4140473109978529412).
Exception: /paddle/paddle/fluid/operators/lookup_table_op.cu:36 Assertion id < N failed (received id: 4140473109978529412).
Exception: /paddle/paddle/fluid/operators/lookup_table_op.cu:36 Assertion id < N failed (received id: 4140473109978529412).
Exception: /paddle/paddle/fluid/operators/lookup_table_op.cu:36 Assertion id < N failed (received id: 4140473109978529412).
Exception: /paddle/paddle/fluid/operators/lookup_table_op.cu:36 Assertion id < N failed (received id: 4140473109978529412).
Exception: /paddle/paddle/fluid/operators/lookup_table_op.cu:36 Assertion id < N failed (received id: 4140473109978529412).
Exception: /paddle/paddle/fluid/operators/lookup_table_op.cu:36 Assertion id < N failed (received id: 4140473109978529412).
Exception: /paddle/paddle/fluid/operators/lookup_table_op.cu:36 Assertion id < N failed (received id: 4140473109978529412).
Exception: /paddle/paddle/fluid/operators/lookup_table_op.cu:36 Assertion id < N failed (received id: 4140473109978529412).