我刚开始使用d3rlpy进行离线RL训练，并使用了pytorch。所以我按照PYtorch doc的建议安装了cuda 1.16：pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116。我在之后安装了d3rlpy，并运行以下示例代码：

from d3rlpy.algos import BC,DDPG,CRR,PLAS,PLASWithPerturbation,TD3PlusBC,IQL
import d3rlpy
import numpy as np
import glob
import time

#models
continuous_models = {
                      "BehaviorCloning": BC,
                      "DeepDeterministicPolicyGradients": DDPG,
                      "CriticRegularizedRegression": CRR,
                      "PolicyLatentActionSpace": PLAS,
                      "PolicyLatentActionSpacePerturbation": PLASWithPerturbation,
                      "TwinDelayedPlusBehaviorCloning": TD3PlusBC,
                      "ImplicitQLearning": IQL,
                    }
#load dataset data_batch is created as a*.h5 file with d3rlpy
dataset = d3rlpy.dataset.MDPDataset.load(data_batch)
        
# preprocess
mean = np.mean(dataset.observations, axis=0, keepdims=True)
std = np.std(dataset.observations, axis=0, keepdims=True)
scaler = d3rlpy.preprocessing.StandardScaler(mean=mean, std=std)

# test models
for _model in continuous_models:
    the_model = continuous_models[_model](scaler = scaler)
    the_model.use_gpu = True
    the_model.build_with_dataset(dataset)

    the_model.fit(dataset = dataset.episodes,
                  n_steps_per_epoch = 10800, 
                  n_steps = 54000,
                  logdir = './logs', 
                  experiment_name = f"{_model}", 
                  tensorboard_dir = 'logs',
                  save_interval = 900, # we don't want to save intermediate parameters
                 )
    #save model
    the_timestamp = int(time.time())
    the_model.save_model(f"./models/{_model}/{_model}_{the_timestamp}.pt")

问题是，尽管设置了use_gpu =True，但没有一个模型实际上使用了GPU。通过pytotch的示例代码和测试torch.cuda.current_device()，我可以看到pytorch被正确设置并检测到gpu。你知道在哪里可以找到解决这个问题的方法吗？我不确定这是来自d3rlpy的bug，所以我会在github上创建一个问题：）