Azure ml Studio对象检测jupyter:访问用于训练模型的mltable时出错

xytpbqjk  于 2023-10-22  发布在  其他
关注(0)|答案(1)|浏览(113)

我是Azure机器学习的新手,我还在学习。有一个名为annotations.jsonl的文件保存在容器“devplaifurnasdlfinal”的子文件夹“image-frames/dados_treinamento”中。容器“devplaifurnasdlfinal”中的子文件夹“image-frames”在azure machine learning studio中以“image_frames”的名称注册为目录。创建了用于训练对象检测模型的代码,并且必须将训练数据传递到名称为“my_training_data_input”的代码。下面是代码:

from azure.identity import DefaultAzureCredential
from azure.ai.ml import MLClient
from azure.ai.ml import automl
from azureml.core import Workspace
from azure.ai.ml.constants import AssetTypes
from azure.ai.ml import Input
from azureml.core.compute import AmlCompute, ComputeTarget
from azureml.exceptions import ComputeTargetException

# Load the workspace from the saved config file
ws = Workspace.from_config()

# Print the details of the workspace
print(f"Workspace name: {ws.name}")
print(f"Azure region: {ws.location}")
print(f"Resource group: {ws.resource_group}")
print(f"Subscription ID: {ws.subscription_id}")

# Define your workspace, datastore, and data paths
datastore_name = 'image_frames'
subscription_id = 'xxxx'
resource_group = 'yyyy'
workspace_name = 'zzzz'

credential = DefaultAzureCredential()
ml_client = None

try:
    ml_client = MLClient.from_config(credential)

except Exception as ex:
    print(ex)
    subscription_id = subscription_id
    resource_group = resource_group
    workspace = workspace_name

    ml_client = MLClient(credential, subscription_id, resource_group, workspace)

training_data_path = 'annotation_jsonl_path_from_container'
# the path is in the format https://devplaifurnasfinal.blob.core.windows.net/ima...

my_training_data_input = Input(
    type=AssetTypes.URI_FILE,
    path=training_data_path)

compute_name = "devplaipipe-run"  # Name of the compute target to retrieve

try:
    # Attempt to retrieve the existing compute target by name
    existing_compute_target = ComputeTarget(workspace=ws, name=compute_name)
    print("Found existing compute target:")
    print(f"Name: {existing_compute_target.name}")
    print(f"Type: {existing_compute_target.type}")

except ComputeTargetException:
    print("Creating a new compute target...")
    # Configure your compute target here
    provisioning_config = AmlCompute.provisioning_configuration(
        vm_size="Standard_NC6",
        idle_seconds_before_scaledown=120,
        min_nodes=0,
        max_nodes=4,)
    
    new_compute_target = ComputeTarget.create(ws, compute_name, provisioning_config)
    new_compute_target.wait_for_completion(show_output=True)
    print("New compute target created:")
    print(f"Name: {new_compute_target.name}")
    print(f"Type: {new_compute_target.type}")

    exp_name = "test-vscode-fine-tunning-objectdetection"

# Create the AutoML job with the related factory-function.
image_object_detection_job = automl.image_object_detection(
    compute=compute_name,
    experiment_name=exp_name,
    training_data=my_training_data_input,
    validation_data_size=0.25,
    target_column_name="label",
    primary_metric="mean_average_precision",
    tags={"my_custom_tag": "My custom value"},)

image_object_detection_job.set_limits(max_trials=5, max_concurrent_trials=2,)

# Submit the AutoML job
returned_job = ml_client.jobs.create_or_update(
    image_object_detection_job)  # submit the job to the backend

print(f"Created job: {returned_job}")

ml_client.jobs.stream(returned_job.name)

annotations.jsonl位于azure对象检测注解模式中,可以从容器和Windows Boot 访问此文件。两种方法我都试过了,总是出现错误。代码本身看起来是正确的,打印信息可以看到正确的工作区,gpu集群等。这个错误说由于权限访问,它不能访问mltable,但我不知道是不是这样,如果是这样,我如何解决这个权限问题?错误如下:

MLTable input is invalid. UserErrorException:
    Message: Encountered user error while fetching data from Dataset. Error: UserErrorException:
    Message: Failed to download mltable yaml with error: Getting permission error when trying to access MLTable,please make sure proper access is configured on storage: Dataflow visit error: ExecutionError(StreamError(PermissionDenied(None)))
    VisitError(ExecutionError(StreamError(PermissionDenied(None))))
=> Failed with execution error: error in streaming from input data sources
    ExecutionError(StreamError(PermissionDenied(None)))
    InnerException None
    ErrorResponse 
{
    "error": {
        "code": "UserError",
        "message": "Failed to download mltable yaml with error: Getting permission error when trying to access MLTable,please make sure proper access is configured on storage: Dataflow visit error: ExecutionError(StreamError(PermissionDenied(None)))\n\tVisitError(ExecutionError(StreamError(PermissionDenied(None))))\n=> Failed with execution error: error in streaming from input data sources\n\tExecutionError(StreamError(PermissionDenied(None)))"
    }
}
    InnerException UserErrorException:
    Message: Failed to download mltable yaml with error: Getting permission error when trying to access MLTable,please make sure proper access is configured on storage: Dataflow visit error: ExecutionError(StreamError(PermissionDenied(None)))
    VisitError(ExecutionError(StreamError(PermissionDenied(None))))
=> Failed with execution error: error in streaming from input data sources
    ExecutionError(StreamError(PermissionDenied(None)))
    InnerException None
    ErrorResponse 
{
    "error": {
        "code": "UserError",
        "message": "Failed to download mltable yaml with error: Getting permission error when trying to access MLTable,please make sure proper access is configured on storage: Dataflow visit error: ExecutionError(StreamError(PermissionDenied(None)))\n\tVisitError(ExecutionError(StreamError(PermissionDenied(None))))\n=> Failed with execution error: error in streaming from input data sources\n\tExecutionError(StreamError(PermissionDenied(None)))"
    }
}
    ErrorResponse 
{
    "error": {
        "code": "UserError",
        "message": "Encountered user error while fetching data from Dataset. Error: UserErrorException:\n\tMessage: Failed to download mltable yaml with error: Getting permission error when trying to access MLTable,please make sure proper access is configured on storage: Dataflow visit error: ExecutionError(StreamError(PermissionDenied(None)))\n\tVisitError(ExecutionError(StreamError(PermissionDenied(None))))\n=> Failed with execution error: error in streaming from input data sources\n\tExecutionError(StreamError(PermissionDenied(None)))\n\tInnerException None\n\tErrorResponse \n{\n    \"error\": {\n        \"code\": \"UserError\",\n        \"message\": \"Failed to download mltable yaml with error: Getting permission error when trying to access MLTable,please make sure proper access is configured on stora

下面是作业的3个图像,输入训练数据,尝试访问输入数据,最后尝试探索输入数据时出现错误:

这真的是一个许可的问题,我需要在哪里(或什么)验证,以纠正这一点?

vh0rcniy

vh0rcniy1#

我花了几天的时间与这个完全相同的错误,但在我的情况下,我使用的是Azure API,有我的应用程序权限设置为“所有者”无处不在,所以我期待完全访问blob存储,什么是失踪,给我这个错误是“允许公共访问”配置标志,这是设置为禁用.所以去我的Blob存储资源,配置,然后设置该标志为启用允许我摆脱这个错误,this后去更详细的信息

相关问题