.net 尝试从Azure Blob存储读取Parquet数据时出现异常(使用ChoETL)

6l7fqoea  于 2023-01-06  发布在  .NET
关注(0)|答案(1)|浏览(123)

我目前使用的ChoETL库读取 parquet 数据,这是代码:

BlobServiceClient blobServiceClient = new BlobServiceClient(azureStorage);
            BlobContainerClient container = blobServiceClient.GetBlobContainerClient(contenedor);
            var blobs = container.GetBlobs().Where(x => x.Name.Contains(".parquet"));

            try
            {
                foreach (var item in blobs)
                {
                    var blob = container.GetBlobClient(item.Name);
                    await blob.OpenReadAsync();
//Here i'm trying to read the parquet file, as is shown in the official documentation https://github.com/Cinchoo/ChoETL/wiki/QuickParquetLoad
                    foreach (dynamic e in new ChoParquetReader(outStream))
                    {
                        Console.WriteLine("Id: " + e.Id + " FormNumber: " + e.FormNumber);
                    }

                }
            }
            catch (Exception ex)
            {

                throw ex;
            }

尝试执行它时,会在此行中引发错误:

foreach (dynamic e in new ChoParquetReader(outStream))
                    {
                        Console.WriteLine("Id: " + e.Id + " FormNumber: " + e.FormNumber);
                    }

有什么办法吗?我试过parquet.net,但我不喜欢

vptzau2j

vptzau2j1#

我找不到outStream在代码中的定义位置,但我认为这就是问题所在,您需要使用blob.OpenReadAsync()提供的Stream

BlobServiceClient blobServiceClient = new BlobServiceClient(azureStorage);
BlobContainerClient container = blobServiceClient.GetBlobContainerClient(contenedor);
var blobs = container.GetBlobs().Where(x => x.Name.Contains(".parquet"));

try
{
    foreach (var item in blobs)
    {
        var blob = container.GetBlobClient(item.Name);
        using var stream = await blob.OpenReadAsync();
        //Here i'm trying to read the parquet file, as is shown in the official documentation https://github.com/Cinchoo/ChoETL/wiki/QuickParquetLoad
        foreach (dynamic e in new ChoParquetReader(stream))
        {
            Console.WriteLine("Id: " + e.Id + " FormNumber: " + e.FormNumber);
        }

    }
}
catch (Exception ex)
{

    throw ex;
}

相关问题