org.apache.beam.sdk.io.FileSystems.matchNewResource()方法的使用及代码示例

x33g5p2x  于2022-01-19 转载在 其他  
字(7.9k)|赞(0)|评价(0)|浏览(118)

本文整理了Java中org.apache.beam.sdk.io.FileSystems.matchNewResource()方法的一些代码示例，展示了FileSystems.matchNewResource()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。FileSystems.matchNewResource()方法的具体详情如下：
包路径：org.apache.beam.sdk.io.FileSystems
类名称：FileSystems
方法名：matchNewResource

FileSystems.matchNewResource介绍

[英]Returns a new ResourceId that represents the named resource of a type corresponding to the resource type.

The supplied singleResourceSpec is expected to be in a proper format, including any necessary escaping, for the underlying FileSystem.

This function may throw an IllegalArgumentException if given an invalid argument, such as when the specified singleResourceSpec is not a valid resource name.
[中]返回新的ResourceId，该ResourceId表示与资源类型对应的类型的命名资源。
所提供的singleResourceSpec应为底层文件系统的正确格式，包括任何必要的转义。
如果给定无效参数，例如指定的singleResourceSpec不是有效的资源名称，则此函数可能会引发IllegalArgumentException。

代码示例

代码示例来源：origin: GoogleCloudPlatform/java-docs-samples

private static String getSchema(String schemaPath) throws IOException {
 ReadableByteChannel channel = FileSystems.open(FileSystems.matchNewResource(
   schemaPath, false));
 try (InputStream stream = Channels.newInputStream(channel)) {
  BufferedReader streamReader = new BufferedReader(new InputStreamReader(stream, "UTF-8"));
  StringBuilder dataBuilder = new StringBuilder();
  String line;
  while ((line = streamReader.readLine()) != null) {
   dataBuilder.append(line);
  }
  return dataBuilder.toString();
 }
}

代码示例来源：origin: GoogleCloudPlatform/java-docs-samples

public static String getSchema(String schemaPath) throws IOException {
 ReadableByteChannel chan = FileSystems.open(FileSystems.matchNewResource(
   schemaPath, false));
 try (InputStream stream = Channels.newInputStream(chan)) {
  BufferedReader streamReader = new BufferedReader(new InputStreamReader(stream, "UTF-8"));
  StringBuilder dataBuilder = new StringBuilder();
  String line;
  while ((line = streamReader.readLine()) != null) {
   dataBuilder.append(line);
  }
  return dataBuilder.toString();
 }
}

代码示例来源：origin: GoogleCloudPlatform/cloud-bigtable-client

@Override
 public ResourceId apply(String input) {
  return FileSystems.matchNewResource(input, true);
 }
}

代码示例来源：origin: GoogleCloudPlatform/cloud-bigtable-client

@Override
 public ResourceId apply(String input) {
  return FileSystems.matchNewResource(input, true);
 }
}

代码示例来源：origin: org.apache.beam/beam-sdks-java-io-google-cloud-platform

static void removeTemporaryFiles(Iterable<String> files) throws IOException {
  ImmutableList.Builder<ResourceId> fileResources = ImmutableList.builder();
  for (String file : files) {
   fileResources.add(FileSystems.matchNewResource(file, false /* isDirectory */));
  }
  FileSystems.delete(fileResources.build());
 }
}

代码示例来源：origin: spotify/dbeam

public static void writeToFile(String filename, ByteBuffer contents) throws IOException {
 ResourceId resourceId = FileSystems.matchNewResource(filename, false);
 try (WritableByteChannel out = FileSystems.create(resourceId, MimeTypes.TEXT)) {
  out.write(contents);
 }
}

代码示例来源：origin: com.spotify/scio-core

private static void copyToRemote(Path src, URI dst) throws IOException {
 ResourceId dstId = FileSystems.matchNewResource(dst.toString(), false);
 WritableByteChannel dstCh = FileSystems.create(dstId, MimeTypes.BINARY);
 FileChannel srcCh = FileChannel.open(src, StandardOpenOption.READ);
 long srcSize = srcCh.size();
 long copied = 0;
 do {
  copied += srcCh.transferTo(copied, srcSize - copied, dstCh);
 } while (copied < srcSize);
 dstCh.close();
 srcCh.close();
 Preconditions.checkState(copied == srcSize);
}

代码示例来源：origin: org.apache.beam/beam-sdks-java-io-google-cloud-platform

TableRowWriter(String basename) throws Exception {
 String uId = UUID.randomUUID().toString();
 resourceId = FileSystems.matchNewResource(basename + uId, false);
 LOG.info("Opening TableRowWriter to {}.", resourceId);
 channel = FileSystems.create(resourceId, MimeTypes.TEXT);
 out = new CountingOutputStream(Channels.newOutputStream(channel));
}

代码示例来源：origin: org.apache.beam/beam-sdks-java-core

@Override
public ResourceId decode(InputStream is) throws IOException {
 String spec = STRING_CODER.decode(is);
 boolean isDirectory = BOOL_CODER.decode(is);
 return FileSystems.matchNewResource(spec, isDirectory);
}

代码示例来源：origin: org.apache.beam/beam-sdks-java-core

private static void writeTextToFileSideEffect(String text, String filename) throws IOException {
  ResourceId rid = FileSystems.matchNewResource(filename, false);
  WritableByteChannel chan = FileSystems.create(rid, "text/plain");
  chan.write(ByteBuffer.wrap(text.getBytes(StandardCharsets.UTF_8)));
  chan.close();
 }
}

代码示例来源：origin: org.apache.beam/beam-sdks-java-io-google-cloud-platform

static String resolveTempLocation(
   String tempLocationDir, String bigQueryOperationName, String stepUuid) {
  return FileSystems.matchNewResource(tempLocationDir, true)
    .resolve(bigQueryOperationName, ResolveOptions.StandardResolveOptions.RESOLVE_DIRECTORY)
    .resolve(stepUuid, ResolveOptions.StandardResolveOptions.RESOLVE_DIRECTORY)
    .toString();
 }
}

代码示例来源：origin: org.apache.beam/beam-sdks-java-core

public static FileNaming relativeFileNaming(
  final ValueProvider<String> baseDirectory, final FileNaming innerNaming) {
 return (window, pane, numShards, shardIndex, compression) ->
   FileSystems.matchNewResource(baseDirectory.get(), true /* isDirectory */)
     .resolve(
       innerNaming.getFilename(window, pane, numShards, shardIndex, compression),
       RESOLVE_FILE)
     .toString();
}

代码示例来源：origin: org.apache.beam/beam-sdks-java-core

private ViaFileBasedSink(Write<DestinationT, UserT> spec) {
 super(
   ValueProvider.NestedValueProvider.of(
     spec.getTempDirectory(),
     input -> FileSystems.matchNewResource(input, true /* isDirectory */)),
   new DynamicDestinationsAdapter<>(spec),
   spec.getCompression());
 this.spec = spec;
}

代码示例来源：origin: org.apache.beam/beam-sdks-java-core

@Test(expected = IllegalArgumentException.class)
public void testInvalidSchemaMatchNewResource() {
 assertEquals("file", FileSystems.matchNewResource("invalidschema://tmp/f1", false));
 assertEquals("file", FileSystems.matchNewResource("c:/tmp/f1", false));
}

代码示例来源：origin: org.apache.beam/beam-sdks-java-core

@Test
public void testValidMatchNewResourceForLocalFileSystem() {
 assertEquals("file", FileSystems.matchNewResource("/tmp/f1", false).getScheme());
 assertEquals("file", FileSystems.matchNewResource("tmp/f1", false).getScheme());
 assertEquals("file", FileSystems.matchNewResource("c:\\tmp\\f1", false).getScheme());
}

代码示例来源：origin: org.apache.beam/beam-sdks-java-core

@Override
public FileResult<DestinationT> decode(InputStream inStream) throws IOException {
 String tempFilename = FILENAME_CODER.decode(inStream);
 BoundedWindow window = windowCoder.decode(inStream);
 PaneInfo paneInfo = PANE_INFO_CODER.decode(inStream);
 int shard = SHARD_CODER.decode(inStream);
 DestinationT destination = destinationCoder.decode(inStream);
 return new FileResult<>(
   FileSystems.matchNewResource(tempFilename, false /* isDirectory */),
   shard,
   window,
   paneInfo,
   destination);
}

代码示例来源：origin: org.apache.beam/beam-runners-flink_2.11

TestSink(String tmpFolder) {
 super(
   ValueProvider.StaticValueProvider.of(FileSystems.matchNewResource(tmpFolder, true)),
   DynamicFileDestinations.constant(FILENAME_POLICY, SerializableFunctions.identity()));
}

代码示例来源：origin: org.apache.beam/beam-sdks-java-core

private static MatchResult.Metadata metadata(Path path, int size) {
 return MatchResult.Metadata.builder()
   .setResourceId(FileSystems.matchNewResource(path.toString(), false /* isDirectory */))
   .setIsReadSeekEfficient(true)
   .setSizeBytes(size)
   .build();
}

代码示例来源：origin: org.apache.beam/beam-runners-core-construction-java

DummySink() {
 super(
   StaticValueProvider.of(FileSystems.matchNewResource("nowhere", false)),
   DynamicFileDestinations.constant(
     new DummyFilenamePolicy(), SerializableFunctions.constant(null)));
}

代码示例来源：origin: org.apache.beam/beam-runners-google-cloud-dataflow-java

@Test
public void testPackageUploadIsSkippedWithNonExistentResource() throws Exception {
 String nonExistentFile =
   FileSystems.matchNewResource(tmpFolder.getRoot().getPath(), true)
     .resolve("non-existent-file", StandardResolveOptions.RESOLVE_FILE)
     .toString();
 assertEquals(
   Collections.EMPTY_LIST,
   defaultPackageUtil.stageClasspathElements(
     ImmutableList.of(nonExistentFile), STAGING_PATH, createOptions));
}

内容来源于网络，如有侵权，请联系作者删除！

FileSystems

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com