使用java将文件移动到hdfs

8ehkhllq  于 2021-06-02  发布在  Hadoop
关注(0)|答案(2)|浏览(393)

如何使用java在hdfs中执行hadoop put文件?有可能吗?
使用此语句:

public abstract boolean rename(Path src, Path dst) throws IOException

?
谢谢!

dnph8jn4

dnph8jn41#

试试这个:

//Source file in the local file system
String localSrc = args[0];
//Destination file in HDFS
String dst = args[1];

//Input stream for the file in local file system to be written to HDFS
InputStream in = new BufferedInputStream(new FileInputStream(localSrc));

//Get configuration of Hadoop system
Configuration conf = new Configuration();
System.out.println("Connecting to -- "+conf.get("fs.defaultFS"));

//Destination file in HDFS
FileSystem fs = FileSystem.get(URI.create(dst), conf);
OutputStream out = fs.create(new Path(dst));

//Copy file from local to HDFS
IOUtils.copyBytes(in, out, 4096, true);
2skhul33

2skhul332#

你应该可以使用 copyFromLocalFile :

Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
Path localPath = new Path("path/to/local/file");
Path hdfsPath = new Path("/path/in/hdfs");
fs.copyFromLocalFile(localPath, hdfsPath);

相关问题