我需要在hadoop文件系统中循环所有csv文件。我可以列出hdfs目录中的所有文件
> hadoop fs -ls /path/to/directory
Found 2 items
drwxr-xr-x - hadoop hadoop 2 2016-10-12 16:20 /path/to/directory/tmp
-rwxr-xr-x 3 hadoop hadoop 4691945927 2016-10-12 19:37 /path/to/directory/myfile.csv
并且可以使用
for filename in /path/to/another/directory/*.csv; do echo $filename; done
但是我怎样才能把两者结合起来呢?我试过了
for filename in `hadoop fs -ls /path/to/directory | grep csv`; do echo $filename; done
但这给了我一些无稽之谈
Found
2
items
drwxr-xr-x
hadoop
hadoop
2
2016-10-12
....
1条答案
按热度按时间5hcedyr01#
这应该管用