我把这个问题作为后webhdfs restapi抛出文件未发现异常的延续
我有一个图像文件,我想通过webhdfsrestapi打开。
该文件存在于hdfs中,并且具有适当的权限
我可以 LISTSTATUS
然后得到答案:
curl -i“http://namenode:50070/webhdfs/v1/tmp/file.png?op=liststatus“
HTTP/1.1 200 OK
Date: Fri, 17 Jul 2020 22:47:29 GMT
Cache-Control: no-cache
Expires: Fri, 17 Jul 2020 22:47:29 GMT
Date: Fri, 17 Jul 2020 22:47:29 GMT
Pragma: no-cache
X-FRAME-OPTIONS: SAMEORIGIN
Content-Type: application/json
Transfer-Encoding: chunked
{"FileStatuses":{"FileStatus":[
{"accessTime":1594828591740,"blockSize":134217728,"childrenNum":0,"fileId":11393739,"group":"hdfs","length":104811,"modificationTime":1594828592000,"owner":"XXXX","pathSuffix":"XXXX","permission":"644","replication":3,"storagePolicy":0,"type":"FILE"}
]}}
Content-Type: application/octet-stream
Content-Length: 0
因此api可以正确读取元数据,但我无法打开该文件:
curl -i“http://namenode:50070/webhdfs/v1/tmp/file.png?op=open“
HTTP/1.1 307 Temporary Redirect
Date: Fri, 17 Jul 2020 22:23:17 GMT
Cache-Control: no-cache
Expires: Fri, 17 Jul 2020 22:23:17 GMT
Date: Fri, 17 Jul 2020 22:23:17 GMT
Pragma: no-cache
X-FRAME-OPTIONS: SAMEORIGIN
Location: http://datanode1:50075/webhdfs/v1/tmp/file.png?op=OPEN&namenoderpcaddress=namenode:8020&offset=0
Content-Type: application/octet-stream
Content-Length: 0
{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"Path is not a file: /tmp/file.png......
因此,根据webhdfs restapi抛出file not found exception,我可以看到请求从namenode传递到datanode1。datanode1在我的hosts文件中,我可以连接到它并从那里检查webhdfs的状态:
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
<final>true</final>
</property>
它是允许的,在namenode上也是如此。
我还查看了 /var/log/hadoop/hdfs/*.{log,out}
看看我是否能发现我 curl 时触发的错误,但似乎什么也没发生。我没有看到与我的文件或webhdfs查询相关的条目。我在namenode和datanode1上试过了。
作为最后的努力,我尝试将权限(不理想)从644(见第2点)增加到666
hdfs-dfs-chmod 666/tmp/file.png文件
curl -i“http://namenode:50070/webhdfs/v1/tmp/file.png?op=liststatus“
HTTP/1.1 403 Forbidden
Date: Fri, 17 Jul 2020 23:06:18 GMT
Cache-Control: no-cache
Expires: Fri, 17 Jul 2020 23:06:18 GMT
Date: Fri, 17 Jul 2020 23:06:18 GMT
Pragma: no-cache
X-FRAME-OPTIONS: SAMEORIGIN
Content-Type: application/json
Transfer-Encoding: chunked
{"RemoteException":{"exception":"AccessControlException","javaClassName":"org.apache.hadoop.security.AccessControlException","message":"Permission denied: user=XXXX, access=READ_EXECUTE, inode=\"/tmp/file.png\":XXXX:hdfs:drw-rw-rw-"}}
所以,它似乎做了开关,但不知何故,我得到了一个权限问题时,放松目前的权限,我没有得到之前?它不像我删除了x标志,它不在那里开始。access=read\u execute需要r和x吗?
现在我不明白为什么我可以用hdfs看到但不能读取这个文件。有人能帮我解决这个问题吗?
1条答案
按热度按时间roejwanj1#
仔细看看你上次的错误,
... inode=\"/tmp/file.png\":XXXX:hdfs:drw-rw-rw-"}
,似乎表明file.png
实际上是一个目录(前导d
符号)而不是文件。这与步骤3中的错误一致*..."message":"Path is not a file: /tmp/file.png....
你可以通过简单地$ hdfs dfs -ls /tmp/file.png/
.回到你的访问错误,你需要一个“执行”(
x
)列出目录中文件的权限。