我正在编写一个python脚本,它使用beeline cli解析元数据,并输出一个文件供我解析。我很难从python->beeline脚本传递正确的转义字符序列。
下面是我的代码片段
# !/usr/bin/python
import commands
import subprocess
import sys
hive_cmd = "beeline -u \"jdbc:hive2://$(hostname -f):10000/;principal=hive/_HOST@some_company.COM\" \
--silent=true\
--outputformat=csv2\
--showHeader=false\
--force=true\
--showWarnings=false\
-e 'USE some_database; "
if __name__ == '__main__':
describe_table_cmd = ""
f = open('run_dml.sql', 'w+')
tables_list_cmd = hive_cmd + "SHOW TABLES;'"
status, tables_list = commands.getstatusoutput(tables_list_cmd)
for table_name in tables_list.splitlines():
if not 'HotSpot' in table_name:
describe_table_cmd += "\!sh echo {0};\ndescribe {0};\n".format(str(table_name))
if table_name == "2table":
break
print(describe_table_cmd)
status2, columns_list = commands.getstatusoutput(hive_cmd + describe_table_cmd + "'")
for line in columns_list.splitlines():
f.write("{0}\n".format(line))
f.close()
这是控制台输出:
[username@server ~]$ ./r.py
\!sh echo 1table;
describe 1table;
\!sh echo 2table;
describe 2table;
[username@server ~]$
这就是run\u dml.sql中的内容
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
Error: Error while compiling statement: FAILED: ParseException line 1:1 cannot recognize input near '!' 'sh' 'echo' (state=42000,code=40000)
显然,我正在尝试在shell上运行echo语句以获取表输出。如果有更好的方法动态获取配置单元元数据(表名及其列),我洗耳恭听。在那之前,这是令人不快的代码。
describe_table_cmd += "\!sh echo {0};\ndescribe {0};\n".format(str(table_name))
暂无答案!
目前还没有任何答案,快来回答吧!