ApachePig—我想使用HadoopPig从xml输入中提取特定信息

w8f9ii69  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(382)

预期输出为: (Hadoop definitive guide,Tom white,24.90) .
我试过使用 Regex_Extract() 功能。但是,还没有运气。有人能帮帮我吗?
我的脚本的输入是:

<CATALOG>
<BOOK>
<TITLE>Hadoop DEFINITIVE GUIDE</TITLE>
<AUTHOR>TOM WHITE</AUTHOR>
<COUNTRY>US</COUNTRY>
<COMPANY>CLOUDERA</COMPANY>
<PRICE>24.90</PRICE>
<YEAR>2012</YEAR>
</BOOK>
<BOOK>
<TITLE>Programming Pig</TITLE>
<AUTHOR>Alan Gates</AUTHOR>
<COUNTRY>USA</COUNTRY>
<COMPANY>Horton Works</COMPANY>
<PRICE>30.90</PRICE>
<YEAR>2013</YEAR>
</BOOK>
</CATALOG>
qv7cva1a

qv7cva1a1#

你必须提取 <TITLE> , <AUTHOR> 以及 <PRICE> 分开,然后用 JOIN 接线员。
下面的脚本实现了:

-- Load input 
A = LOAD '/input.txt' USING PigStorage() AS (f1:chararray);

-- Extract <TITLE>
B1 = FOREACH A GENERATE REGEX_EXTRACT(f1, '<TITLE>(.*)</TITLE>', 1) AS (title:chararray);
C1 = FILTER B1 BY title is not null;
D1 = RANK C1;

-- Extract <AUTHOR>
B2 = FOREACH A GENERATE REGEX_EXTRACT(f1, '<AUTHOR>(.*)</AUTHOR>', 1) AS (author:chararray);
C2 = FILTER B2 BY author is not null;
D2 = RANK C2;

-- Extract <PRICE>
B3 = FOREACH A GENERATE REGEX_EXTRACT(f1, '<PRICE>(.*)</PRICE>', 1) AS (price:chararray);
C3 = FILTER B3 BY price is not null;
D3 = RANK C3;

-- Join 3 data sets
D = JOIN D1 BY $0, D2 BY $0, D3 By $0;

-- Eliminate the ranks
E = FOREACH D GENERATE $1 AS (title:chrarray), $3 AS (author:chararray), $5 AS (price:chararray)

dump E;

对于问题中提到的输入,我得到了以下输出:

(Hadoop DEFINITIVE GUIDE,TOM WHITE,24.90)
(Programming Pig,Alan Gates,30.90)

相关问题