不区分大小写的pig拉丁搜索

7gyucuyw  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(344)

这里是Pig拉丁语的初学者。我试图计算输入文件中多个字符串的出现次数。
现在搜索必须不区分大小写。我知道有一个 LOWER pig中的内置函数,但如何使用它?
例如(输入文件):

28-Oct-13,7:00PM,Viraj,New to hadoop ! Eager to learn.
31-Dec-14,3:00PM,‏Vanguard,Designers, Developers, Doers, don't miss this upcoming San Francisco Hadoop

我需要数一数 hadoop, dec, learn, python ```
hadoop 2
dec 1
learn 1
python 0

如何用Pig拉丁语进行搜索?
谢谢您。
mcdcgff0

mcdcgff01#

你能试试这个吗?
输入

28-Oct-13,7:00PM,Viraj,New to hadoop ! Eager to learn.
31-Dec-14,3:00PM,?Vanguard,Designers, Developers, Doers, don't miss this upcoming San Francisco Hadoop

Pig手稿:

A = LOAD 'input' AS (line:chararray);
B = FOREACH A GENERATE  FLATTEN(TOKENIZE(LOWER(line))) as word;
C = FOREACH B GENERATE ((word matches '.*hadoop.*'? 1:0)) as t1,((word matches '.*dec.*'?1:0)) as t2,((word matches '.*learn.*'?1:0)) as t3,((word matches '.*python.*'?1:0)) as t4;
D = GROUP C ALL;
E  = FOREACH D GENERATE FLATTEN(TOBAG(CONCAT('hadoop',' ',(chararray)SUM(C.t1)),CONCAT('dec',' ',(chararray)SUM(C.t2)),CONCAT('learn',' ',(chararray)SUM(C.t3)),CONCAT('python',' ',(chararray)SUM(C.t4))));
DUMP E;

输出:

(hadoop 2)
(dec 1)
(learn 1)
(python 0)

相关问题