hadoop pig在每个子组中的百分比

ve7v8dk2  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(406)

我有一个文件如下
姓名分数
约翰aa
约翰aa
约翰aa
约翰bb
玛丽抄送
玛丽抄送
玛丽dd
我想输出他们的分数百分比为每个人的分数,所以它会像这样
约翰aa 75
约翰bb 25
玛丽cc 66.6
玛丽dd 33.3
john有3个aa和1个bb,所以aa%=75和bb%=25我想用hadoop pig来做,请帮忙,谢谢
-特洛伊

hgb9j2n6

hgb9j2n61#

你能试试这个吗?
输入:file.dat

john aa
john aa
john aa
john bb
mary cc
mary cc
mary dd

代码:

A = LOAD 'file.dat' USING PigStorage(' ') as (name:chararray,score:chararray);
N = CUBE A BY CUBE(name,score);      
N2 = FOREACH N GENERATE FLATTEN(group) AS (name,score), ((float)COUNT_STAR(cube)) As (totcnt:float); 
N3 = FILTER N2 BY name!='null';
N4 = GROUP N3 BY name; 
N5 = FOREACH N4 {                                                                        
 fil = order N3 BY score;                                                                 
 fil1 = LIMIT fil 1;                                                                      
 fil2 = FILTER N3 BY score!='null';                                                       
 generate FLATTEN(fil2) AS (name:chararray,score:chararray,indcount:float),FLATTEN(fil1.totcnt) as (totcnt:float);
 }
N6 = FOREACH N5 GENERATE name,score,(indcount/totcnt)*100;
DUMP N6;

输出:

(john,aa,75.0)
(john,bb,25.0)
(mary,cc,66.66667)
(mary,dd,33.333336)

相关问题