我正在尝试从hdfs加载两个文件到pig。当我加入司机关系和卡车关系后,我想数一数。如何计算相关行数?我试过这个,但它给我的计数与组没有一个计数:
truck_temp = FOREACH (GROUP truck_join BY drivers_info::driverId) { GENERATE group, COUNT(truck_join); };
drivers_load = LOAD '/Pig-Practice/drivers.csv' USING PigStorage(',') AS (driverId:int,name:chararray,ssn:biginteger,location:chararray,certified:chararray,wageplan:chararray);
drivers_info = FOREACH ( GROUP drivers_load BY (driverId,name)) GENERATE group.driverId,group.name;
event_load = LOAD '/Pig-Practice/truck_event_text_partition.csv' USING PigStorage(',') AS (driverId:int, truckId:int, eventTime:chararray,
eventType:chararray, longitude:double, latitude:double,
eventKey:chararray, correlationId:long, driverName:chararray,
routeId:long,routeName:chararray,eventDate:chararray);
truck_events1 = FILTER event_load BY $0 >1;
truck_events2 = FOREACH (GROUP truck_events1 BY (driverId,driverName,routeId,routeName) ) GENERATE group.driverId,group.driverName,group.routeId,group.routeName;
truck_join = JOIN drivers_info BY driverId, truck_events2 BY driverId;
1条答案
按热度按时间jjjwad0x1#
为了获得一个连接后的总计数,您需要对所有连接进行分组。
count要求前面的group all语句用于全局计数,group by语句用于组计数。引用:count