apache清管器中袋的动态柱

js5cn81o  于 2021-06-21  发布在  Pig
关注(0)|答案(2)|浏览(302)

我是新来的ApachePig。是否可以创建带有动态列的包?下面是脚本示例。

A = LOAD 'student' USING PigStorage() AS (col1:chararray, col2:chararray, col3:chararray ....... );
 B = FOREACH A GENERATE col1, col3; OR
 B = FOREACH A GENERATE col2, col3, col4;

简而言之,获取列列表并创建包。可能吗?怎么可能?这里我需要的是获取动态列的方法。例如,有人将我的脚本定义为在params='col1,col2,col4'上运行,那么我的脚本应该能够解析这个字符串并使用它获得所需的列。

vpfxa7rd

vpfxa7rd1#

A = LOAD 'student' using PigStorage(',');

by this relation A contains all columns which has student file.You can access these columns by using sequence number like $0,$1,$2
$0 --> First column
$1 --> Second column
$2 --> third column and so on
pdkcd3nj

pdkcd3nj2#

我可以建议在脚本中使用宏或参数变量,这里的宏就是一个例子

define dividend_analysis (daily, year, daily_symbol, daily_open, daily_close) 
returns analyzed { 
divs = load '/user/data/NYSE_dividends'as (exchange:chararray,     symbol:chararray, date:chararray, dividends:float); 
divisthisyear = filter divs by date matches '.*$year.*';
dailythisyear = filter $daily by date matches '.*$year.*'; 
jnd = join divisthisyear by symbol, dailythisyear by $daily_symbol;
$analyzed = foreach jnd generate $1 ,$daily_close - $daily_open; };

daily = load '/user/data/NYSE_daily'as (exchange:chararray, symbol:chararray,     date:chararray, open:float, high:float, low:float, close:float, volume:int,     adj_close:float); 
results = dividend_analysis(daily, '2009', 'symbol', 'open', 'close');

相关问题