hive-sql(hadoop)版本的proc-transpose?

8oomwypt  于 2021-06-01  发布在  Hadoop
关注(0)|答案(2)|浏览(452)

我想知道sas配置单元sql(hadoop)中是否有proc transpose的版本?
否则,我可以看到另一种(冗长的)方法是创建许多单独的表,然后重新连接在一起,这是我宁愿避免的。
欢迎任何帮助!
要转换的示例表>打算将月份放在表的顶部,以便按月份拆分费率:

+------+-------+----------+----------+-------+
| YEAR | MONTH |   Geog   | Category | Rates |
+------+-------+----------+----------+-------+
| 2018 |     1 | National | X        |    32 |
| 2018 |     1 | National | Y        |    43 |
| 2018 |     1 | National | Z        |    47 |
| 2018 |     1 | Regional | X        |    52 |
| 2018 |     1 | Regional | Y        |    38 |
| 2018 |     1 | Regional | Z        |    65 |
| 2018 |     2 | National | X        |    63 |
| 2018 |     2 | National | Y        |    14 |
| 2018 |     2 | National | Z        |    34 |
| 2018 |     2 | Regional | X        |    90 |
| 2018 |     2 | Regional | Y        |    71 |
| 2018 |     2 | Regional | Z        |    69 |
+------+-------+----------+----------+-------+

样本输出:

+------+----------+----------+----+----+
| YEAR |   Geog   | Category | 1  | 2  |
+------+----------+----------+----+----+
| 2018 | National | X        | 32 | 63 |
| 2018 | National | Y        | 43 | 14 |
| 2018 | National | Z        | 47 | 34 |
| 2018 | Regional | X        | 52 | 90 |
| 2018 | Regional | Y        | 38 | 71 |
| 2018 | Regional | Z        | 65 | 69 |
+------+----------+----------+----+----+
vlurs2pr

vlurs2pr1#

如果示例数据集是真实数据集的代表,那么可以使用如下所示的简单内部联接。年地理和分类作出独特的组合下面的代码应该工作。

select a.YEAR ,   
    a.Geog ,  
    a.Category , 
    a.Rates ,
    a.month as  month_1, 
    b.month as  month_2 
from have a
inner join
 have b
 on a.year = b.year
 and  a.Geog = b.Geog 
and  a.Category = b.category
where a.month ne b.month;
wwtsj6pe

wwtsj6pe2#

用于转置(或透视)的典型wallpapersql技术是在折叠子查询的组聚合查询中使用group+transform-to-pivot case语句子查询。该组表示单个结果透视行。
例如,您的组是year、geog、category和 min 用于折叠:

proc sql;
  create view want_pivot as
  select year, geog, category
  , min(rate_m1) as rate_m1
  , min(rate_m2) as rate_m2
  from
  ( select
    year, geog, category
    , case when month=1 then rates end as rate_m1
    , case when month=2 then rates end as rate_m2
    from have
  )
  group by year, geog, category
  ;

这里是相同的概念,更一般地说,数据在组内的细节级别重复,并且 mean 是用来折叠重复的。

data have;
input id name $ value;
datalines;
1 a 1
1 a 2 
1 a 3
1 b 2
1 c 3
2 a 2
2 d 4
2 b 5
3 e 1
run;

proc sql;
  create view have_pivot as 
  select
  id
  , mean(a) as a
  , mean(b) as b
  , mean(c) as c
  , mean(d) as d
  , mean(e) as e
  from
  (
    select
      id
      , case when name='a' then value end as a
      , case when name='b' then value end as b
      , case when name='c' then value end as c
      , case when name='d' then value end as d
      , case when name='e' then value end as e
    from have
  )
  group by id
;
quit;

当列名事先未知时,需要编写一个代码生成器来传递所有数据以确定名称值,并编写墙纸查询,该查询将对返回枢轴的数据执行第二次传递。
此外,许多现代数据库都有一个pivot子句,可以通过pass-through来利用它。
hadoop mania post“在hive中转置/透视表”展示了 collect_list 以及 map 以类似的壁纸方式:

select b.id, b.code, concat_ws('',b.p) as p, concat_ws('',b.q) as q, concat_ws('',b.r) as r, concat_ws('',b.t) as t from
 (select id, code,
 collect_list(a.group_map['p']) as p,
 collect_list(a.group_map['q']) as q,
 collect_list(a.group_map['r']) as r,
 collect_list(a.group_map['t']) as t
 from ( select
  id, code,
  map(key,value) as group_map
  from test_sample
 ) a group by a.id, a.code) b;

相关问题