SQL Server Why is the BlockTime variable not being recognized in PySpark on synapse?

7gyucuyw  于 2023-04-19  发布在  Spark
关注(0)|答案(1)|浏览(142)

I am trying to use PySpark on Azure synapse to run a SQL query in a tempview. All of my data has loaded in and the previous two queries are running fine. However, this last query, there are variables needing to be added for calculation: Stage, BlockTime, FuelPerGallon, and LoadFactor. I'm just not understanding what adjustment I need to make in order for the query to run.

This is the current code (note, in this environment DECLARE is not recognized)

%%sql

Set Stage=2500.0;
Set BlockTime = Stage/450.0 + .6;
Set FuelPerGallon = 3.07;
Set LoadFactor =.85;

With NarrowbodyMTOWShare as
(
    Select (Sum(T1_By_Year_Quarter_Carrier.Departures)-Sum(T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY.DEPARTURES))*380000 as WidebodyMTOW ,
    Sum(T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY.DEPARTURES) *183000 as NarrowBodyMTOW,
    (Sum(T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY.DEPARTURES) *183000)/((Sum(T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY.DEPARTURES) *183000)+(Sum(T1_By_Year_Quarter_Carrier.Departures)-Sum(T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY.DEPARTURES))*380000) as PercentNarrowbody,
    T1_By_Year_Quarter_Carrier.Year,
    T1_By_Year_Quarter_Carrier.Quarter,T1_By_Year_Quarter_Carrier.Carrier
    From T1_By_Year_Quarter_Carrier
    Left Join T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY ON
    T1_By_Year_Quarter_Carrier.Carrier=T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Carrier
    and T1_By_Year_Quarter_Carrier.Year=T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Year
    and T1_By_Year_Quarter_Carrier.Quarter = T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Quarter
    Group by T1_By_Year_Quarter_Carrier.Year,T1_By_Year_Quarter_Carrier.Quarter,T1_By_Year_Quarter_Carrier.Carrier
),
Interest AS
(
    Select Sum(InterestExpense) as Interest, Sum(InterestExpense)*Max(PercentNarrowbody) as NarrowbodyInterest,
    (Sum(AircraftTrafficServicing)-Sum(LandingFees))*MAX(PercentNarrowbody) as NarrowbodyAircraftTrafficServicing,
    Sum(LandingFees)*Max(PercentNarrowbody) as NarrowbodyLandingFees,
    (Sum(CommissionsCargo)/Sum(REVTONMILES)) as CommissionsCargo,
    (Sum(CommissionsPax)/Sum(RPM)) as CommissionsPax,
    (Sum(Food)/(Sum(ONBOARD_PAX)*(Sum(BLOCKHOURS)/Sum(Departures)))) as Food,
    (Sum(PassengerServicing)-Sum(Food))/(Sum(ONBOARD_PAX)*(Sum(BLOCKHOURS)/Sum(Departures))) as Inflight,
    (Sum(ReservationsandSales)-sum(CommissionsPax)-sum(CommissionsCargo)-sum(Advertsising))/Sum(ONBOARD_PAX) as ResAndSales,
    Sum(Advertsising)/Sum(ONBOARD_PAX) as Advertising,
    Sum(GeneralAdmin)/Sum(ASM) as GeneralAdmin,
    ((Sum(Maintenance) - Sum(Mx))*Max(PercentNarrowbody))/Sum(Departures) as GroundMX,
    P12_By_Year_Quarter_Carrier.Year,P12_By_Year_Quarter_Carrier.Quarter,P12_By_Year_Quarter_Carrier.Carrier
    From P12_By_Year_Quarter_Carrier
    Left Join NarrowbodyMTOWShare on NarrowbodyMTOWShare.Carrier=P12_By_Year_Quarter_Carrier.Carrier and NarrowbodyMTOWShare.Year=P12_By_Year_Quarter_Carrier.Year
    and P12_By_Year_Quarter_Carrier.Quarter=NarrowbodyMTOWShare.Quarter
    Left Join P6_BY_YEAR_QUARTER_CARRIER on P12_BY_YEAR_QUARTER_CARRIER.Carrier=P6_BY_YEAR_QUARTER_CARRIER.Carrier
    and P12_BY_YEAR_QUARTER_CARRIER.Year=P6_BY_YEAR_QUARTER_CARRIER.Year
    and P12_BY_YEAR_QUARTER_CARRIER.Quarter=P6_BY_YEAR_QUARTER_CARRIER.Quarter
    Left Join P52_BY_YEAR_QUARTER_CARRIER on P52_BY_YEAR_QUARTER_CARRIER.Carrier=P12_BY_YEAR_QUARTER_CARRIER.Carrier
    and P52_BY_YEAR_QUARTER_CARRIER.Year=P12_BY_YEAR_QUARTER_CARRIER.Year
    and P52_BY_YEAR_QUARTER_CARRIER.Quarter=P12_BY_YEAR_QUARTER_CARRIER.Quarter
    Left JOIN T1_By_Year_Quarter_Carrier on T1_By_Year_Quarter_Carrier.Carrier=P12_BY_YEAR_QUARTER_CARRIER.Carrier
    and T1_By_Year_Quarter_Carrier.Year=P12_BY_YEAR_QUARTER_CARRIER.Year
    and T1_By_Year_Quarter_Carrier.Quarter=P12_BY_YEAR_QUARTER_CARRIER.Quarter
    Group By P12_By_Year_Quarter_Carrier.Year,P12_By_Year_Quarter_Carrier.Quarter,P12_By_Year_Quarter_Carrier.Carrier
)

SELECT (FlightCrewCost/BLOCKHOURS)*BlockTime as PilotCost,
((MX*.6)/DEPARTURES)+((MX*.4)/BLOCKHOURS)*BlockTime as MXCost,
(Insurance/BLOCKHOURS)*BlockTime as InsuranceCost,
(OtherDirectCost/BLOCKHOURS)*BlockTime as OtherDirectCost,
(Gallons/BLOCKHOURS)*FuelPerGallon as FuelPerCost,
(((Rent+DepAmort+NarrowbodyInterest)*.5)/Departures)+ (((Rent+DepAmort+NarrowbodyInterest)*.5)/BlockHours)*BlockTime as OwnershipCost,
NarrowbodyAircraftTrafficServicing/DEPARTURES as AircraftAndTrafficServicingCost,
NarrowbodyLandingFees/Departures as LandingFeesCost,
CommissionsCargo*(NonPaxRTMPerDeparture+((Stage*(SeatsPerDeparture*LoadFactor)*35.0)/2000.0)) as CargoCommissionCost,
CommissionsPax*(Stage*(SeatsPerDeparture*LoadFactor)) as PaxCommissionCost,
Food * (SeatsPerDeparture*LoadFactor*BlockTime) as FoodCost,
Inflight * (SeatsPerDeparture*LoadFactor*BlockTime) as InflightCost,
GroundMX as GroundMXCost,
ResAndSales*(SeatsPerDeparture*LoadFactor) as ResAndSalesCost,
Advertising*(SeatsPerDeparture*LoadFactor) as AdvertisingCost,
GeneralAdmin*(SeatsPerDeparture*Stage) as GeneralAdminCost,
P52_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Year
,T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Quarter
,P52_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Carrier
,BlockTime as BlockTime,
Stage as StageLength,
SeatsPerDeparture
 FROM P52_BY_YEAR_QUARTER_CARRIER_NARROWBODY
 Left Join T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY on T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Carrier=P52_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Carrier and 
P52_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Year=T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Year and 
P52_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Quarter = T2_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Quarter
Left Join Interest ON
Interest.Year= P52_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Year and Interest.Quarter=P52_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Quarter
and Interest.Carrier=P52_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Carrier
Where P52_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Year=2022   and P52_BY_YEAR_QUARTER_CARRIER_NARROWBODY.Quarter=1

This is my current error Error

k4aesqcs

k4aesqcs1#

I tried the sample spark SQL code and got the same error.

%%sql
Set Stage=2500.0;
Set BlockTime = Stage/450.0  +  .6;
select BlockTime,Stage

  • As @lptr suggested, while using the variable in spark sql query, you need to refer the variable as ${<variable-name>}

Corrected code:

%%sql
Set Stage=2500.0;
Set BlockTime =  ${Stage}/450.0  +  .6;
select  ${Stage} as stage, ${BlockTime} as BlockTime

Output

stageBlockTime
25006.155556

相关问题