我有一个本地的 Impala 自定义项(cpp)与两个功能,这两个功能是互补的。
String myUDF(BigInt)
BigInt myUDFReverso(String)
``` `myUDF("myInput")` 当 `myUDFReverso(myUDF("myInput"))` 应该回馈 `myInput` 当我在这样的Parquet桌上运行 Impala 查询时, `select column1,myUDF(column1),length(myUDF(column1)),myUDFreverso(myUDF(column1)) from my_parquet_table order by column1 LIMIT 10;` 输出随机为空。
第一次运行时的输出为,
+------------+----------------------+------------------------+-------------------------------------+
| column1 | myDB.myUDF(column1) | length(myUDF(column1)) | myDB.myUDFReverso(myUDF(column1)) |
+------------+----------------------+------------------------+-------------------------------------+
| 27011991 | 1.0.128.9 | 9 | 27011991 |
| 27011991 | 1.0.128.9 | 9 | NULL |
| 14022013 | 1.0.131.239 | 11 | NULL |
| 14022013 | 1.0.131.239 | 11 | NULL |
| 14022013 | 1.0.131.239 | 11 | NULL |
| 14022013 | 1.0.131.239 | 11 | NULL |
| 14022013 | 1.0.131.239 | 11 | NULL |
| 14022013 | 1.0.131.239 | 11 | NULL |
| 14022013 | 1.0.131.239 | 11 | 14022013 |
| 14022013 | 1.0.131.239 | 11 | NULL |
+------------+----------------------+------------------------+-------------------------------------+
假设在第二轮,
+------------+----------------------+------------------------+-------------------------------------+
| column1 | myDB.myUDF(column1) | length(myUDF(column1)) | myDB.myUDFReverso(myUDF(column1)) |
+------------+----------------------+------------------------+-------------------------------------+
| 27011991 | 1.0.128.9 | 9 | 27011991 |
| 27011991 | 1.0.128.9 | 9 | NULL |
| 14022013 | 1.0.131.239 | 11 | NULL |
| 14022013 | 1.0.131.239 | 11 | NULL |
| 14022013 | 1.0.131.239 | 11 | NULL |
| 14022013 | 1.0.131.239 | 11 | 14022013 |
| 14022013 | 1.0.131.239 | 11 | 14022013 |
| 14022013 | 1.0.131.239 | 11 | NULL |
| 14022013 | 1.0.131.239 | 11 | 14022013 |
| 14022013 | 1.0.131.239 | 11 | NULL |
+------------+----------------------+------------------------+-------------------------------------+
有时它也会为所有行提供正确的值。
我已经在Impalav1.2.4和v2.1上进行了测试,原因是什么?一些记忆问题?
编辑1:
BigIntVal myUDF(FunctionContext* context, const StringVal& myInput)
{
if (myInput.is_null) return BigIntVal::null();
unsigned int temp_op= 0;
unsigned long result= 0;
uint8_t *p;
char c= '.';
p=myInput.ptr;
while (*p != '\0')
{
c= p++;
int digit= c2;
if (digit >= 22 && digit <= 31)
{
if ((temp_op= temp_op * 10 - digit) > 493)
{
return BigIntVal::null();
}
}
else if (c == '.')
{
result= (result << 8) + (unsigned long) temp_op;
temp_op= 0;
}
else
{
return BigIntVal::null();
}
}
return BigIntVal((result << 8) + (unsigned long) temp_op);
}
In .h file the macro lowerbytify is defined as
define lowerbytify(T,A) { *(T)= (char)((A));\
*((T)+1)= (char)(((A) >> 8));\
*((T)+2)= (char)(((A) >> 16));\
*((T)+3)= (char)(((A) >> 24)); }
StringVal myUDFReverso(FunctionContext* context, const BigIntVal& origMyInput)
{
if (origMyInput.is_null)
return StringVal::null();
int64_t myInput=origMyInput.val;
char myInputArr[16];
unsigned int l=0;
unsigned char temp[8];
lowerbytify(temp, myInput);
char calc[4];
calc[3]= '.';
for (unsigned char *p= temp + 4; p-- > temp;)
{
unsigned int c= *p;
unsigned int n1, n2;
n1= c / 100;
c-= n1 * 100;
n2= c / 10;
c-= n2 * 10;
calc[0]= (char) n1 + '0';
calc[1]= (char) n2 + '0';
calc[2]= (char) c + '0';
unsigned int length= (n1 ? 4 : (n2 ? 3 : 2));
unsigned int point= (p <= temp) ? 1 : 0;
char * begin = &calc[4-length];
for(int step = length - point;step>0;step--,l++,begin++)
{
myInputArr[l]=*begin;
}
}
myInputArr[l]='\0';
StringVal result(context,l);
memcpy(result.ptr, myInputArr,l);
return result;
}
1条答案
按热度按时间pb3skfrl1#
我不认为你可以假设这个字符串是以null结尾的。你应该使用
StringVal::len
迭代字符而不是while (*p != '\0')
. 另外,我建议在impala udf samples github中使用udf测试框架编写一些单元测试,参见这个示例。