org.apache.parquet.io.api.Binary.fromByteArray()方法的使用及代码示例

x33g5p2x  于2022-01-16 转载在 其他  
字(10.4k)|赞(0)|评价(0)|浏览(146)

本文整理了Java中org.apache.parquet.io.api.Binary.fromByteArray()方法的一些代码示例，展示了Binary.fromByteArray()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Binary.fromByteArray()方法的具体详情如下：
包路径：org.apache.parquet.io.api.Binary
类名称：Binary
方法名：fromByteArray

Binary.fromByteArray介绍

暂无

代码示例

代码示例来源：origin: apache/hive

  1. @Override
  2.  public void write(Object value) {
  3.   byte[] vBinary = inspector.getPrimitiveJavaObject(value);
  4.   recordConsumer.addBinary(Binary.fromByteArray(vBinary));
  5.  }
  6. }

代码示例来源：origin: apache/hive

  1. private Binary decimalToBinary(final HiveDecimal hiveDecimal, final DecimalTypeInfo decimalTypeInfo) {
  2.   int prec = decimalTypeInfo.precision();
  3.   int scale = decimalTypeInfo.scale();
  4.   byte[] decimalBytes = hiveDecimal.bigIntegerBytesScaled(scale);
  5.   // Estimated number of bytes needed.
  6.   int precToBytes = ParquetHiveSerDe.PRECISION_TO_BYTE_COUNT[prec - 1];
  7.   if (precToBytes == decimalBytes.length) {
  8.    // No padding needed.
  9.    return Binary.fromByteArray(decimalBytes);
  10.   }
  11.   byte[] tgt = new byte[precToBytes];
  12.   if (hiveDecimal.signum() == -1) {
  13.    // For negative number, initializing bits to 1
  14.    for (int i = 0; i < precToBytes; i++) {
  15.     tgt[i] |= 0xFF;
  16.    }
  17.   }
  18.   System.arraycopy(decimalBytes, 0, tgt, precToBytes - decimalBytes.length, decimalBytes.length); // Padding leading zeroes/ones.
  19.   return Binary.fromByteArray(tgt);
  20.  }
  21. }

代码示例来源：origin: dremio/dremio-oss

  1. @Override
  2. public void writeValue() throws IOException {
  3.  reader.read(holder);
  4.      IntervalUtility.intToLEByteArray(holder.value, output, 0);
  5.  Arrays.fill(output, 4, 8, (byte) 0);
  6.  Arrays.fill(output, 8, 12, (byte) 0);
  7.  consumer.addBinary(Binary.fromByteArray(output));
  8. }

代码示例来源：origin: dremio/dremio-oss

  1. @Override
  2. public void writeValue() throws IOException {
  3.  reader.read(holder);
  4.      Arrays.fill(output, 0, 4, (byte) 0);
  5.  IntervalUtility.intToLEByteArray(holder.days, output, 4);
  6.  IntervalUtility.intToLEByteArray(holder.milliseconds, output, 8);
  7.  consumer.addBinary(Binary.fromByteArray(output));
  8. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.   consumer.startField(fieldName, fieldId);
  4.   reader.read(holder);
  5.    IntervalUtility.intToLEByteArray(holder.value, output, 0);
  6.    Arrays.fill(output, 4, 8, (byte) 0);
  7.    Arrays.fill(output, 8, 12, (byte) 0);
  8.   consumer.addBinary(Binary.fromByteArray(output));
  9.   consumer.endField(fieldName, fieldId);
  10.  }
  11. }

代码示例来源：origin: org.visallo/visallo-web-structured-ingest-parquet

  1. private Date getDateFromInt96(Object val) {
  2.     if (val instanceof byte[]) {
  3.       byte[] bytes = (byte[]) val;
  4.       ByteBuffer buf = Binary.fromByteArray(bytes).toByteBuffer();
  5.       buf.order(ByteOrder.LITTLE_ENDIAN);
  6.       long timeOfDayNanos = buf.getLong();
  7.       int julianDay = buf.getInt();
  9.       JDateTime date = new JDateTime(new JulianDateStamp(julianDay, (double) timeOfDayNanos / NANOS_PER_DAY));
  10.       return date.convertToDate();
  11.     }
  12.     return null;
  13.   }
  14. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.   consumer.startField(fieldName, fieldId);
  4.   reader.read(holder);
  5.   byte[] bytes = DecimalUtility.getBigDecimalFromSparse(
  6.       holder.buffer, holder.start, Decimal38SparseHolder.nDecimalDigits, holder.scale).unscaledValue().toByteArray();
  7.   byte[] output = new byte[ParquetTypeHelper.getLengthForMinorType(MinorType.DECIMAL38SPARSE)];
  8.   if (holder.getSign(holder.start, holder.buffer)) {
  9.    Arrays.fill(output, 0, output.length - bytes.length, (byte)0xFF);
  10.   } else {
  11.    Arrays.fill(output, 0, output.length - bytes.length, (byte)0x0);
  12.   }
  13.   System.arraycopy(bytes, 0, output, output.length - bytes.length, bytes.length);
  14.   consumer.addBinary(Binary.fromByteArray(output));
  15.   consumer.endField(fieldName, fieldId);
  16.  }
  17. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.   consumer.startField(fieldName, fieldId);
  4.   reader.read(holder);
  5.    Arrays.fill(output, 0, 4, (byte) 0);
  6.    IntervalUtility.intToLEByteArray(holder.days, output, 4);
  7.    IntervalUtility.intToLEByteArray(holder.milliseconds, output, 8);
  8.   consumer.addBinary(Binary.fromByteArray(output));
  9.   consumer.endField(fieldName, fieldId);
  10.  }
  11. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.   consumer.startField(fieldName, fieldId);
  4.   reader.read(holder);
  5.   byte[] bytes = DecimalUtility.getBigDecimalFromSparse(
  6.       holder.buffer, holder.start, Decimal28SparseHolder.nDecimalDigits, holder.scale).unscaledValue().toByteArray();
  7.   byte[] output = new byte[ParquetTypeHelper.getLengthForMinorType(MinorType.DECIMAL28SPARSE)];
  8.   if (holder.getSign(holder.start, holder.buffer)) {
  9.    Arrays.fill(output, 0, output.length - bytes.length, (byte)0xFF);
  10.   } else {
  11.    Arrays.fill(output, 0, output.length - bytes.length, (byte)0x0);
  12.   }
  13.   System.arraycopy(bytes, 0, output, output.length - bytes.length, bytes.length);
  14.   consumer.addBinary(Binary.fromByteArray(output));
  15.   consumer.endField(fieldName, fieldId);
  16.  }
  17. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.   if (!reader.isSet()) {
  4.    return;
  5.   }
  6.   consumer.startField(fieldName, fieldId);
  7.   reader.read(holder);
  8.    IntervalUtility.intToLEByteArray(holder.value, output, 0);
  9.    Arrays.fill(output, 4, 8, (byte) 0);
  10.    Arrays.fill(output, 8, 12, (byte) 0);
  11.   consumer.addBinary(Binary.fromByteArray(output));
  12.   consumer.endField(fieldName, fieldId);
  13.  }
  14. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.   if (!reader.isSet()) {
  4.    return;
  5.   }
  6.   consumer.startField(fieldName, fieldId);
  7.   reader.read(holder);
  8.   byte[] bytes = DecimalUtility.getBigDecimalFromSparse(
  9.       holder.buffer, holder.start, Decimal38SparseHolder.nDecimalDigits, holder.scale).unscaledValue().toByteArray();
  10.   byte[] output = new byte[ParquetTypeHelper.getLengthForMinorType(MinorType.DECIMAL38SPARSE)];
  11.   if (holder.getSign(holder.start, holder.buffer)) {
  12.    Arrays.fill(output, 0, output.length - bytes.length, (byte)0xFF);
  13.   } else {
  14.    Arrays.fill(output, 0, output.length - bytes.length, (byte)0x0);
  15.   }
  16.   System.arraycopy(bytes, 0, output, output.length - bytes.length, bytes.length);
  17.   consumer.addBinary(Binary.fromByteArray(output));
  18.   consumer.endField(fieldName, fieldId);
  19.  }
  20. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.   consumer.startField(fieldName, fieldId);
  4.   reader.read(holder);
  5.    IntervalUtility.intToLEByteArray(holder.months, output, 0);
  6.    IntervalUtility.intToLEByteArray(holder.days, output, 4);
  7.    IntervalUtility.intToLEByteArray(holder.milliseconds, output, 8);
  8.   consumer.addBinary(Binary.fromByteArray(output));
  9.   consumer.endField(fieldName, fieldId);
  10.  }
  11. }

代码示例来源：origin: dremio/dremio-oss

  1. @Override
  2. public void writeValue() throws IOException {
  3.  /* Decimals are now Little Endian. So after reading the vector contents into holder,
  4.   * we need to swap the bytes to get BE byte order. Copy the bytes from holder's
  5.   * buffer and swap them before writing the binary.
  6.   */
  7.  reader.read(holder);
  8.  holder.buffer.getBytes(holder.start, bytes, 0, 16);
  9.  DecimalHelper.swapBytes(bytes);
  10.  consumer.addBinary(Binary.fromByteArray(bytes));
  11. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.   if (!reader.isSet()) {
  4.    return;
  5.   }
  6.   consumer.startField(fieldName, fieldId);
  7.   reader.read(holder);
  8.   byte[] bytes = DecimalUtility.getBigDecimalFromSparse(
  9.       holder.buffer, holder.start, Decimal28SparseHolder.nDecimalDigits, holder.scale).unscaledValue().toByteArray();
  10.   byte[] output = new byte[ParquetTypeHelper.getLengthForMinorType(MinorType.DECIMAL28SPARSE)];
  11.   if (holder.getSign(holder.start, holder.buffer)) {
  12.    Arrays.fill(output, 0, output.length - bytes.length, (byte)0xFF);
  13.   } else {
  14.    Arrays.fill(output, 0, output.length - bytes.length, (byte)0x0);
  15.   }
  16.   System.arraycopy(bytes, 0, output, output.length - bytes.length, bytes.length);
  17.   consumer.addBinary(Binary.fromByteArray(output));
  18.   consumer.endField(fieldName, fieldId);
  19.  }
  20. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.   if (!reader.isSet()) {
  4.    return;
  5.   }
  6.   consumer.startField(fieldName, fieldId);
  7.   reader.read(holder);
  8.    Arrays.fill(output, 0, 4, (byte) 0);
  9.    IntervalUtility.intToLEByteArray(holder.days, output, 4);
  10.    IntervalUtility.intToLEByteArray(holder.milliseconds, output, 8);
  11.   consumer.addBinary(Binary.fromByteArray(output));
  12.   consumer.endField(fieldName, fieldId);
  13.  }
  14. }

代码示例来源：origin: prestosql/presto

  1. @Test
  2. public void testInvalidBinaryLength()
  3. {
  4.   try {
  5.     byte[] invalidLengthBinaryTimestamp = new byte[8];
  6.     getTimestampMillis(Binary.fromByteArray(invalidLengthBinaryTimestamp));
  7.   }
  8.   catch (PrestoException e) {
  9.     assertEquals(e.getErrorCode(), NOT_SUPPORTED.toErrorCode());
  10.     assertEquals(e.getMessage(), "Parquet timestamp must be 12 bytes, actual 8");
  11.   }
  12. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.   if (!reader.isSet()) {
  4.    return;
  5.   }
  6.   consumer.startField(fieldName, fieldId);
  7.   reader.read(holder);
  8.    IntervalUtility.intToLEByteArray(holder.months, output, 0);
  9.    IntervalUtility.intToLEByteArray(holder.days, output, 4);
  10.    IntervalUtility.intToLEByteArray(holder.milliseconds, output, 8);
  11.   consumer.addBinary(Binary.fromByteArray(output));
  12.   consumer.endField(fieldName, fieldId);
  13.  }
  14. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.  // empty lists are represented by simply not starting a field, rather than starting one and putting in 0 elements
  4.  if (reader.size() == 0) {
  5.   return;
  6.  }
  7.  consumer.startField(fieldName, fieldId);
  8.  for (int i = 0; i < reader.size(); i++) {
  9.   consumer.startField(fieldName, fieldId);
  10.   reader.read(holder);
  11.    IntervalUtility.intToLEByteArray(holder.value, output, 0);
  12.    Arrays.fill(output, 4, 8, (byte) 0);
  13.    Arrays.fill(output, 8, 12, (byte) 0);
  14.   consumer.addBinary(Binary.fromByteArray(output));
  15.   consumer.endField(fieldName, fieldId);
  16.  }
  17.  consumer.endField(fieldName, fieldId);
  18.  }
  19. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.  // empty lists are represented by simply not starting a field, rather than starting one and putting in 0 elements
  4.  if (reader.size() == 0) {
  5.   return;
  6.  }
  7.  consumer.startField(fieldName, fieldId);
  8.  for (int i = 0; i < reader.size(); i++) {
  9.   consumer.startField(fieldName, fieldId);
  10.   reader.read(holder);
  11.    Arrays.fill(output, 0, 4, (byte) 0);
  12.    IntervalUtility.intToLEByteArray(holder.days, output, 4);
  13.    IntervalUtility.intToLEByteArray(holder.milliseconds, output, 8);
  14.   consumer.addBinary(Binary.fromByteArray(output));
  15.   consumer.endField(fieldName, fieldId);
  16.  }
  17.  consumer.endField(fieldName, fieldId);
  18.  }
  19. }

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

  1. @Override
  2.  public void writeField() throws IOException {
  3.  // empty lists are represented by simply not starting a field, rather than starting one and putting in 0 elements
  4.  if (reader.size() == 0) {
  5.   return;
  6.  }
  7.  consumer.startField(fieldName, fieldId);
  8.  for (int i = 0; i < reader.size(); i++) {
  9.   consumer.startField(fieldName, fieldId);
  10.   reader.read(holder);
  11.    IntervalUtility.intToLEByteArray(holder.months, output, 0);
  12.    IntervalUtility.intToLEByteArray(holder.days, output, 4);
  13.    IntervalUtility.intToLEByteArray(holder.milliseconds, output, 8);
  14.   consumer.addBinary(Binary.fromByteArray(output));
  15.   consumer.endField(fieldName, fieldId);
  16.  }
  17.  consumer.endField(fieldName, fieldId);
  18.  }
  19. }

内容来源于网络，如有侵权，请联系作者删除！

Binary

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com