parquet.io.api.Binary.getBytes()方法的使用及代码示例

x33g5p2x  于2022-01-17 转载在 其他  
字(6.6k)|赞(0)|评价(0)|浏览(170)

本文整理了Java中parquet.io.api.Binary.getBytes()方法的一些代码示例，展示了Binary.getBytes()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Binary.getBytes()方法的具体详情如下：
包路径：parquet.io.api.Binary
类名称：Binary
方法名：getBytes

Binary.getBytes介绍

暂无

代码示例

代码示例来源：origin: prestodb/presto

  1. @Override
  2. protected void readValue(BlockBuilder blockBuilder, Type type)
  3. {
  4.   if (definitionLevel == columnDescriptor.getMaxDefinitionLevel()) {
  5.     Binary value = valuesReader.readBytes();
  6.     type.writeSlice(blockBuilder, Decimals.encodeUnscaledValue(new BigInteger(value.getBytes())));
  7.   }
  8.   else if (isValueNull()) {
  9.     blockBuilder.appendNull();
  10.   }
  11. }

代码示例来源：origin: prestodb/presto

  1. List<Domain> domains = new ArrayList<>();
  2. for (int i = 0; i < dictionarySize; i++) {
  3.   domains.add(Domain.singleValue(type, Slices.wrappedBuffer(dictionary.decodeToBinary(i).getBytes())));

代码示例来源：origin: prestodb/presto

  1. /**
  2.  * Returns GMT timestamp from binary encoded parquet timestamp (12 bytes - julian date + time of day nanos).
  3.  *
  4.  * @param timestampBinary INT96 parquet timestamp
  5.  * @return timestamp in millis, GMT timezone
  6.  */
  7. public static long getTimestampMillis(Binary timestampBinary)
  8. {
  9.   if (timestampBinary.length() != 12) {
  10.     throw new PrestoException(HIVE_BAD_DATA, "Parquet timestamp must be 12 bytes, actual " + timestampBinary.length());
  11.   }
  12.   byte[] bytes = timestampBinary.getBytes();
  13.   // little endian encoding - need to invert byte order
  14.   long timeOfDayNanos = Longs.fromBytes(bytes[7], bytes[6], bytes[5], bytes[4], bytes[3], bytes[2], bytes[1], bytes[0]);
  15.   int julianDay = Ints.fromBytes(bytes[11], bytes[10], bytes[9], bytes[8]);
  16.   return julianDayToMillis(julianDay) + (timeOfDayNanos / NANOS_PER_MILLISECOND);
  17. }

代码示例来源：origin: prestodb/presto

  1. /**
  2.  * Returns GMT timestamp from binary encoded parquet timestamp (12 bytes - julian date + time of day nanos).
  3.  *
  4.  * @param timestampBinary INT96 parquet timestamp
  5.  * @return timestamp in millis, GMT timezone
  6.  */
  7. public static long getTimestampMillis(Binary timestampBinary)
  8. {
  9.   if (timestampBinary.length() != 12) {
  10.     throw new PrestoException(NOT_SUPPORTED, "Parquet timestamp must be 12 bytes, actual " + timestampBinary.length());
  11.   }
  12.   byte[] bytes = timestampBinary.getBytes();
  13.   // little endian encoding - need to invert byte order
  14.   long timeOfDayNanos = Longs.fromBytes(bytes[7], bytes[6], bytes[5], bytes[4], bytes[3], bytes[2], bytes[1], bytes[0]);
  15.   int julianDay = Ints.fromBytes(bytes[11], bytes[10], bytes[9], bytes[8]);
  16.   return julianDayToMillis(julianDay) + (timeOfDayNanos / NANOS_PER_MILLISECOND);
  17. }

代码示例来源：origin: prestodb/presto

  1. @Override
  2. protected void readValue(BlockBuilder blockBuilder, Type type)
  3. {
  4.   if (definitionLevel == columnDescriptor.getMaxDefinitionLevel()) {
  5.     Binary binary = valuesReader.readBytes();
  6.     Slice value;
  7.     if (binary.length() == 0) {
  8.       value = EMPTY_SLICE;
  9.     }
  10.     else {
  11.       value = wrappedBuffer(binary.getBytes());
  12.     }
  13.     if (isVarcharType(type)) {
  14.       value = truncateToLength(value, type);
  15.     }
  16.     if (isCharType(type)) {
  17.       value = truncateToLengthAndTrimSpaces(value, type);
  18.     }
  19.     type.writeSlice(blockBuilder, value);
  20.   }
  21.   else if (isValueNull()) {
  22.     blockBuilder.appendNull();
  23.   }
  24. }

代码示例来源：origin: prestodb/presto

  1. @Override
  2. protected void readValue(BlockBuilder blockBuilder, Type type)
  3. {
  4.   if (definitionLevel == columnDescriptor.getMaxDefinitionLevel()) {
  5.     long decimalValue;
  6.     // When decimals are encoded with primitive types Parquet stores unscaled values
  7.     if (columnDescriptor.getType().equals(INT32)) {
  8.       decimalValue = valuesReader.readInteger();
  9.     }
  10.     else if (columnDescriptor.getType().equals(INT64)) {
  11.       decimalValue = valuesReader.readLong();
  12.     }
  13.     else {
  14.       decimalValue = getShortDecimalValue(valuesReader.readBytes().getBytes());
  15.     }
  16.     type.writeLong(blockBuilder, decimalValue);
  17.   }
  18.   else if (isValueNull()) {
  19.     blockBuilder.appendNull();
  20.   }
  21. }

代码示例来源：origin: prestodb/presto

  1. Slice minSlice = Slices.wrappedBuffer(binaryStatistics.getMin().getBytes());
  2. Slice maxSlice = Slices.wrappedBuffer(binaryStatistics.getMax().getBytes());
  3. if (minSlice.compareTo(maxSlice) > 0) {
  4.   failWithCorruptionException(failOnCorruptedParquetStatistics, column, id, binaryStatistics);

代码示例来源：origin: com.twitter/parquet-tools

  1. public static BigInteger binaryToBigInteger(Binary value) {
  2.   byte[] data = value.getBytes();
  3.   if (data == null) return null;
  4.   return new BigInteger(data);
  5. }

代码示例来源：origin: com.twitter/parquet-tools

  1. public static String binaryToString(Binary value) {
  2.   byte[] data = value.getBytes();
  3.   if (data == null) return null;
  4.   try {
  5.     CharBuffer buffer = UTF8_DECODER.decode(value.toByteBuffer());
  6.     return buffer.toString();
  7.   } catch (Throwable th) {
  8.   }
  9.   return "<bytes...>";
  10. }

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. @Override
  2. public String toString(ColumnReader columnReader) {
  3.  return Arrays.toString(columnReader.getBinary().getBytes());
  4. }
  5. @Override

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. protected static Binary copy(Binary binary) {
  2.   return Binary.fromByteArray(
  3.     Arrays.copyOf(binary.getBytes(), binary.length()));
  4.  }
  5. }

代码示例来源：origin: org.apache.tajo/tajo-storage

  1. @Override
  2.  final public void addBinary(Binary value) {
  3.   parent.add(new BlobDatum(ByteBuffer.wrap(value.getBytes())));
  4.  }
  5. }

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. /**
  2.  * {@inheritDoc}
  3.  */
  4. @Override
  5. public void addBinary(Binary value) {
  6.  if (DEBUG) log(Arrays.toString(value.getBytes()));
  7.  delegate.addBinary(value);
  8. }

代码示例来源：origin: org.apache.tajo/tajo-storage

  1. @Override
  2.  final public void addBinary(Binary value) {
  3.   parent.add(DatumFactory.createInet4(value.getBytes()));
  4.  }
  5. }

代码示例来源：origin: uk.co.nichesolutions.presto/presto-hive

  1. @Override
  2. public void addBinary(Binary value)
  3. {
  4.   nulls[fieldIndex] = false;
  5.   if (types[fieldIndex] == TIMESTAMP) {
  6.     longs[fieldIndex] = ParquetTimestampUtils.getTimestampMillis(value);
  7.   }
  8.   else {
  9.     slices[fieldIndex] = wrappedBuffer(value.getBytes());
  10.   }
  11. }

代码示例来源：origin: com.twitter/parquet-pig

  1. @Override
  2. final public void addBinary(Binary value) {
  3.  parent.add(new DataByteArray(value.getBytes()));
  4. }

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. @Override
  2.  public void writeBytes(Binary v) {
  3.   int i = 0;
  4.   byte[] vb = v.getBytes();
  5.   int length = previous.length < vb.length ? previous.length : vb.length;
  6.   for(i = 0; (i < length) && (previous[i] == vb[i]); i++);
  7.   prefixLengthWriter.writeInteger(i);
  8.   suffixWriter.writeBytes(Binary.fromByteArray(vb, i, vb.length - i));
  9.   previous = vb;
  10.  }
  11. }

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. @Override
  2. public void writeBytes(Binary v) {
  3.  try {
  4.   lengthWriter.writeInteger(v.length());
  5.   out.write(v.getBytes());
  6.  } catch (IOException e) {
  7.   throw new ParquetEncodingException("could not write bytes", e);
  8.  }
  9. }

代码示例来源：origin: org.apache.tajo/tajo-storage

  1. @Override
  2.  final public void addBinary(Binary value) {
  3.   try {
  4.    ProtobufDatumFactory factory =
  5.      ProtobufDatumFactory.get(dataType.getCode());
  6.    Message.Builder builder = factory.newBuilder();
  7.    builder.mergeFrom(value.getBytes());
  8.    parent.add(factory.createDatum(builder));
  9.   } catch (InvalidProtocolBufferException e) {
  10.    throw new RuntimeException(e);
  11.   }
  12.  }
  13. }

代码示例来源：origin: uk.co.nichesolutions.presto/presto-hive

  1. @Override
  2. public void addBinary(Binary value)
  3. {
  4.   addMissingValues();
  5.   if (type == TIMESTAMP) {
  6.     builder.writeLong(ParquetTimestampUtils.getTimestampMillis(value)).closeEntry();
  7.   }
  8.   else {
  9.     VARBINARY.writeSlice(builder, wrappedBuffer(value.getBytes()));
  10.   }
  11. }

内容来源于网络，如有侵权，请联系作者删除！

Binary

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com