parquet.io.api.Binary.fromByteArray()方法的使用及代码示例

x33g5p2x  于2022-01-17 转载在 其他  
字(8.5k)|赞(0)|评价(0)|浏览(106)

本文整理了Java中parquet.io.api.Binary.fromByteArray()方法的一些代码示例，展示了Binary.fromByteArray()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Binary.fromByteArray()方法的具体详情如下：
包路径：parquet.io.api.Binary
类名称：Binary
方法名：fromByteArray

Binary.fromByteArray介绍

暂无

代码示例

代码示例来源：origin: prestodb/presto

  1. public BinaryDictionary(DictionaryPage dictionaryPage, Integer length)
  2.     throws IOException
  3. {
  4.   super(dictionaryPage.getEncoding());
  5.   byte[] dictionaryBytes = dictionaryPage.getSlice().getBytes();
  6.   content = new Binary[dictionaryPage.getDictionarySize()];
  7.   int offset = 0;
  8.   if (length == null) {
  9.     for (int i = 0; i < content.length; i++) {
  10.       int len = readIntLittleEndian(dictionaryBytes, offset);
  11.       offset += 4;
  12.       content[i] = Binary.fromByteArray(dictionaryBytes, offset, len);
  13.       offset += len;
  14.     }
  15.   }
  16.   else {
  17.     checkArgument(length > 0, "Invalid byte array length: %s", length);
  18.     for (int i = 0; i < content.length; i++) {
  19.       content[i] = Binary.fromByteArray(dictionaryBytes, offset, length);
  20.       offset += length;
  21.     }
  22.   }
  23. }

代码示例来源：origin: prestodb/presto

  1. @Test
  2. public void testInvalidBinaryLength()
  3. {
  4.   try {
  5.     byte[] invalidLengthBinaryTimestamp = new byte[8];
  6.     getTimestampMillis(Binary.fromByteArray(invalidLengthBinaryTimestamp));
  7.   }
  8.   catch (PrestoException e) {
  9.     assertEquals(e.getErrorCode(), NOT_SUPPORTED.toErrorCode());
  10.     assertEquals(e.getMessage(), "Parquet timestamp must be 12 bytes, actual 8");
  11.   }
  12. }

代码示例来源：origin: prestodb/presto

  1. private Binary decimalToBinary(final HiveDecimal hiveDecimal, final DecimalTypeInfo decimalTypeInfo)
  2.   {
  3.     int prec = decimalTypeInfo.precision();
  4.     int scale = decimalTypeInfo.scale();
  5.     byte[] decimalBytes = hiveDecimal.setScale(scale).unscaledValue().toByteArray();
  7.     // Estimated number of bytes needed.
  8.     int precToBytes = ParquetHiveSerDe.PRECISION_TO_BYTE_COUNT[prec - 1];
  9.     if (precToBytes == decimalBytes.length) {
  10.       // No padding needed.
  11.       return Binary.fromByteArray(decimalBytes);
  12.     }
  14.     byte[] tgt = new byte[precToBytes];
  15.     if (hiveDecimal.signum() == -1) {
  16.       // For negative number, initializing bits to 1
  17.       for (int i = 0; i < precToBytes; i++) {
  18.         tgt[i] |= 0xFF;
  19.       }
  20.     }
  22.     System.arraycopy(decimalBytes, 0, tgt, precToBytes - decimalBytes.length, decimalBytes.length); // Padding leading zeroes/ones.
  23.     return Binary.fromByteArray(tgt);
  24.   }
  25. }

代码示例来源：origin: prestodb/presto

  1. case BINARY:
  2.   byte[] vBinary = ((BinaryObjectInspector) inspector).getPrimitiveJavaObject(value);
  3.   recordConsumer.addBinary(Binary.fromByteArray(vBinary));
  4.   break;
  5. case TIMESTAMP:

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. @Override
  2. public void setMinMaxFromBytes(byte[] minBytes, byte[] maxBytes) {
  3.  max = Binary.fromByteArray(maxBytes);
  4.  min = Binary.fromByteArray(minBytes);
  5.  this.markAsNotEmpty();
  6. }

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. @Override
  2. public Binary readBytes() {
  3.  int length = lengthReader.readInteger();
  4.  int start = offset;
  5.  offset = start + length;
  6.  return Binary.fromByteArray(in, start, length);
  7. }

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. @Override
  2. public Binary readBytes() {
  3.  try {
  4.   int start = offset;
  5.   offset = start + length;
  6.   return Binary.fromByteArray(in, start, length);
  7.  } catch (RuntimeException e) {
  8.   throw new ParquetDecodingException("could not read bytes at offset " + offset, e);
  9.  }
  10. }

代码示例来源：origin: com.twitter/parquet-thrift

  1. private void writeBinaryToRecordConsumer(ByteBuffer buf) {
  2.  recordConsumer.addBinary(Binary.fromByteArray(buf.array(), buf.position(), buf.limit() - buf.position()));
  3. }

代码示例来源：origin: asakusafw/asakusafw

  1. private void writeAsBinary(RecordConsumer consumer, BigDecimal original, BigInteger unscaled) {
  2.     byte[] bytes = unscaled.toByteArray();
  3.     Binary binary;
  4.     if (bytes.length == byteLength) {
  5.       binary = Binary.fromByteArray(bytes);
  6.     } else if (bytes.length < byteLength) {
  7.       byte[] newBytes = new byte[byteLength];
  8.       if (unscaled.signum() < 0) {
  9.         Arrays.fill(newBytes, 0, newBytes.length - bytes.length, (byte) -1);
  10.       }
  11.       System.arraycopy(bytes, 0, newBytes, newBytes.length - bytes.length, bytes.length);
  12.       binary = Binary.fromByteArray(newBytes);
  13.     } else {
  14.       throw new IllegalStateException(MessageFormat.format(
  15.           Messages.getString("DecimalValueDriver.errorPrecisionTooSmall"), //$NON-NLS-1$
  16.           original,
  17.           precision));
  18.     }
  19.     consumer.addBinary(binary);
  20.   }
  21. }

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. protected static Binary copy(Binary binary) {
  2.   return Binary.fromByteArray(
  3.     Arrays.copyOf(binary.getBytes(), binary.length()));
  4.  }
  5. }

代码示例来源：origin: asakusafw/asakusafw

  1. @Override
  2.   public void write(Object value, RecordConsumer consumer) {
  3.     Text text = ((StringOption) value).get();
  4.     consumer.addBinary(Binary.fromByteArray(text.getBytes(), 0, text.getLength()));
  5.   }
  6. },

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. public DeltaByteArrayReader() {
  2.  this.prefixLengthReader = new DeltaBinaryPackingValuesReader();
  3.  this.suffixReader = new DeltaLengthByteArrayValuesReader();
  4.  this.previous = Binary.fromByteArray(new byte[0]);
  5. }

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. private Binary decimalToBinary(final HiveDecimal hiveDecimal, final DecimalTypeInfo decimalTypeInfo) {
  2.   int prec = decimalTypeInfo.precision();
  3.   int scale = decimalTypeInfo.scale();
  4.   byte[] decimalBytes = hiveDecimal.setScale(scale).unscaledValue().toByteArray();
  6.   // Estimated number of bytes needed.
  7.   int precToBytes = ParquetHiveSerDe.PRECISION_TO_BYTE_COUNT[prec - 1];
  8.   if (precToBytes == decimalBytes.length) {
  9.    // No padding needed.
  10.    return Binary.fromByteArray(decimalBytes);
  11.   }
  13.   byte[] tgt = new byte[precToBytes];
  14.    if (hiveDecimal.signum() == -1) {
  15.    // For negative number, initializing bits to 1
  16.    for (int i = 0; i < precToBytes; i++) {
  17.     tgt[i] |= 0xFF;
  18.    }
  19.   }
  21.   System.arraycopy(decimalBytes, 0, tgt, precToBytes - decimalBytes.length, decimalBytes.length); // Padding leading zeroes/ones.
  22.   return Binary.fromByteArray(tgt);
  23.  }
  24. }

代码示例来源：origin: asakusafw/asakusafw

  1. @Override
  2.   public void write(Object value, RecordConsumer consumer) {
  3.     StringOption option = (StringOption) value;
  4.     Text text = option.get();
  5.     byte[] bytes = text.getBytes();
  6.     int length = text.getLength();
  7.     if (length > limit) {
  8.       // if byte-length > limit, the string may code-point-count >= limit
  9.       String stripped = HiveBaseChar.getPaddedValue(text.toString(), limit);
  10.       consumer.addBinary(Binary.fromString(stripped));
  11.     } else {
  12.       consumer.addBinary(Binary.fromByteArray(bytes, 0, length));
  13.     }
  14.   }
  15. }

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. @Override
  2. public Binary readBytes() {
  3.  try {
  4.   int length = BytesUtils.readIntLittleEndian(in, offset);
  5.   int start = offset + 4;
  6.   offset = start + length;
  7.   return Binary.fromByteArray(in, start, length);
  8.  } catch (IOException e) {
  9.   throw new ParquetDecodingException("could not read bytes at offset " + offset, e);
  10.  } catch (RuntimeException e) {
  11.   throw new ParquetDecodingException("could not read bytes at offset " + offset, e);
  12.  }
  13. }

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. @Override
  2.  public void writeBytes(Binary v) {
  3.   int i = 0;
  4.   byte[] vb = v.getBytes();
  5.   int length = previous.length < vb.length ? previous.length : vb.length;
  6.   for(i = 0; (i < length) && (previous[i] == vb[i]); i++);
  7.   prefixLengthWriter.writeInteger(i);
  8.   suffixWriter.writeBytes(Binary.fromByteArray(vb, i, vb.length - i));
  9.   previous = vb;
  10.  }
  11. }

代码示例来源：origin: com.facebook.presto.hive/hive-apache

  1. @Override
  2.  public Binary readBytes() {
  3.   int prefixLength = prefixLengthReader.readInteger();
  4.   // This does not copy bytes
  5.   Binary suffix = suffixReader.readBytes();
  6.   int length = prefixLength + suffix.length();
  7.   
  8.   // We have to do this to materialize the output
  9.   if(prefixLength != 0) {
  10.    byte[] out = new byte[length];
  11.    System.arraycopy(previous.getBytes(), 0, out, 0, prefixLength);
  12.    System.arraycopy(suffix.getBytes(), 0, out, prefixLength, suffix.length());
  13.    previous =  Binary.fromByteArray(out);
  14.   } else {
  15.    previous = suffix;
  16.   }
  17.   return previous;
  18.  }
  19. }

代码示例来源：origin: uk.co.nichesolutions.presto/presto-hive

  1. @Test
  2. public void testInvalidBinaryLength()
  3. {
  4.   try {
  5.     byte[] invalidLengthBinaryTimestamp = new byte[8];
  6.     getTimestampMillis(Binary.fromByteArray(invalidLengthBinaryTimestamp));
  7.   }
  8.   catch (PrestoException e) {
  9.     assertEquals(e.getErrorCode(), HIVE_BAD_DATA.toErrorCode());
  10.     assertEquals(e.getMessage(), "Parquet timestamp must be 12 bytes, actual 8");
  11.   }
  12. }

代码示例来源：origin: com.facebook.presto/presto-parquet

  1. @Test
  2. public void testInvalidBinaryLength()
  3. {
  4.   try {
  5.     byte[] invalidLengthBinaryTimestamp = new byte[8];
  6.     getTimestampMillis(Binary.fromByteArray(invalidLengthBinaryTimestamp));
  7.   }
  8.   catch (PrestoException e) {
  9.     assertEquals(e.getErrorCode(), NOT_SUPPORTED.toErrorCode());
  10.     assertEquals(e.getMessage(), "Parquet timestamp must be 12 bytes, actual 8");
  11.   }
  12. }

代码示例来源：origin: com.facebook.presto/presto-hive

  1. private Binary decimalToBinary(final HiveDecimal hiveDecimal, final DecimalTypeInfo decimalTypeInfo)
  2.   {
  3.     int prec = decimalTypeInfo.precision();
  4.     int scale = decimalTypeInfo.scale();
  5.     byte[] decimalBytes = hiveDecimal.setScale(scale).unscaledValue().toByteArray();
  7.     // Estimated number of bytes needed.
  8.     int precToBytes = ParquetHiveSerDe.PRECISION_TO_BYTE_COUNT[prec - 1];
  9.     if (precToBytes == decimalBytes.length) {
  10.       // No padding needed.
  11.       return Binary.fromByteArray(decimalBytes);
  12.     }
  14.     byte[] tgt = new byte[precToBytes];
  15.     if (hiveDecimal.signum() == -1) {
  16.       // For negative number, initializing bits to 1
  17.       for (int i = 0; i < precToBytes; i++) {
  18.         tgt[i] |= 0xFF;
  19.       }
  20.     }
  22.     System.arraycopy(decimalBytes, 0, tgt, precToBytes - decimalBytes.length, decimalBytes.length); // Padding leading zeroes/ones.
  23.     return Binary.fromByteArray(tgt);
  24.   }
  25. }

内容来源于网络，如有侵权，请联系作者删除！

Binary

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com