如何读取用7z压缩的文本文件?

lyr7nygr  于 2021-08-20  发布在  Java
关注(0)|答案(3)|浏览(493)

我想从一个7z压缩的csv(文本)文件中逐行读取(在Python2.7中)。我不想解压缩整个(大)文件,而是流式传输行。
我试过了 pylzma.decompressobj() 没有成功。我得到一个数据错误。请注意,此代码尚未逐行读取:

  1. input_filename = r"testing.csv.7z"
  2. with open(input_filename, 'rb') as infile:
  3. obj = pylzma.decompressobj()
  4. o = open('decompressed.raw', 'wb')
  5. obj = pylzma.decompressobj()
  6. while True:
  7. tmp = infile.read(1)
  8. if not tmp: break
  9. o.write(obj.decompress(tmp))
  10. o.close()

输出:

  1. o.write(obj.decompress(tmp))
  2. ValueError: data error during decompression
4ktjp1zp

4ktjp1zp1#

这将允许您迭代这些行。它部分来自我在回答另一个问题时发现的一些代码。
此时此刻( pylzma-0.5.0 ) py7zlib 模块未实现允许将存档成员作为字节或字符流读取的api—它的 ArchiveFile 类只提供一个 read() 函数,该函数一次解压缩并返回成员中未压缩的数据。有鉴于此,可以做的最好的事情就是通过python生成器以迭代方式返回字节或行,并将其用作缓冲区。
下面是后者,但如果问题是归档成员文件本身太大,则可能没有帮助。
下面的代码应该可以在Python3.x和2.7中使用。

  1. import io
  2. import os
  3. import py7zlib
  4. class SevenZFileError(py7zlib.ArchiveError):
  5. pass
  6. class SevenZFile(object):
  7. @classmethod
  8. def is_7zfile(cls, filepath):
  9. """ Determine if filepath points to a valid 7z archive. """
  10. is7z = False
  11. fp = None
  12. try:
  13. fp = open(filepath, 'rb')
  14. archive = py7zlib.Archive7z(fp)
  15. _ = len(archive.getnames())
  16. is7z = True
  17. finally:
  18. if fp: fp.close()
  19. return is7z
  20. def __init__(self, filepath):
  21. fp = open(filepath, 'rb')
  22. self.filepath = filepath
  23. self.archive = py7zlib.Archive7z(fp)
  24. def __contains__(self, name):
  25. return name in self.archive.getnames()
  26. def readlines(self, name, newline=''):
  27. r""" Iterator of lines from named archive member.
  28. `newline` controls how line endings are handled.
  29. It can be None, '', '\n', '\r', and '\r\n' and works the same way as it does
  30. in StringIO. Note however that the default value is different and is to enable
  31. universal newlines mode, but line endings are returned untranslated.
  32. """
  33. archivefile = self.archive.getmember(name)
  34. if not archivefile:
  35. raise SevenZFileError('archive member %r not found in %r' %
  36. (name, self.filepath))
  37. # Decompress entire member and return its contents iteratively.
  38. data = archivefile.read().decode()
  39. for line in io.StringIO(data, newline=newline):
  40. yield line
  41. if __name__ == '__main__':
  42. import csv
  43. if SevenZFile.is_7zfile('testing.csv.7z'):
  44. sevenZfile = SevenZFile('testing.csv.7z')
  45. if 'testing.csv' not in sevenZfile:
  46. print('testing.csv is not a member of testing.csv.7z')
  47. else:
  48. reader = csv.reader(sevenZfile.readlines('testing.csv'))
  49. for row in reader:
  50. print(', '.join(row))
展开查看全部
c7rzv4ha

c7rzv4ha2#

如果您使用的是Python3.3+,则可以使用 lzma 在该版本中添加到标准库的模块。
见: lzma 例子

ppcbkaq5

ppcbkaq53#

如果您可以使用python 3,那么有一个有用的库py7zr,它支持部分7zip解压缩,如下所示:

  1. import py7zr
  2. import re
  3. filter_pattern = re.compile(r'<your/target/file_and_directories/regex/expression>')
  4. with SevenZipFile('archive.7z', 'r') as archive:
  5. allfiles = archive.getnames()
  6. selective_files = [f if filter_pattern.match(f) for f in allfiles]
  7. archive.extract(targets=selective_files)

相关问题