zlib state

Zlib-State

디코딩 상태를 캡처 할 수있는 Zlib 라이브러리의 저수준 인터페이스.

설치하다

PYPI에서 :

 pip install zlib-state

출처 :

 pip install .

Python 3.7-3.12로 Ubuntu/MacOS/Windows에서 테스트.

gzipstatefile

버퍼 리더로 감압제를 랩핑합니다.

내 벤치마킹을 기반으로, 이것은 Python의 GZIP보다 약간 느립니다.

일반적인 사용 패턴은 다음과 같습니다.

 import zlib_state

TARGET_LINE = 5000 # pick back up after around the 5,000th line
# Specify keep_last_state=True to tell object to grab and keep the state and pos after each block
with zlib_state . GzipStateFile ( 'testdata/frankenstein.txt.gz' , keep_last_state = True ) as f :
    for i , line in enumerate ( f ):
        if i == TARGET_LINE :
            state , pos = f . last_state , f . last_state_pos

with zlib_state . GzipStateFile ( 'testdata/frankenstein.txt.gz' ) as f :
    f . zseek ( pos , state )
    remainder = f . read ()

감압제

까다 롭고 용서할 수없는 매우 기본적인 감압 객체.

내 벤치마킹을 기반으로 Python의 GZIP보다 GZIP 파일을 더 빨리 반복 할 수 있습니다.

일반적인 사용 패턴은 다음과 같습니다.

 import zlib_state

decomp = zlib_state . Decompressor ( 32 + 15 ) # from zlib; 32 indicates gzip header, 15 window size
block_count = 0
with open ( 'testdata/frankenstein.txt.gz' , 'rb' ) as f :
    while not decomp . eof ():
        needed_input = decomp . needs_input ()
        if needed_input > 0 :
            # decomp needs more input, and it tells you how much.
            decomp . feed_input ( f . read ( needed_input ))
        # next_chunk may be empty (e.g., if finished with gzip headers) or may contain data.
        # It sends as much as it has left in its output buffer, or asks zlib to continue.
        next_chunk = decomp . read () # you can also pass a maximum size to take and/or a buffer to write to
        if decomp . block_boundary ():
            block_count += 1
            # When it reaches the end of a deflate block, it always stops. At these times, you can grab the state
            # if you wish.
            if block_count == 4 : # resume after the 4th block
                state = decomp . get_state () # includes zdict, bits, byte -- everything it needs to resume from pos
                pos = decomp . total_in () # the current position in the binary file to resume from
    print ( f' { block_count } blocks processed' )
    # resume from somewhere in the file. Only possible spots are the block boundaries, given the state
    f . seek ( pos )
    decomp = zlib_state . Decompressor ( - 15 ) # from zlib; 15 window size, negative means no headers
    decomp . set_state ( * state )
    while not decomp . eof ():
        needed_input = decomp . needs_input ()
        if needed_input > 0 :
            # decomp needs more input, and it tells you how much.
            decomp . feed_input ( f . read ( needed_input ))
        next_chunk = decomp . read ()