[RFE] tarfile: add an option to change the "blocking factor"

[stratakis]

BPO	31774
Nosy	@gustaebel, @vstinner, @stratakis, @nitishch
Files	tartest.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-10-12.14:17:19.775>
labels = ['3.8', 'type-feature', 'library']
title = '[RFE] tarfile: add an option to change the "blocking factor"'
updated_at = <Date 2018-06-20.13:04:55.361>
user = 'https://github.com/stratakis'

bugs.python.org fields:

activity = <Date 2018-06-20.13:04:55.361>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-10-12.14:17:19.775>
creator = 'cstratak'
dependencies = []
files = ['47218']
hgrepos = []
issue_num = 31774
keywords = []
message_count = 4.0
messages = ['304241', '304457', '304458', '320075']
nosy_count = 4.0
nosy_names = ['lars.gustaebel', 'vstinner', 'cstratak', 'nitishch']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue31774'
versions = ['Python 3.8']

Linked PRs

• gh-75955: tarfile: Allow configuring the record size #129260

[stratakis]

Trying to create an archive with the tarfile module, by specifying a different blocking factor, doesn't seem to work as only the default value is being used. The issue is reproducible on all the active python branches.

Attaching a script to reproduce it.

Original bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1492157

[nitishch]

Seems like bufsize is used only in streaming modes. Even in the documentation bufsize is described only in the context of streaming modes.

Even TarFile constructor doesn't take bufsize as an argument. Why is it so?

[nitishch]

Sorry. My bad. There *is* an argument 'copybufsize' in TarFile.

[vstinner]

Extract of tartest.py:

tarfile.open(name=_tmp_tar, mode='w', bufsize=EXPECTED_SIZE)

Hum, the bufsize argument only applies to the "...|compression" modes (ex: "w|gz"). Moreover, it shouldn't change the size of the tarfile, it only changes the size of parameters of read() and write() arguments.

Extract of Stream.__write():

    while len(self.buf) > self.bufsize:
        self.fileobj.write(self.buf[:self.bufsize])
        self.buf = self.buf[self.bufsize:]

Trying to create an archive with the tarfile module, by specifying a different blocking factor, (...)

I didn't know that we can change the "blocking factor". In the UNIX tar command, I see the -b option:

   -b, --blocking-factor=BLOCKS
          Set record size to BLOCKSx512 bytes.

In Lib/tarfile.py, I see:

BLOCKSIZE = 512                 # length of processing blocks
RECORDSIZE = BLOCKSIZE * 20     # length of records

So values are hardcoded in Python.

So this issue is a feature request, not a bug report :-)