Unclear documentation for TarFile

[yngvem]

Documentation

The tarfile docs does not make it clear how a programmer can read data from a tarfile into memory without doing a round-trip writing it to the file system. As far as I understand, reading partial data from a tar file essentially amounts to the following steps:

import tarfile

with open("myfile.tar") as f:
    tar = tarfile.TarFile(fileobj=f)

    tar_info = next(member for member in f.getmembers() if member.is_file())
    f.seek(tar_info.offset_data)
    data = f.read(tar_info.size)

However, to arrive at this, you either need to be confident enough to read the CPython source code, or you need to know that tar-files stores the byte-contents unchanged, and that TarInfo.size is the size of the data without the file header. Neither of these are obvious for less experienced programmers.

I suggest that we make two changes to the tarfile docs:

1. Expand the documentation for TarInfo.size so it says more than just "Size in bytes". Size of what exactly? The archived file as far as I can tell.
2. Include a minimal example (like I have above, but slightly more pedagogical maybe) to the Reading Examples section.

I can propose a PR with these changes if you think that is useful.

[zware]

I believe you're looking for TarFile.extractfile(member). To adapt your example (without testing it :))

import tarfile

with tarfile.open("myfile.tar") as tar:
    for tar_info in tar:
        reader = tar.extractfile(tar_info)
        if reader is not None:
            data = reader.read().decode()
            break

An example using extractfile might be good to add.

[dagangtj]

I would like to work on this documentation improvement. I can see the confusion around reading tarfile data into memory without writing to the filesystem. I will propose a documentation update to clarify the use of TarFile.extractfile() for in-memory access and document the TarInfo.offset_data / TarInfo.size attributes more clearly.

I will submit a PR shortly.

[yngvem]

I believe you're looking for TarFile.extractfile(member). To adapt your example (without testing it :))

import tarfile

with tarfile.open("myfile.tar") as tar:
    for tar_info in tar:
        reader = tar.extractfile(tar_info)
        if reader is not None:
            data = reader.read().decode()
            break

An example using extractfile might be good to add.

That's exactly what I was looking for. I thought it was odd that I couldn't find an easy-to-use function for this in the docs (I can't believe I didn't see it, I was certain I looked at extractmember too, but I guess that was a fogment of my imagination...).

Oh well, my points still stands on adding the example and writing what size is the size of.