Text of peps.python.org › pep-0706 as Styx crawled it on Nov 19, 2026. The page may have changed since. Only its words are kept, without pictures or layout, and only the first part of a long page.

Open the live page

PEP 706 – Filter for tarfile.extractall | peps.python.org

PEP 706 – Filter for tarfile.extractall PEP 706 – Filter for tarfile.extractall Author : Petr Viktorin <encukou at gmail.com> Discussions-To : Discourse thread Status : Final Type : Standards Track Created : 09-Feb-2023 Python-Version : 3.12 Post-History : 25-Jan-2023 , 15-Feb-2023 Resolution : Discourse message Table of Contents Abstract Motivation Rationale Setting a precedent Full disclosure & redistributor info Specification Modifying and forgetting member metadata Filters Defaults and their configuration FilterError Errorlevel, and fatal/non-fatal errors Hints for further verification TarInfo identity, and offset tarfile CLI Other archive libraries Shutil Complex filters Backwards Compatibility Backporting & Forward Compatibility Security Implications How to Teach This Reference Implementation Rejected Ideas SafeTarFile Add absolute_path option to tarfile Other names for the 'tar' filter Possible Further Work Adding filters to zipfile and shutil.unpack_archive Thanks Copyright Important This PEP is a historical document.

The up-to-date, canonical documentation can now be found at tarfile documentation . × See PEP 1 for how to propose changes. Abstract The extraction methods in tarfile gain a filter argument, which allows rejecting files or modifying metadata as the archive is extracted. Three built-in named filters are provided, aimed at limiting features that might be surprising or dangerous. These can be used as-is, or serve as a base for custom filters.

After a deprecation period, a strict (but safer) filter will become the default. Motivation The tar format is used for several use cases, many of which have different needs. For example: A backup of a UNIX workstation should faithfully preserve all kinds of details like file permissions, symlinks to system configuration, and various kinds of special files. When unpacking a data bundle, it’s much more important that the unpacking will not have unintended consequences – like exposing a password file by symlinking it to a public place.

To support all its use cases, the tar format has many features. In many cases, it’s best to ignore or disallow some of them when extracting an archive. Python allows extracting tar archives using tarfile.TarFile.extractall() , whose docs warn to never extract archives from untrusted sources without prior inspection . However, it’s not clear what kind of inspection should be done. Indeed, it’s quite tricky to do such an inspection correctly.

As a result, many people don’t bother, or do the check incorrectly, resulting in security issues such as CVE-2007-4559 . Since tarfile was first written, it’s become more accepted that warnings in documentation are not enough. Whenever possible, an unsafe operation should be explicitly requested ; potentially dangerous operations should look dangerous. However, TarFile.extractall looks benign in a code review. Tarfile extraction is also exposed via shutil.unpack_archive() , which allows the user to not care about the kind of archive they’re dealing with.

The API is very inviting for extracting archives without prior inspection, even though the docs again warn against it. It has been argued that Python is not wrong – it behaves exactly as documented – but that’s beside the point. Let’s improve the situation rather than assign/avoid blame. Python and its docs are the best place to improve things. Rationale How do we improve things? Unfortunately, we will need to change the defaults, which implies breaking backwards compatibility.

TarFile.extractall is what people reach for when they need to extract a tarball. Its default behaviour needs to change. What would be the best behaviour? That depends on the use case. So, we’ll add several general “policies” to control extraction. They are based on use cases , and ideally they should have straightforward security implications: Current behavior: trusting the archive. Suitable e.g. as a building block for libraries that do the check themselves, or extracting an archive you just made yourself.

Unpacking a UNIX archive: roughly following GNU tar , e.g. stripping leading / from filenames. Unpacking a general data archive: the shutil.unpack_archive() use case, where it’s not important to preserve details specific to tar or Unix-like filesystems. After a deprecation period, the last option – the most limited but most secure one – will become the default. Even with better general defaults, users should still verify the archives they extract, and perhaps modify some of the metadata.

Superficially, the following looks like a reasonable way to do this today: Call TarFile.getmembers Verify or modify each member’s TarInfo Pass the result to extractall ’s members However, there are some issues with this approach: It’s possible to modify TarInfo objects, but the changes to them affect all subsequent operations on the same TarFile object. This behavior is fine for most uses, but despite that, it would be very surprising if TarFile.extractall did this by default.

Calling getmembers can be expensive and it requires a seekable archive . When verifying members in advance, it may be necessary to track how each member would have changed the filesystem, e.g. how symlinks are being set up. This is hard. We can’t expect users to do it. To solve these issues we’ll: Provide a supported way to “clone” and modify TarInfo objects. A replace method, similar to dataclasses.replace() or namedtuple._replace should do the trick.

Provide a “filter” hook in extractall ’s loop that can modify or discard members before they are processed. Require that this hook is called just before extracting each member, so it can scan the current state of the disk. This will greatly simplify the implementation of policies (both in stdlib and user code), at the cost of not being able to do a precise “dry run”. The hook API will be very similar to the existing filter argument for TarFile.add .

We’ll also name it filter . (In some cases “policy” would be a more fitting name, but the API can be used for more than security policies.) The built-in policies/filters described above will be implemented using the public filter API, so they can be used as building blocks or examples. Setting a precedent If and when other libraries for archive extraction, such as zipfile , gain similar functionality, they should mimic this API as much as it’s reasonable.

To enable this for simple cases, the built-in filters will have string names; e.g. users can pass filter='data' instead of a specific function that deals with TarInfo objects. The shutil.unpack_archive() function will get a filter argument, which it will pass to extractall . Adding function-based API that would work across archive formats is out of scope of this PEP. Full disclosure & redistributor info The PEP author works for Red Hat, a redistributor of Python with different security needs and support periods than CPython in general.

Such redistributors may want to carry vendor patches to: Allow configuring the defaults system-wide, and Change the default as soon as possible, even in older Python versions. The proposal makes this easy to do, and it allows users to query the settings. Specification Modifying and forgetting member metadata The TarInfo class will gain a new method, replace() , which will work similarly to dataclasses.replace . It will return a copy of the TarInfo object with attributes replaced as specified by keyword-only arguments: name mtime mode linkname uid gid uname gname Any of these, except name and linkname , will be allowed to be set to None .

When extract or extractall encounters such a None , it will not set that piece of metadata. (If uname or gname is None , it will fall back to uid or gid as if the name wasn’t found.) When addfile or tobuf encounters such a None , it will raise a ValueError . When list encounters such a None , it will print a placeholder string. The documentation will mention why

The rest of this page is on the live site.