Text of peps.python.org › pep-0456 as Styx crawled it on Nov 19, 2026. The page may have changed since. Only its words are kept, without pictures or layout, and only the first part of a long page.

Open the live page

PEP 456 – Secure and interchangeable hash algorithm | peps.python.org

PEP 456 – Secure and interchangeable hash algorithm PEP 456 – Secure and interchangeable hash algorithm Author : Christian Heimes <christian at python.org> BDFL-Delegate : Alyssa Coghlan Status : Final Type : Standards Track Created : 27-Sep-2013 Python-Version : 3.4 Post-History : 06-Oct-2013, 14-Nov-2013, 20-Nov-2013 Resolution : Python-Dev message Table of Contents Abstract Rationale Requirements for a hash function Current implementation with modified FNV Examined hashing algorithms SipHash MurmurHash CityHash DJBX33A Other Conclusion Small string optimization C API additions hash secret hash function definition autoconf hash function selection Python API addition sys module Necessary modifications to C code _Py_HashBytes() (Objects/object.c) bytes_hash() (Objects/bytesobject.c) memory_hash() (Objects/memoryobject.c) unicode_hash() (Objects/unicodeobject.c) generic_hash() (Modules/_datetimemodule.c) Performance Hash value distribution Typical length Grand Unified Python Benchmark Suite Backwards Compatibility Alternative counter measures against hash collision DoS Discussion Pluggable Non-aligned memory access ASCII str / bytes hash collision References Copyright Abstract This PEP proposes SipHash as default string and bytes hash algorithm to properly fix hash randomization once and for all.

It also proposes modifications to Python’s C code in order to unify the hash code and to make it easily interchangeable. Rationale Despite the last attempt [issue13703] CPython is still vulnerable to hash collision DoS attacks [29c3] [issue14621] . The current hash algorithm and its randomization is not resilient against attacks. Only a proper cryptographic hash function prevents the extraction of secret randomization keys.

Although no practical attack against a Python-based service has been seen yet, the weakness has to be fixed. Jean-Philippe Aumasson and Daniel J. Bernstein have already shown how the seed for the current implementation can be recovered [poc] . Furthermore, the current hash algorithm is hard-coded and implemented multiple times for bytes and three different Unicode representations UCS1, UCS2 and UCS4. This makes it impossible for embedders to replace it with a different implementation without patching and recompiling large parts of the interpreter.

Embedders may want to choose a more suitable hash function. Finally the current implementation code does not perform well. In the common case it only processes one or two bytes per cycle. On a modern 64-bit processor the code can easily be adjusted to deal with eight bytes at once. This PEP proposes three major changes to the hash code for strings and bytes: SipHash [sip] is introduced as default hash algorithm. It is fast and small despite its cryptographic properties.

Due to the fact that it was designed by well known security and crypto experts, it is safe to assume that its secure for the near future. The existing FNV code is kept for platforms without a 64-bit data type. The algorithm is optimized to process larger chunks per cycle. Calculation of the hash of strings and bytes is moved into a single API function instead of multiple specialized implementations in Objects/object.c and Objects/unicodeobject.c .

The function takes a void pointer plus length and returns the hash for it. The algorithm can be selected at compile time. FNV is guaranteed to exist on all platforms. SipHash is available on the majority of modern systems. Requirements for a hash function It MUST be able to hash arbitrarily large blocks of memory from 1 byte up to the maximum ssize_t value. It MUST produce at least 32 bits on 32-bit platforms and at least 64 bits on 64-bit platforms. (Note: Larger outputs can be compressed with e.g. v ^ (v >> 32) .) It MUST support hashing of unaligned memory in order to support hash(memoryview).

It is highly RECOMMENDED that the length of the input influences the outcome, so that hash(b'\00') != hash(b'\x00\x00') . The internal interface code between the hash function and the tp_hash slots implements special cases for zero length input and a return value of -1 . An input of length 0 is mapped to hash value 0 . The output -1 is mapped to -2 . Current implementation with modified FNV CPython currently uses a variant of the Fowler-Noll-Vo hash function [fnv] .

The variant is has been modified to reduce the amount and cost of hash collisions for common strings. The first character of the string is added twice, the first time with a bit shift of 7. The length of the input string is XOR-ed to the final value. Both deviations from the original FNV algorithm reduce the amount of hash collisions for short strings. Recently [issue13703] a random prefix and suffix were added as an attempt to randomize the hash values.

In order to protect the hash secret the code still returns 0 for zero length input. C code: Py_uhash_t x ; Py_ssize_t len ; /* p is either 1 , 2 or 4 byte type */ unsigned char * p ; Py_UCS2 * p ; Py_UCS4 * p ; if ( len == 0 ) return 0 ; x = ( Py_uhash_t ) _Py_HashSecret . prefix ; x ^= ( Py_uhash_t ) * p << 7 ; for ( i = 0 ; i < len ; i ++ ) x = ( 1000003 * x ) ^ ( Py_uhash_t ) * p ++ ; x ^= ( Py_uhash_t ) len ; x ^= ( Py_uhash_t ) _Py_HashSecret . suffix ; return x ; Which roughly translates to Python: def fnv ( p ): if len ( p ) == 0 : return 0 # bit mask, 2**32-1 or 2**64-1 mask = 2 * sys . maxsize + 1 x = hashsecret . prefix x = ( x ^ ( ord ( p [ 0 ]) << 7 )) & mask for c in p : x = (( 1000003 * x ) ^ ord ( c )) & mask x = ( x ^ len ( p )) & mask x = ( x ^ hashsecret . suffix ) & mask if x == - 1 : x = - 2 return x FNV is a simple multiply and XOR algorithm with no cryptographic properties.

The randomization was not part of the initial hash code, but was added as counter measure against hash collision attacks as explained in oCERT-2011-003 [ocert] . Because FNV is not a cryptographic hash algorithm and the dict implementation is not fortified against side channel analysis, the randomization secrets can be calculated by a remote attacker. The author of this PEP strongly believes that the nature of a non-cryptographic hash function makes it impossible to conceal the secrets.

Examined hashing algorithms The author of this PEP has researched several hashing algorithms that are considered modern, fast and state-of-the-art. SipHash SipHash [sip] is a cryptographic pseudo random function with a 128-bit seed and 64-bit output. It was designed by Jean-Philippe Aumasson and Daniel J. Bernstein as a fast and secure keyed hash algorithm. It’s used by Ruby, Perl, OpenDNS, Rust, Redis, FreeBSD and more.

The C reference implementation has been released under CC0 license (public domain). Quote from SipHash’s site: SipHash is a family of pseudorandom functions (a.k.a. keyed hash functions) optimized for speed on short messages. Target applications include network traffic authentication and defense against hash-flooding DoS attacks. siphash24 is the recommend variant with best performance. It uses 2 rounds per message block and 4 finalization rounds.

Besides the reference implementation several other implementations are available. Some are single-shot functions, others use a Merkle–Damgård construction-like approach with init, update and finalize functions. Marek Majkowski C implementation csiphash [csiphash] defines the prototype of the function. (Note: k is split up into two uint64_t): uint64_t siphash24 ( const void * src , unsigned long src_sz , const char k [ 16 ]) SipHash requires a 64-bit data type and is not compatible with pure C89 platforms.

MurmurHash MurmurHash [murmur] is a family of non-cryptographic keyed hash function developed by Austin Appleby. Murmur3 is the latest and fast variant of MurmurHash. The C++ reference implementation has been released into public domain. It features 32- or 128-bit output with a 32-bit seed. (Note: The out parameter is a buffer with either 1 or 4 bytes.) Murmur3’s function prototypes are: void MurmurHash3_x86_32 ( const void * key ,

The rest of this page is on the live site.