Text of discuss.python.org › t › atomic-and-thread-safe-in-python-world › 51575 as Styx crawled it on Nov 19, 2026. The page may have changed since. Only its words are kept, without pictures or layout, and only the first part of a long page.

Open the live page

Atomic and thread safe in Python World - Python Help - Discussions on Python.org

Atomic and thread safe in Python World Python Help allenling (Allen Ling) April 22, 2024, 4:35am 1 i recently read about something related to asyncio, and thread safe just like everywhere, and that led me to what atomic and thread safe really mean in Python. from my shallow understandnigs, an operation is atomic as it takes one instruction to complete, and here the global interpreter is the one executing instructions in Python, and instructions are OP codes like LOAD_NAME, CALL_FUNCTION, etc. and atomic operations are a way to achive thread safety. and thread safety is for exclusive control, basically it means when i am doing something, accessing, modifying something, i have such a guarantee that no one has a chance meddling in, they have to wait. and the GIL provides thread safety to some extent because only one thread can run at a time, but thread switching would still happen in between OP codes. and i found this link and i felt kind of confused about x = L[i] is considered thread safe. let’s take a look at the op codes for L.append(x) , In [72]: dis.dis("L.append(x)") 1 0 LOAD_NAME 0 (L) 2 LOAD_METHOD 1 (append) 4 LOAD_NAME 2 (x) 6 CALL_METHOD 1 8 RETURN_VALUE append takes one instruction to complete, so when there’s multiple threads calling append but still only one thread can acquire the GIL and actually execute append , which is CALL_METHOD . or maybe there’s one thread calling L.append , and another one is calling L.pop() , and pop takes one instruction to complete as well.

In [73]: dis.dis("L.pop()") 1 0 LOAD_NAME 0 (L) 2 LOAD_METHOD 1 (pop) 4 CALL_METHOD 0 6 RETURN_VALUE even though two threads switch between LOAD_METHOD, say thread 1 pauses at LOAD_METHOD , and the thread 2 runs to LOAD_METHOD , and then we back to thread 1, there’s still only one thread holding the GIL in order to execute the op code CALL_METHOD . so there’s always one manipulating the list, either appending data to the list, or popping data from the list by executing op code CALL_METHOD . and here are op codes for x = x + 1 In [74]: dis.dis("x = x + 1") 1 0 LOAD_NAME 0 (x) 2 LOAD_CONST 0 (1) 4 BINARY_ADD 6 STORE_NAME 0 (x) 8 LOAD_CONST 1 (None) 10 RETURN_VALUE suppose x was initialized to 0, and thread 1 is running, and it has finished LOAD_NAME, thread 1 knows that x is zero, and before it exectues BINARY_ADD, thread 2 acquires the GIL, and then thread 1 will be paused, and then thread 2 will be the running thread, and let’say thread 2 will execute LOAD_NAME, and BINARY_ADD before we switching back to thread 1, and then data corrupts. so x = x + 1 is not thread safe. and op codes for x = L[i] In [75]: dis.dis("x = L[i]") 1 0 LOAD_NAME 0 (L) 2 LOAD_NAME 1 (i) 4 BINARY_SUBSCR 6 STORE_NAME 2 (x) 8 LOAD_CONST 0 (None) 10 RETURN_VALUE what about thread 1 pausing before BINARY_SUBSCR but after LOAD_NAME L ? suppose i is was, and there was 3 elements in the list L , and thread 2 might just pop the last element from L , which would result in L being shrinked to a size of 2. then thread 1 resumed and executed L[2], then out of range. alicederyn (Alice) April 22, 2024, 6:26am 2 An operation is atomic if it appears to occur instantaneously.

The number of instructions isn’t strictly relevant in general: the further towards hardware you get, the more you find things that look like a single instruction actually aren’t (e.g. microinstructions, or LL/SC on ARM). Python achieves this appearance of instantaneous action with a global interpreter lock (GIL), which it holds around every interpreter instruction, so your intuition turns out to be correct ( edit : or not, see next post), at least in current versions of Python.

However, it helps to know the underlying idea, as you can achieve atomicity yourself with correct use of mutexes or other algorithms. Or take asyncio: as scheduling is cooperative, not pre-emptive, any block between two await s can be considered atomic, provided you don’t run any background threads accessing the same data. hauntsaninja (Shantanu Jain) April 22, 2024, 6:32am 3 What is and isn’t atomic is indeed quite subtle. (You mention asyncio, that’s a little different and easier to reason about because it’s cooperative.

Unless you await, nothing else will run) I personally think reasoning about it in terms of bytecodes is more confusion than its worth. Sure, if an operation spans multiple bytecodes, certainly don’t treat it as atomic. But CALL is a single bytecode and that’s clearly not atomic! (Also the details of when bytecodes can drop the GIL changes, e.g. in Python 3.10 bpo-29988: Only check evalbreaker after calls and on backwards egdes. by markshannon · Pull Request #18334 · python/cpython · GitHub means a lot of trivial attempts to demonstrate non-atomicity will no longer reproduce in Python 3.10 and newer, but this isn’t guaranteed by the language) I think a way to think about it is in terms of segments of uninterrupted C code.

If you call into C and don’t give up the GIL and don’t call into Python code and don’t decref (since reference count hitting zero can call __del__ ), whatever you do will (probably) be atomic with respect to all other Python code. See also delete misleading faq entry about atomic operations · Issue #89598 · python/cpython · GitHub and PEP 703 – Making the Global Interpreter Lock Optional in CPython | peps.python.org allenling (Allen Ling) April 22, 2024, 9:11am 4 thanks for replies. @alicederyn with regard to the definition of atomicity in programming world, i agree with that many things seem like one instruction but actually might be not. there’s a lot of talks, articles, videos around atomicity, and some people say and someone would refere to something like all-or-nothing , etc, but i guess we can just narrow the scope here. the term instantaneously , IMHO, implies that one-instruction, uncuttable, can-not-be-interrupted property. if one operation can be interrupted, then it can not be perfomed instantaneously since someone can just jump in and make that operation stop(the term instantaneously here does not refer to speed, right?), and that operation can not finish instantaneously and will be finished in future, meaning that operation stops at some smaller instruction. but given that we are talking about Python world, and let’s just put that defination of atomicity a side, and say, roughly speaking, an operation is atomic if it can not be interrupted, at least from a higher level perspective, and the underlying OS or libs would take care of the details for us. i hope that we can achieve an consensus on this, but please correct me if i am wrong. @hauntsaninja yes we can manually release the GIL in C code, and python does release the GIL somewhere at certain point, like handling network events. let’s just don’t include any C code and consider things at Python level only, and for simplicity, make some assumptions here, like, for instance, L.append will not release the GIL(i haven’t digged into the implementation details of append , i hope i am right). and first, let’s say we all agree that in the context of the GIL, thread switching happens among op codes. what i wanted to figure out is that why x = L[i] is thread safe. or could someone explain and do a dry-run or simulation on how the interpreter works and switch two threads step by step to demonstrate x=L.append[i] would be thread safe. i read delete misleading faq entry about atomic operations · Issue #89598 · python/cpython · GitHub , and posted that question, and i hope that someone can help. at last, if there’s so many kind of exceptions in terms of atomicity, maybe we should just simply treat no python code thread safe?

Rosuav (Chris Angelico) April 22, 2024, 9:44am 5 Let’s start with a definition of “thread-safe”. The simplest way to think about it is that, given two threads doing two operations, it is exactly the same as if thread A performed its operation before thread B does, or the other way around. I’ll

The rest of this page is on the live site.