Text of discuss.python.org › t › adding-finalizer-to-the-threading-library › 54186 as Styx crawled it on Nov 19, 2026. The page may have changed since. Only its words are kept, without pictures or layout, and only the first part of a long page.

Open the live page

Adding “finalizer” to the threading library - Ideas - Discussions on Python.org

Adding “finalizer” to the threading library Ideas BernieHuang2008 (Jinyuan Huang) May 25, 2024, 3:28am 1 Well, as far as i known, the current solution of doing some cleanup stuffs after a thread ended, is to start a new monitor-thread. But this is limited. E.g. in sqlite3, you can’t use or close a connection object created in other threads. So this way of cleanup can’t be used. As the result, we can only create a subclass of the threading.Threading , and modify the Thread.run() method to do the cleanup job.

Here’s an example: class MyThread(threading.Thread): def __init__(self): ... def run(self): super().run() cleanup(...) I wonder if we can impliment a cleanup api in the threading library. It will surely be a great help in some situations. 1 Like Rosuav (Chris Angelico) May 25, 2024, 4:12am 2 The run() method doesn’t really do much. You can just as easily put whatever code you need directly into your own run() method and not bother calling super.

Having a callback on a thread would lead to some confusion as to which thread runs that callback. Is it run in the thread itself, or in the thread that set the callback, or in the main thread? All three could be useful. For the semantics you’re describing, where the callback happens in the same thread that ran the original function, overriding run() is easily the best way to do it. (You may want to stick your callback into a try/finally to ensure that it happens even if your main thread function bombs.) If the thread were to grow an actual callback, I would expect that it happens in the caller’s context - something like this (pseudocode): t = Thread(target=some_func) t.start() ... ... t.join(); callback() but without explicitly joining.

However, this sort of thing is easy to get wrong, and if I found myself wanting this, I would most likely reach for ThreadPoolExecutor or some other such API instead. 2 Likes elis.byberi (Elis Byberi) May 25, 2024, 8:18am 3 add_done_callback(*callback* , *** , *context=None* ) : from concurrent.futures import ThreadPoolExecutor def task(): # Perform some work return "Result" def callback(future): print(f"Callback got: {future.result()}") executor = ThreadPoolExecutor(max_workers=2) future = executor.submit(task) future.add_done_callback(callback) executor.shutdown(wait=True) Would you consider moving this thread to Python Help ?

It will receive much more exposure there. storchaka (Serhiy Storchaka) May 25, 2024, 11:13am 4 This is a good idea, and we actually need this to solve some issues in ThreadPoolExecutor (see concurrent.future.ThreadPoolExecutor should parameterize class used for threads · Issue #89502 · python/cpython · GitHub ). There is a way to execute the user code when initializing a new thread in ThreadPoolExecutor , but there is no way to execute the user when finalizing it, therefore resources acquired in the initializer are leaked ar depend on the garbage collector to be released.

It is less needed in the explicitly created threads because you can simply use try / finally in your target functuin or Thread.run method, it would be just a convenience. It would be great to have something like atexit.register() or TestCase.addCleanup() to register a cleanup code executed when the thread is finalizing. There are only two questions: Where to place it? Should it be the threading function (applied to the current thread) or the Thread method?

How to name it? We can also add a convenient function/method for context managers, like ExitStack.enter_context() and TestCase.enterContext() . 2 Likes BernieHuang2008 (Jinyuan Huang) May 25, 2024, 11:31am 5 Currently, i can use a thread local object ( threading.local() ) to try to do some cleanup works. I write a special class which only has a __del__() method. When the thread is terminated, the __del__() method will be executed, and it will do some cleanup.

However, according to my experiment, the __del__() method is executed by the grabage collector, which is in the main thread, and this is the inconvenient point for me now. 1 Like pf_moore (Paul Moore) May 25, 2024, 1:12pm 6 Agreed this would be nice. It’s not something I’ve had much need for in practice, but it’s a logical addition to the API, and as you say makes cleanup of thread pool workers easier. A thread method is more flexible, as you can always do threading.current_thread().add_finalizer() .

And while I don’t have an actual use case, there may be reasons to want to add a finalizer to a thread that isn’t the current one. I don’t have strong feelings on the name or API - I went for add_finalizer (with a corresponding remove_finalizer ) on the assumption that the API would be like atexit , managing a “stack” of finalizers, but that may be more complexity than we actuallyu need. 1 Like BernieHuang2008 (Jinyuan Huang) May 26, 2024, 11:56am 7 That’s a good idea, add_finalizer() , pretty clear.

Btw, does anyone known how can i change the title of this topic?? pf_moore (Paul Moore) May 26, 2024, 12:10pm 8 There should be a little pencil icon by the title on the first post: image 917×95 6.38 KB MegaIng (Cornelius Krupp) May 26, 2024, 12:30pm 9 You can’t, you don’t have the necessary privileges . Specifically, you can only edit your own post for 24 hours after creation. If you tell us what the title should be, we can still do that.

BernieHuang2008 (Jinyuan Huang) May 26, 2024, 12:36pm 10 Adding “finalizer” to the threading library How do you think of this? It will be appreciated if you can do so. csm10495 (Charles Machalow) May 26, 2024, 7:23pm 11 Would it be valid to add a finalizer to a different thread? … what if that thread is already in progress (or in the middle of processing other finalizers… or just completed?.. can a finalizer add another finalizer?) 2 Likes BernieHuang2008 (Jinyuan Huang) May 27, 2024, 4:39am 12 Good question!

In my opinion, If the thread is in progress, it shall be able to add new finalizers, since it may open some resources that needs to be free up. Otherwise, if the finalizer can only be added before the thread starts, then why not just use a try/finally? However, inside the finalizer, adding another finalizer is meaningless, since itself is supposed to do all the clean-up works. csm10495 (Charles Machalow) May 27, 2024, 5:35am 13 I had figured there could be more than one finalizer.

Kind of like how there can be multiple atexit s. In that world I assumed that a finalizer could add another finalizer if they were in FIFO order. Is there just one finalizer allowed? Would there be a getter for it too? One other random thought is that it might be nice to have a symmetric API on the multiprocessing side. storchaka (Serhiy Storchaka) May 27, 2024, 8:47am 14 Supporting a queue of finalizers is more converninent.

You can imitate this with an ExitStack, but you need to have control on all finalizers, it is not so easy if some finalizer was already added and you need to add another. So it is better to support this from the start. I do not think that there is a need to remove finalizers. This can create new problems due to breaking the FIFO order. Context managers in many cases can be imitated by add_finalizer(cm.__exit__, None, None, None) .

But there is more here. The context manager can work differently depending whether the exception was raised, and it can silence the current exception, so we may need a separate API for context managers if we want to support this feature. It is easy to implement support of finalizers in the thread created in Python, but there may be problems with other threads (the main thread and threads created in extensions). So this feature may be limited. keita (Keita Oda) August 28, 2024, 1:54pm 15 It’s not something I’ve had much need for in practice Hi, I recently came across a case that I really need this thread_finalizer for ThreadPoolExecutor.

I have a web service with Django and MySQL, and nightly it recreates the whole database from several CSVs. Some tables have a few

The rest of this page is on the live site.