g_thread_get_ident = thread.get_ident

def start_embedded_debugger(..., get_ident = thread.get_ident):
    global g_thread_get_ident
    g_thread_get_ident = get_ident
    ...

Then replace all (~10) calls to thread.get_ident with g_thread_get_ident. If this done properly and well tested, it may be useful as a patch to Winpdb.

Yes, our C code does create a thread state structure for each thread and does use the Python GIL properly. But Winpdb uses thread.get_ident() to detect, display and switch between threads. And thread.get_ident() returns the same value for all of my C threads, even though they have separate thread states. So without my hack, Winpdb will only debug the first thread in which I have called rpdb2.settrace(). It ignores the others. Apart from that limitation, there is no functional bug.

I do not expect to break your modification in the near future.

What is the situation now?
Can Winpdb switch between your C threads?

Do you conform to the following documentation regarding C threads?
http://docs.python.org/c-api/init.html

“It is important to note that when threads are created from C, they don’t have the global interpreter lock, nor is there a thread state data structure for them. Such threads must bootstrap themselves into existence, by first creating a thread state data structure, then acquiring the lock, and finally storing their thread state pointer, before they can start using the Python/C API. When they are done, they should reset the thread state pointer, release the lock, and finally free their thread state data structure.”

If so, once you create the thread state, is it still missing a Python thread identifier?

Sorry for that. The idea was to discuss more generally Winpdb’s support for other threading possibilities than only “thread” and “threading”. Using my project only as an example.

It would be great if there were an “official” way to instrument a thread package to make it visible to Winpdb, or maybe better, to derive something from Winpdb that understands my threads.

We are using Python embedded in a C application and we use C threads not Python threads (we don’t use the thread or threading modules). Instead the multi-threaded C can call into Python code and Python code can yield. Threading is non-preemptive.

When I first tried debugging this with Winpdb, it worked functionally OK except that Winpdb could not list the threads or switch between them. It was just able to see the context of the thread currently waiting at a breakpoint.

After a bit of hacking I was able to improve the situation. I replaced the get_ident() method of Python’s thread module (also the reference to it in threading) with a new method that detects my C -created threads and returns their identifier. I’ve included the function below. I also replaced threading._DummyThread in order to provide the right thread names to Winpdb.

My question is whether this is good practice or a nasty hack. Are you planning to provide more support for this sort of thing in the future? Is it likely that future versions of Winpdb will break my hack?

Many thanks.

import rpdb2

# set of Python threads which are base-threads for C thread creation
# filled in during thread creation
# (this enables us to avoid changing the ids of threads used by Winpdb)
sc_elab_threads = set()

def get_ident():
“replacement function for thread.get_ident() which is gs-aware”
tmp = orig_get_ident()
if tmp in sc_elab_threads:
# this gets the ID of the C thread
return gs.gsp_sc_get_curr_process_handle()
else:
return tmp

# replace things in thread and threading modules
# note that threading tries to keep a reference to original thread.get_ident()
orig_get_ident = rpdb2.thread.get_ident
rpdb2.thread.get_ident = get_ident
rpdb2.threading._get_ident = get_ident