Multiprocessing and breakpoints

Aug 13, 2012 at 9:37 AM

I'm trying to debug a multi-threaded script and cannot set a breakpoint in the callback function.

from multiprocessing import Pool

def f(x):
    return x*x # cannot set breakpoint here!

if __name__ == '__main__':
    pool = Pool(processes=4)
    print pool.map(f, range(10))
    exit(0)

Is there a way to get around this? I know I can set a breakpoint using pywin's debugger by calling brk(). Perhaps there's a similar mechanism in pytools?

Coordinator
Aug 16, 2012 at 11:06 PM

The problem here is that you're spawning new subprocesses and the debugging doesn't automatically flow to those subprocesses.  You can do Debug->Attach to Process to attach to the subprocesses once they're up and running.  Or you can use ThreadPool instead of Pool to run everything in the same process.

Aug 29, 2012 at 2:31 PM

I am also using multiprocessing and am attaching to the spawned processes after they start, however, after I attach, the interface locks up and I cannot

reach the breakpoints I have set. In other words, the debug interface is not enabled.

 

I'm stuck.

Coordinator
Aug 29, 2012 at 3:12 PM

This feature (when added) will help for this case: http://pytools.codeplex.com/workitem/536

Our F5 debugger (as opposed to Attach to Process) runs a different script that connects to the debugger before running your script. The multiprocessing package doesn't know about this, so when it starts new Python processes it doesn't use our script.

For now, switching to a thread pool while debugging will work best.