debugging embedded python interpreter: breakpoints fail to bind

Oct 15, 2013 at 3:44 AM
I am trying to debug a python script that is run from a C++ application via embedded interpreter.
I am using VS2012, with PTVS 2.0 RC.
The C++ app is a 64 bit debug build.
I am using python 3.3.2, 64 bit (release build, downloaded from "official" site, and with debugging symbols available).

I start the C++ app, wait for it to load the python interpreter, and then attach the Native code and Python code debugger to it.

I can step the debugger from the C/C++ side and into, and through, the python script no problem.
However, I cannot set any breakpoints in the python script.
When I try to do so, Visual Studio reports "The following breakpoint cannot be set: At, line 522. The breakpoint failed to bind."

Does anyone have any ideas?

Breakpoints work fine for me when debugging python scripts that are run "natively" (ie. not via C/C++ embedded python)
Oct 15, 2013 at 4:47 AM
How exactly do you launch the script from your hosting app? The common problem there is that if you're using PyRun_String or PyRun_File, and don't specify the correct filename parameter - and the Python code object that gets run does not have file information associated with it.

There are some other exotic scenarios which result in a loss of this information even when it's available, but they are far less likely. In particular, in order to set Python breakpoints in a file when attaching, it must either be loaded (as a module, or by any other means that associate the proper filename with the code objects produced) after you attach, or else the module must be present in sys.modules at the moment of attach. If e.g. you import some module and then manually remove it from sys.modules, or you run the contents of some file (directly, not as a module) before attach and stash away references to some functions/classes defined in that file and then try to use them after attach, PTVS will not be able to resolve the running code to filename. When that happens in mixed mode, you should see stack frames labelled as <unknown> in the Call Stack window.
Oct 15, 2013 at 10:21 PM
Hi pminaev
Thanks for your reply, you have given me some things to look at.

I could be in the “exotic scenarios” category? The way I am launching the script from the hosting app is as follows:
1) Host app calls Py_Initialise(). Not sure if this is relevant but I have Py_NoSiteFlag set to 1, so the site libraries are not imported.
2) Host app imports (via PyImport_ImportModule(“sys”)) the “sys” module.
3) Host app the sets the sys.path to a list of directories representing the path that I want python to search for modules. This path includes the directory that my script (the one I’m trying to debug) is in. The path also includes a .zip archive of the parts of the python standard library that I want the embedded interpreter to be able to use.
4) Host app then imports the script using PyImport_ImportModule( filename )
5) Debugger is attached here
6) I set breakpoint in C++ code just before I call out to python script.
7) Function in the python script is called via PyObject_CallObject(), and I can step the debugger through this call and into the python code.
8) Ok, so now I have the debugger paused on the first line of the function inside my script.
I can see the script file name, complete with its path in Visual Studio’s “modules” window.
In the call stack window, it says the current stack frame is!SomeFunction Line 522, and just below that there is a line that says “Native to Python Transition”. That all looks fine.
9) Still can’t set a breakpoint in the python script though. “The breakpoint failed to bind”.

One other observation that may or may not be relevant:
In Visual Studio Output window, after I step into the python function (but before I try to set a breakpoint in the python) I see the following:
'HostingApp.exe' (Python): Loaded '<frozen importlib._bootstrap>'. Module was built without symbols.

There are literally thousands of these “builtins…Error” lines. I wonder where they are coming from.

And finally, the reason why I say I might be in the “exotic scenarios” category…
After I attach the debugger, and before the breakpoint in the C++ code is hit… there is a chance that the python interpreter is unloaded (via a call to Py_Finalise()) and then re-loaded again (as per steps 1..4 above). Could this be preventing PTVS from resolving the running code to a filename (or whatever other reason it could be that the breakpoint cant be bound)
Oct 15, 2013 at 11:23 PM
This doesn't sound particularly exotic, and, as described, should just work. I'm not 100% sure about "recycling" the state with Py_Finalize/Py_Initialize, but I can't see any obvious reasons as to why this should break. I'll do a minimal repro of your scenario locally and see if I can debug this. If I cannot get a repro that way, would you be willing to share your project with us for debugging purposes?

Exceptions are unrelated and normal - basically, it's what gets thrown and handled by Python itself and its import machinery when initializing and loading modules, e.g. when it looks up loader and _path on modules and gets AttributeError when it can't find them. You'll see more of that under mixed-mode debugging, because that also shows all Python exceptions raised in native code, even if they are subsequently handled in native without ever crossing the boundary into Python.

Simply put, it reports a raised exception whenever PyErr_SetObject is called. This is by design, so that if your native code throws when called from Python, it breaks immediately and you get the complete state at the precise point of error, as opposed to only breaking at transition boundary when the entire native part of the stack, with locals etc, is already lost. It can be annoying though, as exceptions are so common in Python for not really exceptional scenarios - it's also why we disable breaking on exceptions by default in mixed mode, though you can still enable it via Debug -> Exceptions.
Oct 16, 2013 at 12:24 AM
I think I might have a repro. Are you using relative paths in sys.path, and does your module show up with a relative path in the Modules window?
Oct 16, 2013 at 1:01 AM
Hmph, I was hoping I was exotic ;-) Thanks for looking into this. I am putting full paths into sys.path, and in the modules window my script does show up with its full path. I would be willing to share the project, but might not be possible in practise because it is a behemoth. I'll see if I can repro with a very small example and then share that.
Oct 16, 2013 at 1:05 AM
I thought that you were hitting what I was observing in Come to think of it, you can still hit that with absolute paths if they are not normalized (e.g. have any . or .. in them), or if there is a case mismatch between what the module thinks its __file__ is, and the actual casing of the path in the filesystem. Can you check for those, too?
Oct 16, 2013 at 1:37 AM
OK, I checked. When I've stepped into the script I look in the Locals window, under "globals" I can see the file variable.
It's value is (almost) identical to the Path reported for the script in the modules window, which is identical to the path to the script in the filesystem itself. There is no case mismatch, or ".."
The only difference is that file says D:\Users\me\somefolder\, and the Path in the modules window says D:\Users\me\somefolder\ (i.e. single backslash instead of double backslash)
Oct 16, 2013 at 1:42 AM
file in previous post was meant to come out as __file__ (I',m just getting used to this markup).
Oct 16, 2013 at 1:45 AM
you know, if you two got on skype & screen-shared you would've found the root cause by now. just saying :).
Oct 16, 2013 at 5:01 AM
So now I have a very small C++ (MFC) application calling python script. Everything appears to work perfectly, including setting breakpoints in the python script.
Next step.. progressively copy the more "exotic" features from the app where breakpoints don't work to try and see what it is that breaks the breakpoints.
Nov 27, 2013 at 3:36 AM
I'm wondering if this can be turned into a tutorial for debugging embedding python in native c++ code ?
Because I'm trying the same steps as freeman2 did without any success. Thanks in advance.
Nov 27, 2013 at 4:32 AM
Check out the video - it's pretty much step by step:
Nov 27, 2013 at 4:54 AM
Edited Nov 27, 2013 at 4:59 AM
Hi pminaev:
I have watched the video many times and it used a python project to step into native code. But my case is the python scripts are put in the C project instead and my main program is a native C code. Do I need to create a python project and put my scripts there so that I can step into the script from c code?
Nov 27, 2013 at 5:28 AM
Hi stevenmbiz.
No, you don't need a python project. You need to start your native C++ app without the debugger, and then attach the debugger to it after it has initialized the python interpreter.
Nov 27, 2013 at 5:34 AM
By the way, can you give a brief description of what is actually going wrong?
Nov 27, 2013 at 6:16 PM
Hi freeman2:
What do you mean attach the debugger? I'm using attach to process under the debug menu then select Microsoft.Pythontools.Attacher.exe and my main program.exe to attach. However, I cannot set any breakpoints in the python script. When I try to do so, Visual Studio reports "No symbols have been loaded for this document". Any clue for this? Tks.
Nov 27, 2013 at 6:30 PM
You do not need to attach to Attacher.exe - it's a helper process that performs the attach when attaching to 64-bit Python. You only need to attach to your main .exe, and make sure that you have both Native and Python code types selected in the dialog.

(Note that if you only need to debug Python, you can select just the Python code type - you won't be able to debug the native side of your program that way, but Python debugging will be richer in functionality.)

You don't need a Python project for your .py files.
Nov 27, 2013 at 7:49 PM
I've done as you suggested but the problem remains. What does it mean by "No symbols have been loaded..." if the attach was successful?
BTW My python script is a multi-threaded program in which call back functions are created.
Nov 27, 2013 at 7:57 PM
When this message occurs on a breakpoint, it means that VS is unaware of this particular document being loaded in the process, but the reasons can be many. It may be because Python debugging did not properly initialize, e.g. because you're using an unsupported Python version. Or it may be that the script was loaded in such a way that it lost its association with the filename from which it was loaded, which will make it impossible to set breakpoints in there.

Can you provide your VS version and Python version (including bitness, debug/release build, and whether it is from or a third-party distro or your own build)?

Also, try breaking into the running process via Debug -> Break All, and then take a look at the call stack. If you see python??.dll there, this means that Python debugging is not active at all. If you do see Python frames, then what do they look like - in particular - do they have meaningful .py filenames?
Nov 28, 2013 at 12:47 AM
I checked the call stack and did not see my python27_d.dll there. But, in the output window, I noticed that the following report was shown:
'QtMMO.exe' (Python): Loaded 'C:\Qt\MMO12\Scripts\ctypesTest.pyc'. Module was built without symbols.
Is it the reason I got the message "No symbols have been loaded..."? My python version is 2.7.1 32 bit debug build, and it's from download.
Nov 28, 2013 at 4:32 AM
No, this message is normal (Python modules don't have symbols, they are self-describing; only native modules need symbols). Furthermore, if you see this message in Output, then it means that Python debugger is in fact active.

Can you describe how you load and run Python code in your native application? A snippet of C++ code that does that would be ideal.
Nov 28, 2013 at 5:19 AM
Edited Nov 28, 2013 at 5:27 AM
The way I am launching the script from the hosting app basically is same as freeman2 described and as follows:

  const char *scriptDirectoryName = strcat(execpath, "\\Scripts\\");

  PyObject *pDict, *pClass;
  PyObject * pFunc   = NULL;
  PyObject *path = PyString_FromString(scriptDirectoryName);
  PyObject *_ctypespath = PyString_FromString("c:\\Python27\\DLLs");
  PyObject *sysPath = PySys_GetObject("path");

  int result = PyList_Insert(sysPath, 0, path);
  result = PyList_Insert(sysPath, 0, _ctypespath);
  pModule = PyImport_ImportModule("ctypesTest");  
     pDict = PyModule_GetDict(pModule);

     // Build callable class name 
     pClass = PyDict_GetItemString(pDict, "MyThread");

     if (pClass == NULL || !PyCallable_Check(pClass))
        gameEditor->Log("The class \"MyThread\" is not callable");
     // Create instance
     pInstance = PyObject_CallObject(pClass, NULL); 
     if (pInstance == NULL)
        gameEditor->Log("Failed to create the class instance \"MyThread\"");
    PyObject_CallMethod(pInstance, "start", NULL);
    int i = 0;
        if(i == 0) {
            gameEditor->Log("Wait for the python modules.");
        } else
        // !!!Important!!! C thread will not release CPU to Python thread without the following call.
        PyObject_CallMethod(pInstance, "join", "(f)", 0.0001);       
  } else {
The python script I'm testing is as follows:
from ctypes import cdll
from Player import *
from Debug import *

import threading
from pdb import run

# CallBackDispatch() wants a callback function with the following C prototype:
#     typedef int (*FLUFFYBUNNY_CALLBACK)(void *context);

# Get the function from the library. Its C prototype is:
#     int RegisterMainCallBack(DISPATCH_CALLBACK cb);
# Note that I use "py_object" instead of "c_void_p" for the context argument.
RegisterCB          = dllMain.RegisterMainCallBack
RegisterCB.restype  = c_int
RegisterCB.argtypes = [DISPATCH_CALLBACK]
RegisterPlayer = dllMain.registerPlayer
EndGame = dllMain.endGame
EndGame.restype = c_bool

# Create Python version of the callback function.
def _private_Dispatch(obj):
    # Convert the object argument to a py_object, and extract its value.
    # This gives us the original Game object that was passed in to 
    # FluffyBunny().
    gameObj = cast(obj, py_object).value
    # Do something with the context object.
    if gameObj:
        if gameObj.className == "Player":
        elif gameObj.className == "Item":
            print 'Item'
        # Return non-zero as signal that FluffyBunny() should terminate
        return 0 #if (furball.count < 10) else -1
        return -1

# Convert it to a C-callable function.
Disaptch = DISPATCH_CALLBACK(_private_Dispatch)
EditorLog = GameEditor()

class MyThread(threading.Thread):
    def __init__(self):
    def run(self):
       EditorLog.Log('Python thread start...')
       player = Player()
       result = RegisterCB(Disaptch)
       while(not EndGame()):   # call mainLoop function ()!
if __name__ == '__main__':
    print 'Create and run MyThread'
    background = MyThread()
    print  'Main thread continues to run in foreground.'
    print  'Main thread joins MyThread and waits until it is done...'
    background.join() # Wait for the background task to finish
    print  'The program completed gracefully.'
Nov 28, 2013 at 6:42 AM
If you try to step in on various calls that will end up in your Python code - e.g. PyObject_CallObject or PyObject_CallMethod - does the stepping work? And if it does, then what does your call stack look like when you're stepped, and what is the value of __file__ on the topmost frame?
Nov 28, 2013 at 6:12 PM
I turned my callback mechanism into using PyObject_CallObject and PyObject_CallMethod and the stepping works now! Later I'll check if the callback is really needed for my project. I'm wondering if the additional attach process can be avoided in next release of PTVS? Anyway thanks for all your prompt reply and the tool really helps me a lot for my project.
Nov 28, 2013 at 6:19 PM
Oh, does this mean that the script works once you removed "import ctypes" from it?

Attaching is going to be necessary so long as the primary project (the one that you launch) is a C++ project - we don't control that project system, and they don't include the ability to select arbitrary debug engines.

However, what you could do is add your binary as a custom interpreter, and then set up your Python project to run that interpreter on F5, while checking the "Enable native debugging" box.
Nov 29, 2013 at 4:18 AM
Actually, the problem did not come from import ctypes but the way I call my python method. I used the function pointer to call python method as follows:
A method I was experimenting with before and I'll try alternative way to do that later.