Mixed-mode debugging is only showing me the C code

Nov 8, 2013 at 2:32 PM
Edited Nov 8, 2013 at 2:33 PM
The "subject" could probably have been clearer, but the issue I'm having is this: I have a python project compiled C plus plus code that I'm trying to debug in mixed mode. If I place a breakpoint somewhere on a python source file, once the program reaches that line I'm being placed at a line in ceval.c (JUMPTO(f->f_lasti);), which I guess means I'm inside the python vm. The call stack shows:
python27_d.dll!PyEval_EvalFrameEx(_frame * f, int throwflag) Line 1054  C
[Native to Python Transition]   
python_d.exe!main(int argc, char * * argv) Line 24  C
python_d.exe!__tmainCRTStartup() Line 555   C
python_d.exe!mainCRTStartup() Line 371  C
kernel32.dll!000000007739652d() Unknown
ntdll.dll!00000000774cc541()    Unknown
...where python_d is CPython 2.7.5, 64 bits. I've disabled "Just My Code" and as far as I can tell the other options are correct - the breakpoint bullet is solid. Is there something I'm doing wrong?

Thanks a lot,

Tiago Nobrega
Nov 8, 2013 at 4:52 PM
Since you see [Native to Python ...] in the call stack, and the breakpoints appear solid (and get hit), this means that the Python runtime is recognized and is, in fact, being debugged - so there's nothing that you're doing wrong, but rather this is a bug in the call stack filter. It is supposed to replace PyEval_EvalFrameEx with the corresponding Python frame, but for some reason it didn't.

I suspect this has something to do with it being a debug build of Python - it should support those as well, but they did get less testing compared to regular builds. Let me see if I can repro this locally on a debug build of the same Python version and get back to you.
Nov 8, 2013 at 5:06 PM
Edited Nov 8, 2013 at 5:26 PM
Thanks for your reply. I changed the interpreter to CPython 2.7.5 (non-debug) 64 bits, and now the proper symbols are being loaded and the breakpoints are being hit and correctly shown (on the python side), but I can't step into C code. Any hints?


PS: I forgot to mention something potentially relevant: I'm using VS 2013 but the Cpp project is a VS 2010 one - it compiles using the "Visual Studio 2010 (v100)" platform toolset.
Nov 8, 2013 at 6:05 PM
I couldn't repro this locally, unfortunately. I wonder if it's specific to the code that you're trying to debug - can you share that and the breakpoint location?

Since you're building your project against release Python now, can you check that you're building it with symbols? If you're using setup.py to build, then it suppresses symbol generation when building extensions in release mode, which is really annoying. If you're using a VC++ project, that would be in project properties, under C/C++ -> General -> Debug Information Format. If it is disabled, that would certainly prevent stepping into your C code, or, indeed, setting breakpoints in it.

Generally speaking, try setting breakpoints in your C code. If it doesn't work for whatever reason (no symbols, source/symbol mismatch etc), then stepping won't work either.

Mismatch between VS version used to build and to debug is fine and should not affect this in any way. In fact, the extension doesn't even have to be built with VC++ - any compiler that generates symbol files in the format that VS understands (i.e. PDB) will do, such as Intel C++ or Intel Fortran.
Nov 8, 2013 at 6:34 PM
I'm using a VC++ project to build it, and indeed I just double-checked that the PDBs are being created (the Debug Information Format option is set to "Program Database (/Zi)").

I can set breakpoints in the C code, yes. I can also my application through the VC++ project by setting "python.exe" as the debugger command, passing they .py as argument. The breakpoints are hit fine then.

In any case, thanks a lot for looking into it. Early next week I'll try to create a small reproduction of my problem, one that I can share.

Thanks again.
Nov 8, 2013 at 6:40 PM
Can you also clarify what exactly are you trying to step into (i.e. is it a plain function call, method call, operator, ctypes call, something else)? And also, what exactly happens - do you just end up on the next line of Python code when you step in?
Nov 8, 2013 at 6:42 PM
It's a method call on an object with bindings created with boost::python. When trying to step into (via the menu or via F10), I indeed end up on the next line of Python code.
Nov 8, 2013 at 6:45 PM
Thanks. I'll see if I can repro that, as well. There's no reason why Boost.Python should be any different - in the end, it's all just C code invoked in the same way - but I wonder if it triggers some corner case that we haven't seen before.

As a side note, F10 is a shortcut for Step Over; Step Into is F11.
Nov 8, 2013 at 6:48 PM
Very true, I wrote F10 but meant F11.
Nov 11, 2013 at 10:47 AM
Update: Over the weekend I experimented with the following setup at home:
  • VS 2013 Premium;
  • boost 1.55 trunk (needs to be trunk here to compile with the VC120 platform);
  • CPython 2.7.5 download through, well, PTVS;
  • Cpp code compiled with Visual C++, Visual Studio platform 120 (the new one);
The Cpp code is the one from boost::python's tutorial:
#include <Python.h>
#include <boost/python.hpp>

char const* greet()
   return "hello, world";

    using namespace boost::python;
    def("greet", greet);
... with this setup, I can use mixed-mode debugging just fine. As a side note, once I step from Python to Cpp I end up somewhere in boost's binding code (it's templates all the way down), but I can manually set a breakpoint to my Cpp code and continue to there. Everything seems to work.

Then I looked into it a bit more and found out that, on the setup I'm having a problem with, the version of CPython we're using has actually been compiled with VC++ 100 (the one from VS 2010), so I guess my setup isn't precisely standard. I apologize for this. Next, I'll try to reproduce the problem in a smaller test case.

Nov 11, 2013 at 3:25 PM
Thank you for looking into this!

Having Python compiled with a different C++ version should also not affect how it's handled. What can break things is if you build it with certain optimization flags. In particular, the debugger really hates PGO - however, it should detect PGO Python builds, and simply refuse to load the runtime in that case...

Regarding stepping through Boost templates, we can't really control it from the Python side of things - from Python debugger perspective, when control crosses over the python##.dll boundary, it's native code, and native debugger is the one to handle it; and as far as native debugger is concerned, it's just as good code as any other, and so it ends the step.

However, in VS 2013, VC++ has added support for "Just My Code", complete with the ability to designate arbitrary code as the one through which stepping should not pause. You might want to try adding the Boost headers to the list as described here, and see if that improves the stepping
Nov 18, 2013 at 11:23 AM
Sorry for taking this long to reply, I just couldn't find time last week to make good on my promise. Anyway, I did have an unexpected turn of events. I had the following setup:
  • VS 2013 with an opened VC++ project which generates a .pyd via boost::python;
  • A (closed) PTVS project with native debugging enabled;
  • Just My Code enabled, Attach To Process debugging with Native + Python as the languages to debug.
In this setup, I launch my python application externally and attach to it from the VC++ project. I place a breakpoint in the C++ code, and once the breakpoint is hit I can walk up the frames, all the way to the python code, and get the PTVS-provided Python view! In addition, if I keep stepping with F10 I eventually end up back in Python, from where I can continue stepping normally. I guess what I'm saying is, I can kind of use Mixed-mode debugging (and it's very, very nice) by starting from a C++ project. I'll keep investigating. I'd also be willing to switch to dev/unstable builds, if you think that would help.

Shoutout to my coworker that pointed out this process.
Nov 18, 2013 at 4:22 PM
I suspect that the difference, whatever it may be, is solely due to the fact that you're attaching rather than launching via Project -> Debug. Attach doesn't really "attach from project" (which is why it still works even if you don't have any projects open at all) - it just attaches to the designated process. Any source files that it opens, it opens directly from disk based on information obtained from the PDBs (for C++) and from frame objects (for Python).

So the problem then is specific to the "launch" code path, and its differences with "attach". I still can't quite imagine what can possibly happen to give the picture that you have described in the first post - i.e. Python breakpoints working, and call stack with python??.dll frames stripped, but with PyEval_EvalFrame not replaced with the corresponding Python frame. I would love to get to the bottom of this. If you can repro the original problem on your smaller test project, can you share that?
Nov 18, 2013 at 4:25 PM
Sure, I'm working on reproducing the problem on a smaller test project specifically to be able to share it. I should have it soon.
Nov 18, 2013 at 4:34 PM
Oh, and regarding dev builds. There is one launch-related mixed-mode bug that is fixed in them:

but this does not sound at all similar to what you're seeing, so I'm doubtful that it is related. Still, it might be worth a try, just to make sure.

There have been some other, likely also unrelated, mixed-mode fixes in dev builds:


And quite a few fixes in other areas (general debugging, project system, virtual envs, code completion) - as well as the beginnings of the new feature work.

The caveat, of course, is that all this is largely untested beyond developer machines and automated tests, so stability will suffer relative to RTM. Having said that, most areas of instability would be the ones that are affected by the new feature work, and this is pretty much exclusively web development so far - so unless you're doing Django development, it shouldn't be too rough of a ride.
Nov 18, 2013 at 4:39 PM
Oh I'm more than willing to take my chances, I'm also very curious about this issue. Should I uninstall the previous version of PTVS before installing the dev build?
Nov 18, 2013 at 4:41 PM
Nope, it will update in-place.

I believe you will need to uninstall the dev build first if you want to roll back to 2.0 RTM, though.
Nov 18, 2013 at 4:59 PM
Wow this is very curious. I installed the dev version from 11/15, and now I can launch the python application from PTVS in Debug (via F5) and once I reach a breakpoint in python, the 'Step-Into' function is skipping over the C++ calls, instead of stepping into the vm. However, if I manually place a breakpoint in the corresponding C++ code (the body of the function I'm calling from python), the breakpoint is hit and I can debug both languages!

While I can debug just fine now, I'll try to reproduce this new behaviour in a smaller project to send to you guys. Thanks a lot!
Nov 18, 2013 at 6:05 PM
once I reach a breakpoint in python, the 'Step-Into' function is skipping over the C++ calls, instead of stepping into the vm.
This would indicate an undesired regression. I wonder, however, if Just My Code has anything to do with it - can you please check whether you still observe this behavior with it turned off?
Nov 19, 2013 at 2:23 AM
I've tried to repro this locally, but on a simple project the stepping does work fine for me. However, I discovered an issue in the current dev build:


I'm going to fix it for the next build. Additionally, it seems that C++ Just My Code does not work all that well in this scenario (though it won't break things):

Nov 19, 2013 at 10:53 AM
I confirm that this happens with Just My Code both enabled and disabled, but I can't promise that this is a regression on your side (I'm afraid I might have changed something in my set up in the meantime, I'm trying to better isolate this). In my opinion it's more likely that I screwed something up with my setup.
Nov 20, 2013 at 6:46 PM
Alright so I created a small sample at home and brought it to work. There's a VC++ project (Win32Project1) and a PVTS project (PythonApplication2): here.

After some fiddling with the different compiler and boost versions (at work it's v100 and boost 1.52, respectively), I managed to generate the hello_ext.pyd file, but when trying to add it as a reference in the PTVS project I get an error: "Cannot add reference to X:\vstmp\Win32Porject1\x64\Debug\hello_ext.pyd: Fatal Python error: PyThreadState_Get: no current thread".

Any ideas?

Nov 20, 2013 at 6:50 PM
According to the latest Boost docs, Boost.Python only supports up to Python 2.2.2b1. PTVS doesn't support anything earlier than 2.5, and mixed-mode in particular only supports Python 2.7 and 3.3.

If you're not using Boost.Python for the bindings then there is another issue, but I'd be quite surprised if Boost.Python is going to work well with significantly newer versions of Python.
Nov 20, 2013 at 6:58 PM
While that page is indeed part of the latest docs, I think it's stale: the changelog shows more info: http://www.boost.org/doc/libs/1_55_0/libs/python/doc/news.html

In particular:
Current SVN
Python 3 support:
All the current Boost.Python test cases passed. Extension modules using Boost.Python expected to support Python 3 smoothly.
According to PEP 3123, when building Boost.Python against Python older than 2.6, the following macros will be defined in Boost.Python header: (

19 October 2005 - 1.33.1 release
The build now assumes Python 2.4 by default, rather than 2.2
Either way, we've been using boost::python for our bindings needs without much issue, though that could certainly be the problem.
Nov 20, 2013 at 7:10 PM
As a side note, adding .pyd as a reference is only useful for the purpose of getting code completion for members defined in that module - you do not need it to actually run or debug the code.
Nov 20, 2013 at 7:48 PM
The dates on that page certainly don't give me much confidence :)

The code that gave you the error message is Python code (we redirected stderr and displayed it), so this probably means you can't import your module right now either. If you can, then we may be using a different Python version (if you have more than one installed) or it may be because we're running in a different working directory. We're also calling dir() and accessing __dict__ members on all exposed types, so it's entirely possible that one of these is not working.

Are you able to try some of these from a normal Python console? You can look at ExtensionScraper.py and BuiltinScraper.py (or in our install directory) to see exactly what we're trying to do.
Nov 21, 2013 at 10:26 AM
I tried to add the .pyd as a reference after I added the pyd's directory to the search path and python couldn't find the module to import. So maybe it's the same issue causing both, or maybe there are two separate issues with my setup...
Nov 21, 2013 at 11:02 AM
Alright I was wrongly linking against python27_d.lib on my .pyd. I fixed that and now I can import the hello_ext.pyd, and now I indeed can't, with the sample file I linked earlier, step into C++ code from Python. Placing breakpoints in the C++ code works, but F11 just skips over the call on the Python side.

Thanks for all the help.
Nov 21, 2013 at 4:32 PM
Thank you! We are a bit busy today :) but I'll try to get back to this and see if I can repro & debug it on your project as soon as possible.
Nov 21, 2013 at 4:39 PM
Of course, take your time. Congratulations, by the way :)
Nov 27, 2013 at 11:11 PM
I can't seem to be able to reproduce it. I used the provided projects as is, only changing include and lib path to point to my local Python install and Boost build. I'm using VS 2013, Python 2.7.5 64-bit, and Boost 1.55.0. Stepping on the line with the call to greet lands me in function_call in Boost sources on the C++ side.

Are you sure that the version of Boost that you're using is built with symbols, and that those symbols are included in your binary? I can imagine that C++ debugger might get confused if the immediate step target is not in the code with symbols, and end up skipping it altogether.
Nov 29, 2013 at 10:52 AM
Thanks a lot for taking the time to try it. Your setup is the same as the one I used at home (where everything works), but is different from the one I'm having the problem with, at work:
  • While I too use VS 2013, everything is compiled with the VS 2010 platform (v100).
  • At work we use Boost 1.52, also compiled with v100. I'm not sure about the .pdbs, I'll check. Did you build the Win32Project1 in Release or Debug?
... I can't in good conscience ask you to build Boost 1.52 (God knows how long that would take), so can you think of anything else I could try on my side? I'll see about getting the pdbs, if they're missing.

Thanks again.
Nov 29, 2013 at 8:19 PM
Ah, I thought that you could repro it at your home configuration as well. I'll try switching to VS 2010 and see if that changes things (I don't think that Boost version matters; anyway, building Boost is not that long if you only build Python lib).

I was building Win32Project1 in Release.
Dec 2, 2013 at 7:10 PM
It also works fine for me with v100 toolset, so symbols, or rather lack thereof, would be my prime suspect at this point.

MSVC has a somewhat messy story with respect to symbols for static libs - by default they are indeed compiled to a PDB, but that is not usually distributed, or even copied around. The established way of handling symbols there seems to be compiling with /Z7 rather than the more common /Zi, which embeds symbols into the .lib itself. Looking at Boost build scripts, it seems that the default is to build with /Z7 in --debug, but not in --release - I wonder if that could be the problem?