Breakpoints are not hit anymore with PTVS 2.1RC

Sep 4 at 1:52 PM
Edited Sep 4 at 1:53 PM
Hello,

I have upgraded PTVS from 2.1 beta 2 to 2.1 RC (Visual Studio 2012). In pure Python debugger, my breakpoints are not hit anymore with this version. More precisely :
  • In some of my python files they are hit once, and never hit again.
  • In some files, they are never hit.
These files do correctly appear in the Loaded modules window. Debugging has always worked fine with previous version of PTVS (I'm a user for about 2 years now), but now it is completely useless, at least for the debugging part.

I cannot tell you if it is better in mixed mode debbuging because it has never worked for me. I have to say that my scenario is quite complicated : custom Python interpreter (embedded) executing a GUI application ( mainloop, events, ... ) written in Python.
But as I said, I never had problems (for pure python debugging, mixed-mode is another story).
Can you please help me, I don't want to get stuck with 2.1 beta 2...

Thanks,
Coordinator
Sep 4 at 6:12 PM
Can you tell more about your configuration - in particular, which Python version is it?
Sep 5 at 9:26 AM
Thank you for your answer. My python version is 2.7.5. The problem appears with PTVS 2.1 RC on VS2012. I tried also on another machine, with the same python version and the problems are the same.

I tried launching the debugger using F5 or attaching to the existing process. In the latter case, the debugger is correctly loading and running, but sometimes after a breakpoint is hit, the debugger suddenly stops for no apparent reason... By stopping, I mean no error dialog is displayed or else but in VS I'm no longer in debug mode.

As I told you in my previous message, my configuration is kind of special because I have a GUI application entirely written in python scripts, with a main script which is the entry point. I use a custom Python interpreter (embedded) containing a lot of C++ modules that can be called from the Python layer (CPU-consuming tasks delegated to C++ kernel). The entry point script is executed by the custom interpreter to launch the GUI application (using PyExec_SimpleFile or something like this...).

Let me know if you need more information, or if you want me to test something.

Thanks
Coordinator
Sep 5 at 9:56 AM
One thing that comes to mind is a weird bug that was reproducible on 2.7.5, but not on 2.7.7. However, that one involved mixed mode debugging. On the other hand, breakpoint implementation in both debug engines is similar in many ways, even if the code is distinct, so perhaps it affects both equally. Can you try updating to 2.7.8 and see if that helps?

It might also be worth looking into why mixed mode doesn't work for you in general. Your scenario should generally be supported - the only sticking point there is whether Python is linked dynamically, and how much have you modified the interpreter source code. If it's stock Python in terms of code, and you're linking dynamically, then it should work, and I'd like to figure out why it doesn't.
Sep 8 at 8:43 AM
I can not update to Python 2.7.8 because this is a lot of work to do. We have a huge number of external dependencies that rely on python so that would mean recompiling almost every of them ... Looking at the bug you are refering to, the "Step over", "Step into", and other debugging navigation does not work for me, and I'm pretty sure this is due to the main event loop involved in our GUI (any of these buttons is equivalent to hitting "F5", so continuing until the next breakpoint). If the main script passed to the custom interpreter is not GUI-related (no event loop triggered), then "Step over", "Step into", ... do correctly work.

About the mixed-mode debugging : it works, but simply the breakpoints in the Python layer are never hit. If I set a breakpoint in the C++ layer, I have a stack that shows frames like "<Unknwown>!PythonFile", "PythonFile" being the correct name of the Python module. Going to the scope of this frame, I have nothing in "Locals" and I am not able to watch anything.
Using F5 to start mixed-mode debugging, all Python files are properly loaded (even if this can take up to 10 min, since a lot of AttributeError exceptions are printed in the Output Window), but when a file containing a breakpoint is loaded, then a message is displayed like "The breakpoint can not be set in file <correct_name_of_python_file>". Same thing when attaching to existing process, even if in this case the loading of Python files is quicker...
Oct 2 at 10:08 AM
Hello,

Just an update to inform you that I still got the problems mentionned above in PTVS 2.1.RC2.

Regarding your previous answer, do you think it is linked to my python version (2.7.8) ? If this is the case, then I'll just wait for the day our IT manager will update to a next version of Python and until then I'll just use 2.1 beta 2.

Did you proceed to some important changes in the way the debugger is working between 2.1 beta 2 and 2.1RC ? There should only be bug fixing in pre-release period rigth? I don't understand why it completely broke from a beta version to an RC :-(. I could have understood from a minor version to another... Anyway, if you think about anything else that could be the source of the problem, I'll be there waiting :-)

Thanks
Coordinator
Oct 2 at 6:07 PM
Ignoring the mixed-mode side of things for a while, there are certainly issues that we fixed with the pure-Python debugger between 2.1b2 and RC2 that may affect you, especially since you aren't running through the normal python.exe.

The first thing to check is whether we're incorrectly detecting standard library code. This can be fairly easily tested by enabling the option to allow debugging the stdlib (Tools->Options->Python Tools->Debugging->Enable debugging of the Python standard library). FWIW, I don't think this is your issue, but it's the easiest thing to check.

Next thing is to check whether any of your code patches thread launching. We have to patch this in our debugger so that we know about new threads, otherwise we can't handle them properly. Also, if you are creating new threads from C code, we won't know about them either. That said, this would have affected you on 2.1b2 as well, so it's probably not the problem here if your current code still works against beta 2 (that is, you haven't made a change to the debuggee that has affected this).

It's also possible that something inside our debugger is throwing an exception, but we have no nice way to handle/report that. With a small modification to visualstudio_py_debugger.py (should be in your install directory) you can extend the output to include debugger errors. Lines 2287-2294 (in RC2) deal with exceptions that terminate the program, basically pausing if requested (do_wait()) and displaying the exception. If you add traceback.print_exc() before the print_exception() call, then you'll get the "uncensored" version of the traceback. If there is a problem within our debugger code, it should show up here.

And thanks for poking us. We want to fix this, but we're very busy right now and it's far too easy for things to slip through the cracks :(
Oct 29 at 12:31 PM
Hello,

Thank you for your answer.
I had the opportunity to investiguate in more details (using official PTVS 2.1 - congratulations for this great release) what is exactly going wrong. But first, I tried the things you pointed out :
  • The Python library is correctly detected
  • We do not patch thread launching so I don't think this is the cause of the problem
  • I added the prints in visualstudio_py_debugger but nothing is printed, so no exception seems to be launched by the debugger.
However, I noticed how to systematically reproduce the problem (breakpoints not hit). This seems linked to vtk calls from Python. Let's consider the following exemple :
  • Module_1 :
def main():
sub_function_1()

def sub_function_1():
//some gui code
//some gui code
//some gui code
import Module_2
Module_2.sub_function_2()
  • Module_2 :
from vtk import *

def sub_function_2():
//some gui code
array = vtkFloatArray()
//some gui code
sub_function_3()

def sub_function_3():
//some gui code
Let's imagine I set some breakpoints : one in main, one in sub_function_1, one in sub_function_2 and one in sub_function_3 (the exact place in the function do not matter for the problem). When pure Python debugging, the breakpoint in main is hit. Pressing F5, the breakpoint in sub_function_1 is hit also. But, pressing F5 again, the breakpoint in sub_function_2 is NOT hit. Visual Studio is still in debug mode, but my GUI application either :
  • Freezes and is not usable anymore (like it is stucked in a loop or waiting to get the hand back from something - the debugger I suppose)
  • Is still usable, but no breakpoints can be hit again (for instance, if calling sub_function_1, the breakpoint is not hit anymore)
What is interesting, is that if I comment the vtk call :
def sub_function_2():
//some gui code
#array = vtkFloatArray()
//some gui code
sub_function_3()
then I'm able to hit all the breakpoints, from main to sub_function_3 ! As many times as I want.

More generally, this means that the debugger is working well as long as I don't set a breakpoint in a function that has some vtk code on it (the place of the breakpoint in the function has no importance for the problem).
I did not have the problem in 2.1b2, I checked on another machine and breakpoints can be reached in parts where vtk code is present.

Hope that will give you extra ideas...
Coordinator
Oct 29 at 5:44 PM
This is very helpful, thank you! I'll investigate.
Coordinator
Oct 29 at 5:46 PM
This discussion has been copied to a work item. Click here to go to the work item and continue the discussion.