Not able to load breakpoints (debug) if the startup file executes other python files in project

Jan 8 at 5:53 PM
Hi,

Here is the configuration :
  • Win 7, 64 bit
  • VS 2012
  • Latest PyTools
  • IronPython 2.7.4
I am trying to write BDD scenarios using "behave" framework (had to patch it make it work with ironpython though).

In "behave" here is the convention that is used :

root
steps
 -- mypythonfile.py
myfeature.feature

you go to "root" directory and issue following command

ipy <location_of_behave>\behave

Essentially your startup script is main.py in behave.

Now in VS2012 ironpython project I make "Startup file" to be <localtion_of_behave>\behave and it is able to execute the steps as I want however none of my breakpoints in mypythonfile.py are hit and studio says brekpoint will not be hit as no symbols have been loaded for it.

I then added Main.py to my project and made it a startup file and I can put the breakpoints in this file. Inside main.py I did behave.main.main() which has the same effect as previous i.e. able to execute my steps but not able to hit any breakpoint in mypythonfile.py.

Please help

Regards & thanks
Kapil
Coordinator
Jan 8 at 7:10 PM
I tried to repro this with regular Python 2.7, but it seems to be working fine there. Can you share the changes that you have made to behave to make it run with IronPython?

There are also some things that I'd like to clarify.

When you say that you "made "Startup file" to be <localtion_of_behave>\behave", how did you do that? The startup file property for the project points to some file in that project - do you mean that you have used Add Existing File ->Add as Link to add __main__.py from behave to the project, and then right-click -> Set as Startup File?

In your VS project, if you go to project properties, Debug tab, what is the launch mode? If it's set to "IronPython (.NET) Launcher", try setting it to "Standard Python launcher" (you won't be able to debug other .NET languages in that mode, only Python code; but it will give you a much better debugging experience in all other ways).

Also, try doing the following. In mypythonfile.py, in one of the functions, add a line of code to raise some exception that does not normally occur (e.g. divide by zero). In Debug -> Exceptions, set that exception to break when thrown (the default should be breaking only on user-unhandled). Then run your test, and when the debugger breaks with the notification of exception, look at the callstack - does it name your module correctly?
Jan 8 at 7:38 PM
Hi Pminaev,

Thanks for taking the time to look at my problem.
Can you share the changes that you have made to behave to make it run with IronPython?
Main Changes are around adding 'cli' as a check whereever sys.platform is used to determine the platform. The one change is in runner which is related to differences in traceback behavior betwee cpython and ironpython. I am going to fork the behave repository and push the changes in there. I must say that sys.platform is very annoying and is the main cause for breaking the compatibility with IronPython
When you say that you "made "Startup file" to be <localtion_of_behave>\behave", how did you do that?
I simply put <localtion_of_behave>\behave as the value. Since "behave" directory has main.py in it, it is picked by the python (or ironpython).
do you mean that you have used Add Existing File ->Add as Link to add main.py from behave to the project, and then right-click -> Set as Startup File?
Since above was not working; just to try the normal mode. I created in my project main.py file (note it is not main.py) and add following lines to it and made it the startup file. But this does not change anything.

<content_of_main.py>
import behave
import behave.main

behave.main.main()
</content_of_main.py>

Tried changing to Standard Python launcher but did not help either. Same issue.

Since I am able to put the break point at line "behave.main.main()" I stepped in its code and here is what I have observed which may be the reason the breakpoint is not hitting.

Behave discovers the python files in the script folder and executes them using either execfile or exec and hence Visual studio is not picking up the file.

Regards & thanks
Kapil
Coordinator
Jan 8 at 7:58 PM
Edited Jan 8 at 7:59 PM
When trying with regular Python 2.7, I tried both setting behave's __main__.py as the startup file, and using your alternative of writing my own entry point script that just imports behave.__main__ and calls its main function. Both resulted in breakpoints being hit in my steps file, so I suspect that this has something to do with how IronPython implements exec/execfile.

Now, exec/execfile should not be a problem in and of itself, provided that the file name is correctly provided to the Python interpreter so that it can associate it with the code objects that debugger then works with. Basically, if you see the correct __file__ in your module when running (try printing it out!), it should work. OTOH, if __file__ is messed up, that would explain why debugger doesn't pick them up - and the problem would then be in figuring out why it goes wrong.

In addition to printing out __file__, can you also try the trick with forcing the debugger to break inside your steps code by raising an exception that I have described earlier in this thread? It would expose the call stack as the debugger sees it, which would help diagnose the problem.

By the way, to avoid CodePlex formatting issues with double-underscore names (which it interprets as bold formatting), you can surround identifiers in backticks - i.e. __main__ instead of main.
Jan 8 at 8:08 PM
Tried printing __file__ and it does print the correct path to my modules.

So if it is working with Python 2.7 then it is really IronPython's execfile implementation that is troublesome. Unfortunately I do have to use IronPython as my python scripts invoke lot of C# libraries.

Any other suggestions.

Regards
Kapil
Coordinator
Jan 8 at 8:33 PM
Edited Jan 8 at 8:33 PM
Can you try obtaining the call stack using the exception?

Also, is the file name that you see in __file__ an absolute or relative path? Normally those are supposed to be absolute, but this is much less certain with exec - I wonder if you somehow end up with a relative path there. If that's the case, then it might be that this triggers https://pytools.codeplex.com/workitem/1695.

I will also try to repro this locally with IronPython, but for that I'll need the changes that you've made to behave. Can you please email the changes (just the changed files, or diffs if you prefer) to ptvshelp@microsoft.com?
Jan 8 at 8:39 PM
'file' printed absolute path.

There is an update/progress though. Based on your first recommendation I had set the launcher in Debug configuration to be "Standard python launcher" but it did not work at that time. I restarted my IDE just now and it started to work i.e. I can hit the breakpoints in python code when using this launcher. As expected using this launcher I can't do mix mode debugging.

Is this the limitation when using execfile or it should work with IronPython launcher as well ? I do have a significant code base in C# that could use the debugging. Unless you tell me that it is not possible to use IronPython launcher I think the problem is solved by using the Standard python launcher.

Sending you my behave package by email.

Regards
Kapil
Coordinator
Jan 8 at 8:58 PM
Ah, that explains it. I've just tried a very simple app with execfile on IronPython:
# main.py
print('before')
execfile('module1.py')
print('after')
raw_input()

# module1.py
print('ok')
1 / 0
And I can already observe the problem. The breakpoint in module1.py is not hit. Furthermore, when 1/0 raises an exception, the call stack simply does not include any frame that can be mapped to module1.py.

It seems that the problem here is with IronPython implementation of execfile not correctly propagating the source file information to the .NET debugger. Note that, when running in IronPython launcher mode, the debugger that you're using is the standard .NET debugger, which PTVS does not control. I would suggest filing an issue against IronPython in their tracker.

Meanwhile, you might want to consider regular Python with pythondotnet module to call into your .NET code, and using our mixed-mode debugging feature to debug that. It would require you to use VS 2013 (the managed debugger in 2012 cannot handle arbitrary mixing of debug engines, it only supports mixed with native), but you can use the PTVS Integrated installer to get it for free.