How to debug IronPython?

Jan 14, 2015 at 6:05 PM
I have an application which hosts ironpython for some scriptable customization, and now I'm trying to figure out how to elegantly debug one of these scripts. If I use the "IronPython (.NET) launcher" I can hit breakpoints in the script, but the watch/local/variable inspection only contains a bunch of incomprehensible jibberish (tupleArg, $gotoRouter, $updException etc) and does not allow me to actually inspect my python variables. Additionally the debug interactive window (REPL) just doesn't work at all. I don't need to step across language boundaries (into the c#) or anything like that so I hoped the "Standard Python Laucher" would work...but with it my breakpoints don't event get hit. Is this the expected behavior? If so, am I wrong in concluding the pytools is effectively useless for debugging ironpython scripts?

I am using Visual Studio 2010 with pytools 2.1, ironpython 2.7.4
Jan 14, 2015 at 6:32 PM
Standard Python debugger should work exactly as you expect it to (i.e. variable inspection, call stack etc all show properly, but you can't step into another language). Can you file a bug?
Jan 14, 2015 at 6:44 PM
Also, I can't seem to be able to repro this on a simple IronPython console app - can you tell a bit more about what kind of code it is, and where you are setting breakpoints?
Jan 14, 2015 at 6:48 PM
Oh, one thing that I've missed - you're hosting it. So are you attaching to your host via Attach to Process, or have you registered your own host as a custom Python interpreter?

One other thing of note when hosting. For pure Python debugging to work with IronPython, you need to enable frame reporting and tracing. On command line, this is -X:Frames -X:Tracing switches. I'm not sure how to specify the same using the API, but this seems to describe something similar.
Jan 14, 2015 at 11:02 PM
I have registered my own host as a custom Python interpreter. At this point I'm just trying to get a simple POC up and running just to prove that it can be done in this manner. Adding the "Frames" option actually gets the .NET launcher mostly working (I can inspect some variables, not globals tho), but then adding the "Tracing" flag totally breaks it (no longer even breaks at breakpoints). I have yet to hit a breakpoint using the Standard Launcher using any combination of flags. It is encouraging to hear that this theoretically should be possible :) If you want to try to repro what I'm doing here is my simple python script:
foo = "hello world!"
print foo
def DoPrint(goo):
    print goo
and here is my POC host (which is configured as a custom interpreter:
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Windows.Forms;
using Microsoft.Scripting;
using Microsoft.Scripting.Runtime;

namespace POC
    static class Program
        /// <summary>
        /// The main entry point for the application.
        /// </summary>
        static void Main()
                var args = Environment.GetCommandLineArgs();
                var script = args[args.Length - 1];
                var options = new Dictionary<string, object>();
                options["Debug"] = ScriptingRuntimeHelpers.True;
                options["Frames"] = ScriptingRuntimeHelpers.True;
                //options["Tracing"] = ScriptingRuntimeHelpers.True;
                var scriptEngine = IronPython.Hosting.Python.CreateEngine(options);
                var compiledCode = scriptEngine.CreateScriptSourceFromFile(script).Compile();
            catch (Exception e)
Jan 15, 2015 at 4:15 AM
When you're using the default launcher, it expects to be dealing with something that looks like a Python interpreter, from command line perspective. It will set it up to load the bootstrap debugger script that registers all the hooks necessary to intercept things like thread creation or tracing (the latter is used for breakpoints and stepping), and then loads your code. This is all passed as command line arguments - if you change your Main to take string[] args, you'll see what ends up there. Since you're basically skipping all that, there's no debugging :)

If your host is really meant to be a custom interpreter, then you probably want to handle those command line options and pass all the relevant ones to IronPython. Otherwise (which is usually the case), I would suggest using Attach to Process and ptvsd instead - look at the "Remote Debugging" section in the documentation, it explains how to set things up.

(Note that Attach to Process without ptvsd will not work with IronPython - it is CPython-specific, unfortunately.)
Jan 15, 2015 at 4:52 PM
Great! This helps a lot. I am able to connect to my ipy-hosted script, and the debugging experience is excellent, full callstack, variable inspection, REPL, it's wonderful.....however I am still unable to connect when hosted in my POC host. The script appears to run successfully - the ptvsd module imports successfully, enable_attach() completes, wait_for_attach() blocks.....the port (5678) is listening, but when I attempt to attach nothing happens - the proc never shows up in the 'Available Processes' box. No errors are bubbling up, nothing is written to the console....I'm not really sure what to try next....
Jan 15, 2015 at 8:58 PM
After you typed the URL in the Qualifier box in the Attach to Process dialog, did you press Enter? That dialog is funky like that - merely switching the focus away won't actually refresh anything, but pressing Enter does that (also note that if you click "Refresh", it will reset the qualifier, so that is also not the same thing).
Jan 15, 2015 at 9:12 PM
Yes, I did press enter. I follow the exact same steps as when using ipy.exe (where it works). The cursor turns to hourglass for a second or 2, then turns back to the normal pointer but nothing is displayed in the 'Available Processes' box.
Jan 15, 2015 at 9:32 PM
Just trying to debug this a little bit....I've opened up, and it does connect, and queries for "INFO", and the server responds with "IronPython 2.7.4 (cli 32-bit)", then the client does the exact same thing (queries for INFO again), server responds the same way, and then that's it...
Jan 15, 2015 at 9:36 PM
I just repro'd it. A very amusing bug, in fact. When you're hosting the script, sys.executable is None, and so the process name is reported as <python>. Now the problem is that apparently VS is not happy about processes with names containing special characters like "<" and ">", and so it doesn't display it in the list.

To fix, open and find this line:
exe = sys.executable or ''
and change the empty string to '?' or something like that. Should work then.

Thanks for reporting this! I think it can also get broken for the same reason when remote debugging on Unix if a file name ends up containing some characters that are valid in Unix filenames, but not in Windows ones. Either way, definitely a bug to be fixed.
Jan 15, 2015 at 9:38 PM
This discussion has been copied to a work item. Click here to go to the work item and continue the discussion.
Jan 15, 2015 at 10:11 PM
Heheh, I had actually just found that out on my own too, and am now able to attach, however there does seem to be an additional issue. After attaching I have variable inspection/callstack/breakpoints however the REPL does not work. When I type something in the REPL I press enter, the cursor moves to the next line, my pointer turns to hourglass, and it hangs just like that....

Thanks for all your help, this is awesome
Jan 16, 2015 at 2:28 PM
Edited Jan 16, 2015 at 2:28 PM
This is what I'm seeing now, FYI
Unhandled exception on thread
Traceback (most recent call last):
  File "c:\Program Files (x86)\IronPython 2.7\lib\site-packages\ptvsd\", line 1587, in connect_to_repl_backend_using_socket
  File "c:\Program Files (x86)\IronPython 2.7\lib\site-packages\ptvsd\", line 1025, in __init__
  File "c:\Program Files (x86)\IronPython 2.7\lib\site-packages\ptvsd\", line 537, in __init__
KeyError: __main__
Jan 16, 2015 at 2:51 PM
FYI I was able to fix this by creating my primary execution scope as a module rather than just an anonymous scope, like so
var scope = IronPython.Hosting.Python.CreateModule(scriptEngine, "__main__");
Kinda unfortunate that this is required, but it'll work. My debugging all appears to be working perfectly now, thank you so much for your help!