Python debugging doesn't import


Here's the problem, I have this snippet:
import urllib.request
url = 'http://www.pythonchallenge.com/pc/def/linkedlist.php?nothing=12345'
filehandle = urllib.request.urlopen(url)
for lines in filehandle.readlines():
    print(lines) #returns b'and the next nothing is 44827'
and it works just fine...unless I'm debugging it.
As soon as I step into import urllib.request, or if I put my break point below and then step into
filehandle = urllib.request.urlopen(url), I get an error from MS Visual Studio (not from Python)

Where disassembly says 'No disassembly available.'

Just to be clear, this issue DOEN'T COME UP:
  • if my first breakpoint is AFTER statements using urllib.request
  • when I use math module (to make sure that it isn't just all the modules)
I searched the net, but nothing came up.

Any help will be greatly appreciated
Closed Jan 16 at 11:39 PM by pminaev


pminaev wrote Jan 2 at 7:51 PM

Stepping is disabled by default for standard library code; the expected behavior for both of these is actually for "step in" to work as "step over" (unless you call one of your own functions in a parameter expression, then it should step into those). This is controlled by the setting in Tools -> Options -> Python Tools -> Debugging -> Enable debugging of the Python standard library.

The reason why this breaks is because urllib seems to be using namedtuple internally as part of its initialization code, and namedtuple uses exec. Here's what the stack looks like in my repro of this:
>   <exec or eval> line 1   Python
    namedtuple in __init__ line 367 Python
    parse module line 225   Python
Since <exec or eval> frame is not considered a part of the standard library, step-in stops at it. On the other hand, because it is an exec/eval frame, the debugger cannot obtain the source code for it (the code object associated with the frame doesn't have a filename), hence why you're seeing this message.

I'm not sure what the correct fix for this particular problem would be. In this particular case, we clearly want F11 to step over the exec code (if the appropriate setting is enabled), as it is really a part of the standard library. However, there is nothing identifying it as such a part. We could walk the stack, and, if the topmost frame is exec/eval, and the one preceding it is stdlib, then assume that exec/eval is also stdlib - but that is not necessarily correct, as stdlib code can be given a user callback, and that callback can be generated by exec/eval.

On the other hand, the utility of stepping into exec/eval code in the first place is questionable, since you don't see the source code for it (and it's not something that we can fix - Python simply doesn't provide this info), and can't set breakpoints. So the only thing you get out of it is the ability to inspect & edit locals and use REPL in its context. Perhaps we should have a separate setting for stepping into exec/eval that would be disabled by default?

Zooba wrote Jan 2 at 8:23 PM

I'd phrase it as "Automatically step over modules when source code is not found", but the idea for the option is good.

Even better would be if we could disassemble (or even just display) the bytecode, but that's very CPython specific and probably more work than it's worth. I think we'd take that as a contribution though, if it were done well.

pminaev wrote Jan 2 at 8:26 PM

I created a separate item to track that feature: