"Python Tools for Visual Studio background analyzer has stopped working"

May 6, 2013 at 11:25 PM
When I click the Generate Intellisense Database button on Tools->Python Tools->Interpreter Options, after a while the following crash dialog message appears:

"Python Tools for Visual Studio background analyzer has stopped working".

I have quite a few extra packages installed including:
beautifulsoup4-4.1.3-py2.7.egg
jinja2-2.6-py2.7.egg
matplotlib
mechanize-0.2.5-py2.7.egg
numpy
PyQt4
Sphinx-1.1.3-py2.7.egg
I don't know if any of those can cause a problem?

Windows 7 64bit, VS2012 Update 2, PTVS 2.0 Alpha
ActivePython 2.7.2.5 (ActiveState Software Inc.) based on
Python 2.7.2 (default, Jun 24 2011, 12:22:14) [MSC v.1500 64 bit (AMD64)] on win32
Coordinator
May 6, 2013 at 11:42 PM
We test most of these, so they shouldn't be giving any trouble. If you open (in VS) Tools->Python Tools->Diagnostic Information there may be some hints towards the end - if not, you can copy it into a text file and email it to ptvshelp@microsoft.com and we can take a look.

There may also be helpful information (for us) in your event log - VS quite often manages to write a stack trace. Also, if VS wants to send a crash report back, please do it. Those are incredibly helpful for us.
May 7, 2013 at 6:13 PM
Not sure that this is useful but the Event viewer shows this:
Faulting application name: Microsoft.PythonTools.Analyzer.exe, version: 2.0.60306.0, time stamp: 0x5137d3d3
Faulting module name: clr.dll, version: 4.0.30319.18034, time stamp: 0x50b5a8bb
Exception code: 0xc00000fd
Fault offset: 0x00071a96
Faulting process id: 0x1320
Faulting application start time: 0x01ce4aaeeaaf603b
Faulting application path: C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\Microsoft.PythonTools.Analyzer.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
Report Id: 56bfcfb8-b6a2-11e2-8069-005056c00008
and the Python Tools for Visual Studio - Diagnostic Info says at the end:
Traceback (most recent call last):
      File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\ExtensionScraper.py", line 36, in <module>
        __import__(mod_name)
    ImportError: DLL load failed: %1 is not a valid Win32 application.
    __import__("matplotlib.backends._gtkagg")
    Traceback (most recent call last):
      File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\ExtensionScraper.py", line 36, in <module>
        __import__(mod_name)
    ImportError: DLL load failed: %1 is not a valid Win32 application.
    __import__("matplotlib.backends._tkagg")
    Traceback (most recent call last):
      File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\ExtensionScraper.py", line 36, in <module>
        __import__(mod_name)
    ImportError: DLL load failed: The specified module could not be found.
    __import__("pil-1.1.7-py2.7-win-amd64.egg._imaging")
    Traceback (most recent call last):
      File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\ExtensionScraper.py", line 36, in <module>
        __import__(mod_name)
    ImportError: No module named pil-1.1.7-py2.7-win-amd64.egg._imaging
    __import__("pil-1.1.7-py2.7-win-amd64.egg._imagingmath")
    Traceback (most recent call last):
      File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\ExtensionScraper.py", line 36, in <module>
        __import__(mod_name)
    ImportError: No module named pil-1.1.7-py2.7-win-amd64.egg._imagingmath
    Traceback (most recent call last):
      File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\ExtensionScraper.py", line 54, in <module>
        analysis = PythonScraper.generate_module(module)
      File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\PythonScraper.py", line 329, in generate_module
        all_members[attr] = generate_member(attr_value)
      File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\PythonScraper.py", line 198, in generate_member
        member_table['value'] = generate_type(obj, is_hidden=is_hidden)
      File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\PythonScraper.py", line 262, in generate_type
        type_table['mro'] = oldstyle_mro(type_obj, [])
      File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\PythonScraper.py", line 248, in oldstyle_mro
        for base in type_obj.__bases__:
    TypeError: C++ type 'QWidget*' is not supported as  type
    __import__("win32com.axdebug.axdebug")
    Traceback (most recent call last):
      File "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\ExtensionScraper.py", line 36, in <module>
        __import__(mod_name)
    ImportError: DLL load failed: The specified module could not be found.
    osoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.0\CompletionDB\imp.idb

"2013/05/06 4:10:36.189 PM" "F:\Python27\python.exe" "C:\Users\abc\AppData\Local\Python Tools\CompletionDB\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" "DONE (SCRAPE)"
"2013/05/06 4:10:36.191 PM" "F:\Python27\python.exe" "C:\Users\abc\AppData\Local\Python Tools\CompletionDB\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" "START_STDLIB /dir "F:\Python27\Lib" /version V27 /outdir "C:\Users\abc\AppData\Local\Python Tools\CompletionDB\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /indir "C:\Users\abc\AppData\Local\Python Tools\CompletionDB\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7""
"2013/05/06 4:22:47.674 PM" "F:\Python27\python.exe" "C:\Users\abc\AppData\Local\Python Tools\CompletionDB\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" "FAIL_STDLIB -1073741571"
Coordinator
May 8, 2013 at 5:47 PM
Unfortunately you're right, and that information isn't very helpful. Crashes inside clr.dll are hard to diagnose, and we haven't had any other reports like this one come in yet.

I'm about to start a big push at making sure that we never crash (or get stuck in an infinite loop, another common issue) in the analyzer, so I'll make sure to include these packages in my test data. Hopefully that will flush out whatever issue is causing the crash you're seeing.
Jul 23, 2013 at 3:36 AM
I think I'm having a related issue (though not a crash). On Windows 8 Pro 64-bit, the process 'Python Tools for Visual Studio background analyzer (32 bit)' runs often and for a very long time, even after PTVS has exited. In PTVS, the Python Interpreters view shows it is gradually Refreshing DB, but it often takes over an hour. This tends to be CPU intensive. But more to the point, I'm not sure why it's running so often, and don't see a way to get more details. Any help is appreciated. If there are more details I could provide, please let me know.
Jul 23, 2013 at 7:45 PM
I am running on a Windows 7 home premium with a 64-bit processor, and I am having a similar problem. This started once I switched from PTVS 1.5 to 2.0 beta. Once I run a python solution on visual studios the Microsoft.PythonTools.Analyzer.exe *32 will max out my memory. Even after I close VS, Microsoft.PythonTools.Analyzer.exe *32 will still consume almost half of my memory. The only solution I have come across is to restart my computer. Any suggestions would be great.
Coordinator
Jul 23, 2013 at 10:14 PM
The analyzer is supposed to take a lot of CPU and memory - this is why we only run it once and do it in the background. If it is taking hours, you may be encountering a known (and fixed in our next update) issue related to .pyd files. Commonly, the .pyd files in scipy will cause the analyzer to take hours and hours to run, to the point where it is not worth trying.

A workaround is to move any .pyd files out of your C:\Python##\Lib\site-packages directory (and subdirectories) while running your analysis, then put them back afterwards. If you put an empty .py file there with the same name while analysis runs, then we shouldn't tell you that you need to refresh the database you replace them with the original .pyds.

And it's relatively safe to kill Microsoft.PythonTools.Analyzer.exe with Task Manager - obviously we'll try and run it again next time we need the analysis, since it didn't complete, but if you need the CPU/RAM more than the analysis, it can be killed.
Coordinator
Jul 23, 2013 at 10:23 PM
Or you can just rename the .pyd files to another extension (other than .py, .pyw or .pyd) while running the analysis. Then we'll ignore them.

To quickly do this for your entire site-packages, you can use these commands in cmd.exe:
cd C:\Python27\Lib\site-packages
for /R %f in (*.pyd) do @rename %~f %~nf.pyd.bak

# refresh the DB through Visual Studio

for /R %f in (*.pyd.bak) do @rename %~f %~nf.pyd
Jul 28, 2013 at 5:12 PM
I have the same problem (with both WinPython and Anaconda) and I tried the solution above. It did not work. About 25% of CPU time was being used for no effect. So I renamed Microsoft.PythonTools.Analyzer.exe and Microsoft.PythonTools.Analysis.dll and the problem stopped.

I also have the standard CPython interpreter and for that one I get "Completion DB is up to date". However, numpy, etc. are not installed so I cannot do much with that interpreter.

I still get code suggestions/completions in WinPython interactive for numpy, etc in IPython, but not in the editor. I get code completion for standard functions like print() in the editor.

I switched recently to PTVS and I think it is the best thing since sliced bread. I hope this gets fixed soon.

Thanks.

FS
Jul 28, 2013 at 7:24 PM
Zooba's shell commands to backup site-packages did not work for me, also. In fact, they hosed my site-packages and I had to re-install several of them after that. I want to second everything Soldalma just posted.
Jul 28, 2013 at 8:30 PM
I Also tried the solution above and the problem was still not resolved. For some reason, whenever I kill Microsoft.PythonTools.Analyzer.exe with Task Manager, it starts back up again automatically. This is very strange. The only solution that completely kills Microsoft.PythonTools.Analyzer.exe is a system restart, which is not very practical.

I switched back to PTVS 1.5 and the problem stopped. I can't complain - PTVS 1.5 is absolutely incredible (particularly intellisense) and greatly increases the amount of work I can get done. However, I would like to see this fixed because there are features of PTVS 2.0 that I can see making my life even easier.
Jul 28, 2013 at 11:08 PM
You can kill Microsoft.PythonTools.Analyzer.exe by renaming it and the related DLL file. I did it and there were no ill effects on PTVS. It should be an option for the user to disable the analyzer. It shouldn't be difficult to implement.
Coordinator
Jul 28, 2013 at 11:26 PM
The ill effects are that you won't get completion information beyond the tiny amount that we ship with the product. We do have some known issues in the beta that are already fixed and will be in our next update (if you want to build PTVS from source you can have them sooner). If you need to go back to the last stable release (1.5) until then, that's fine. If you desperately want to delete or rename files, I obviously can't stop you, but I'd recommend going back to an earlier release instead.

The interactive window uses a different method for getting information about Python objects, which is why you get some information without letting the analyzer finish, though you won't get function parameter types or return types.

Can you tell me the exact version of Anaconda and WinPython you're using (download links are great)? We can't possibly test with everything available on the internet, so focusing on packages where we know there are issues is helpful.
Coordinator
Jul 29, 2013 at 4:29 PM
You can also prevent the analyzer from automatically starting by opening Tools->Options->Python Tools->Advanced and deselecting "Automatically analyze standard library in background".
Aug 10, 2013 at 6:19 AM
Edited Aug 10, 2013 at 6:20 AM
Zooba wrote:
Or you can just rename the .pyd files to another extension (other than .py, .pyw or .pyd) while running the analysis. Then we'll ignore them.

To quickly do this for your entire site-packages, you can use these commands in cmd.exe:
cd C:\Python27\Lib\site-packages
for /R %f in (*.pyd) do @rename %~f %~nf.pyd.bak

# refresh the DB through Visual Studio

for /R %f in (*.pyd.bak) do @rename %~f %~nf.pyd
Hi Zooba:

That last line appends an extra ".pyd" onto the file names. The following works for me:
cd C:\Python27\Lib\site-packages
for /R %f in (*.pyd) do @rename %~f %~nf.pyd.bak

# refresh the DB through Visual Studio

for /R %f in (*.pyd.bak) do @rename %~f %~nf
Coordinator
Aug 10, 2013 at 1:41 PM
Thanks for the correction!