Refreshing Database does not complete properly

Jan 18, 2015 at 3:54 PM
Hello,
I was using PVTS for a while almost 2 months and it worked perfectly fine :D Good Job Devs.
Now due to some reason i had to re-install windows 8 and Visual Studio 2013 (its on update 3).then i installed PVTS same version as before (one i downloaded 2 months back) and installed Canopy with python 2.7 . But now refreshing database never completes correctly.
As i press refresh database every time (tried 3 times so far) . Ok some more info first , my Environments tab is showing 2 as before Canopy and python 2.7 where canopy is default. Before pressing Refresh DB Canopy says "Base interpreter is out of date: \n currently regenerating"  when pressed "?" and Python 2.7 says "Database at C:\Users\***\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7 is regenerating" .
After pressing Refresh DB on Python 2.7 :
                        1) ToolTip Label (Floating when MouseOver) says "Collecting files" and returns to same state as before 
After Pressing Refresh DB on Canopy: (Text from ToolTip Label)
Note:(this procedure takes almost 20 mins with around 500 mb ram and 40% cpu)
                         1) Cleaning old files
                         2) Some kind of reading packages (not sure about string)
                         3) Analyzing package "Package Name" ....
                         4) Goes from packages a to z then some starting with _
                         5) Returns to same state as before
Any Ideas what may be going wrong ?
Jan 18, 2015 at 3:59 PM
Edited Jan 18, 2015 at 4:41 PM
More info AnalysisLog log is almost 1.4 mb text file with last part suspicious:-
Starting
2015-01-17T20:16:09: Start analysis
2015-01-17T20:16:30: IncludeModulesFromModulePath = win32com
2015-01-17T20:16:33: Deleting 0 files
2015-01-17T20:16:33: Scraping builtin modules
2015-01-17T20:16:33: Command: C:\Users*\AppData\Local\Enthought\Canopy\User\python.exe "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\PythonScraper.py" "C:\Users*\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB"
2015-01-17T20:16:38: Output
Middle part skipped as it looks all OK
2015-01-18T21:10:05: Start analysis
2015-01-18T21:10:27: IncludeModulesFromModulePath = win32com

2015-01-18T21:13:15: Start analysis
2015-01-18T21:13:16: IncludeModulesFromModulePath = win32com

2015-01-18T21:16:13: Start analysis
2015-01-18T21:16:14: IncludeModulesFromModulePath = win32com

2015-01-18T21:49:42: Start analysis
2015-01-18T21:49:43: IncludeModulesFromModulePath = win32com

2015-01-18T21:49:44: Start analysis
2015-01-18T21:49:45: IncludeModulesFromModulePath = win32com

2015-01-18T21:49:46: Start analysis
2015-01-18T21:49:46: IncludeModulesFromModulePath = win32com

2015-01-18T21:49:47: Start analysis
2015-01-18T21:49:48: IncludeModulesFromModulePath = win32com

2015-01-18T21:49:49: Start analysis
2015-01-18T21:49:49: IncludeModulesFromModulePath = win32com

2015-01-18T21:51:47: Start analysis
2015-01-18T21:51:48: IncludeModulesFromModulePath = win32com

2015-01-18T21:51:49: Start analysis
2015-01-18T21:51:50: IncludeModulesFromModulePath = win32com

2015-01-18T21:56:48: Start analysis
2015-01-18T21:56:49: IncludeModulesFromModulePath = win32com

2015-01-18T22:10:24: Start analysis
2015-01-18T22:10:25: IncludeModulesFromModulePath = win32com

2015-01-18T22:20:05: Start analysis
2015-01-18T22:20:06: IncludeModulesFromModulePath = win32com
There is one more folder with some random name having Analysis Log file of almost 9 mb
it contains "Starting analysis of 96 modules" 4 times (Yes i have refreshed database 4 times so far)

and one in root of Completion DB folder:-
which has
2015-01-17T20:16:09 START_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Users*\AppData\Local\Enthought\Canopy\User\python.exe /library C:\Users*\AppData\Local\Enthought\Canopy\User\lib /outdir "C:\Users***\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users*\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users*\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
this one is there many times 10 to 15




Edit:-
I re installed PTVS 2.1
Refreshed DB of Python 2.7
Log file contains only 1 error
2015-01-18T22:51:00: Scraping dde
2015-01-18T22:51:00: Command: C:\Users*\AppData\Local\Enthought\Canopy\User\python.exe "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\ExtensionScraper.py" scrape dde C:\Users*\AppData\Local\Enthought\Canopy\User\lib\site-packages\Pythonwin "C:\Users***\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\site-packages_Pythonwin\dde"
2015-01-18T22:51:00: Output
__import__("dde")
2015-01-18T22:51:00: [WARNING] Errors
Traceback (most recent call last):
File "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\ExtensionScraper.py", line 43, in <module>
__import__(mod_name)
ImportError: This must be an MFC application - try 'import win32ui' first
2015-01-18T22:51:00: [ERROR] Failed to scrape dde (Exit code: 1)
Jan 18, 2015 at 4:44 PM
The Main Log file in CompletionDB\12.0 contains
2015-01-18T22:50:54 START_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\python.exe /library C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\lib /outdir "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
FAIL_STDLIB: (-3) "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\python.exe /library C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\lib /outdir "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
This interpreter is already being analyzed.
2015-01-18T23:02:24 DONE_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\python.exe /library C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\lib /outdir "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
2015-01-18T23:07:48 START_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\python.exe /library C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\lib /outdir "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
2015-01-18T23:07:52 DONE_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\python.exe /library C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\lib /outdir "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
2015-01-18T23:08:00 START_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\python.exe /library C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\lib /outdir "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
2015-01-18T23:08:05 DONE_STDLIB "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\Microsoft.PythonTools.Analyzer.exe" /id {9a7a9026-48c1-4688-9d5d-e5699d47d074} /version 2.7 /python C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\python.exe /library C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\lib /outdir "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7" /basedb "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\2.1\CompletionDB" /log "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\9a7a9026-48c1-4688-9d5d-e5699d47d074\2.7\AnalysisLog.txt" /glog "C:\Users\Ninad\AppData\Local\Python Tools\CompletionDB\12.0\AnalysisLog.txt" /wait ""
Coordinator
Jan 19, 2015 at 11:31 PM
Normally this would mean that one of the DB files is missing at the end of refresh, though it can be really hard to tell which one. Sometimes the log will list modules that it's deleted at the end, but not always.

The best suggestion I can offer is to delete the AppData\Local\Python Tools folder and refresh all your DBs. It will take a while (Canopy has a lot of packages), but it's the easiest way to clean up any corruption.

(I do plan to include some more helpful diagnostics in our next release, but that's a way off yet and it won't help you right now.)
Jan 23, 2015 at 5:43 AM
Edited Jan 23, 2015 at 6:00 AM
Sry for late reply i was bit busy , the log file do says "deleted 1 module or something" at end of refresh but does not specify which one . I will delete DB and refresh to let you know right away.

Edit:-
It just regenerated DB , yet same problem
2015-01-23T12:24:15: Analysis complete
2015-01-23T12:24:15: Saving group "C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\lib\site-packages\win32comext"
2015-01-23T12:24:15: End of group "C:\Users\Ninad\AppData\Local\Enthought\Canopy\User\lib\site-packages\win32comext"
2015-01-23T12:24:22: Deleting 1 files

2015-01-23T12:27:51: Start analysis
2015-01-23T12:27:51: IncludeModulesFromModulePath = win32com
2015-01-23T12:27:55: Deleting 0 files
Coordinator
Jan 28, 2015 at 12:16 AM
I can see the same deletion at the end of an analysis run for Canopy as well, so I'll try re-running with more diagnostics and see if I can figure out what's being removed. I suspect it is something that we need...