12

Resolved

Autocompletion does not work for modules specified in PYTHONPATH

description

There are custom python modules which are specified in PYTHONPATH environment variable.
These modules are imported but autocomplete does not work

comments

Zooba wrote Feb 27, 2013 at 10:42 PM

We'll look into adding a way to add the global PYTHONPATH to a project's list of search paths.

At the moment you can manually add $(PYTHONPATH) to the <SearchPath> element of the project file, but any changes to the search paths made through Visual Studio will expand the variable before saving. Autocomplete works fine with this approach.

We're very unlikely to make it always be included, since that will interfere with having multiple versions of Python installed. We also set PYTHONPATH based on the search paths, which further complicates our ability to automatically use a global value.

MenkHilroy wrote Feb 28, 2013 at 7:23 AM

Its not convenient to include PYTHONPATH to the SearchPath for each project.

I have a fix:
PythonScraper.py at the end:
print "Parsing PYTHONPATH modules"
pythonpath = os.getenv('PYTHONPATH')
if pythonpath:
    for p in pythonpath.split(";"):
        print p
        for root, dirs, files in os.walk(p):
            package_inspector(p, root, files)
PythontypeDatabase.cs in BuildArgumens method:
 var pythonPath = Environment.GetEnvironmentVariable("PYTHONPATH");
        if (!string.IsNullOrEmpty(pythonPath))
        {
            foreach (var path in pythonPath.Split(';'))
            {
                args += " /dir \"" + path + "\"";
            }
        }

Zooba wrote Feb 28, 2013 at 3:30 PM

That works fine if you only have one version of Python installed, or if your PYTHONPATH happens to apply to all versions (including interpreters other than CPython), but because we support people who use multiple versions of Python we can't use a global variable when analyzing libraries.

Also, if you debug scripts through PTVS then we will overwrite the global PYTHONPATH variable (see DefaultPythonLauncher.SetupEnvironment). The best way right now to ensure those paths end up in sys.path is to add them to your project's search paths, and the only way to use the global var is the workaround I mentioned below.

Ultimately, PYTHONPATH is designed for being set immediately prior to running a script (as is typical in the *nix world), and not for Windows's "set once and use forever" approach to environment variables. For people using PTVS and Python with version control systems, it makes much more sense to have the paths in the project file.

We may consider adding a global option to "Always include PYTHONPATH in project search paths", but this would be off by default. It would, however, apply to all projects automatically. Does that sound better? Certainly having a context menu item on "Search Path" that reads "Add global PYTHONPATH" is something that we can do, but you'd have to do that for each project.

There's also issue 760, which would let you add a custom interpreter and set a value for it's PYTHONPATH. This would not automatically use your global setting, but it would make it a little easier to share between projects.

MenkHilroy wrote Feb 28, 2013 at 5:00 PM

I have more complex problem. We deploy VS + PyTools to customer as Python IDE.
Out product consists of set of python modules which cannot be installed into Python because customer uses several different versions of product simultaneously.
So our solution was to deploy VS shortcut with VS Command which prepare PYTHONPATH enviroment to our modules location for specific version of the product. Its should not be user problem to specify PYTHONPATH or modules location.

MenkHilroy wrote Feb 28, 2013 at 5:19 PM

In general I think that PYTHONPATH is global variable so therefore the user knowingly sets up it for all interpreters. PyTools should supports it as written in the documentation, and there is no need for additional logic. If the user wants to use different interpreters then there is no reason to set a global environment variable

dinov wrote Feb 28, 2013 at 5:44 PM

One possible work around for you might be to deploy the generated completion database with your app as well. The DB gets deployed to %LOCALAPPDATA%\Python Tools\CompletionDB. You could install your packages into your local Python install, generate the DB, and then when you install deploy these files.

You could even look at deploying your completion files as the default completion database which lives in the CompletionDB folder where PTVS is installed (http://pytools.codeplex.com/wikipage?title=Where%20is%20PTVS%20installed%3f).

Zooba wrote Feb 28, 2013 at 6:21 PM

Ah, I see what you're trying to achieve. I don't think Dino's workaround will suffice unless the different paths have different sets of modules (i.e. different names). And we're still not going to pass it to the debugger, though it will implicitly flow through if no search paths have been specified.

If we added the global option to always include PYTHONPATH like I described in my last comment, you would be able to turn this on as part of your deployment process. Your users would be able to see the paths, but could not change them (except by disabling the option). Would this meet your needs?

MenkHilroy wrote Feb 28, 2013 at 6:24 PM

Dinov. Ok, but what about CompletionDB version for each product version?
Also in our product there is a user site code generation tool which generates python interface module.
So number of python modules increases dynamicaly and PyTools does not track changes and autocompletion does not work for these modules.
And one additional issue that user can't see autocomletion db generation progress which may confuse user why autocompletion does not work on first usage.

MenkHilroy wrote Feb 28, 2013 at 6:37 PM

Zooba. If we set global option how user can run VS for specific product version (specific PYTHONPATH)?
PS. We have already solved all these problems by changing the PyTools, I just want to improve official version.

dinov wrote Feb 28, 2013 at 7:23 PM

There's actually separate completion DBs for each Python version, so you could deploy your files to all the supported versions. But maybe that's starting to get out of hand now :)

We do have some plans to auto-regenerate the completion DB when site-packages changes and make it more obvious that the completion DB is being regenerated.