2
Vote

No support for Canopy Venv ?

description

Hi,

Canopy virtual environment (venv) seems incompatible with the virtualenv support in PTVS (despite venv seems supported).

I did the following:
  1. created a venv using Canopy: 'canopy_cli.exe venv d:\temp\venv'
  2. tried adding 'd:\temp\venv' in an existing PTVS project via Python Environments > Add an existing virtual environment in Solution explorer.
I got the following error:
We could not find a base interpreter at C:\Program Files (x86)\Enthought... (see attached)

When I use CPython 3.3.1 (32-bit) to create a venv, I can select it from PTVS without an issue.

I used
  • Visual Studio Premium 2012 Update 1 (11.0.51106.01)
  • Python Tools for Visual Studio (2.1.20211.00)
  • Canopy 1.3.0 (32-bit)

file attachments

comments

Zooba wrote Apr 7 at 11:10 PM

I have spoken with the developers of Canopy in the past about their special venv implementation - it is not the same as virtualenv (for Python 2.x) or the venv in the standard library - but last time we tested it there was no problem.

If you open the virtual environment folder do you see a pyvenv.cfg file? If not, under Lib do you see an orig-prefix.txt file? If so, the file will contain the C:\Program Files... path in the error message - can you share exactly what that path is? As I understand, Canopy does not normally install into Program Files, so we may be miscalculating the path for some reason.

An alternative approach that may work: you should be able to create a new virtual environment using our UI. We will detect and use a package called venv if it is there, and we don't do anything to distinguish between Canopy's version and the official version.

PetrWolf wrote Apr 8 at 1:37 AM

Hi Zooba!

The file pyvenv.cfg exists and contains:
home = C:\Program Files (x86)\Enthought\Canopy32\App\appdata\canopy-1.3.0.1715.win-x86
include-system-site-packages = false
version = 2.7.6
The path above is valid and corresponds to a system-wide installation of Canopy.

An attempt to build a Canopy venv from within Visual Studio leads to the attached dialog, which seems to suggest that virtualenv should be installed. I did not proceed with that.

Zooba wrote Apr 8 at 4:11 PM

I guess they've changed it in their latest version, since it used to work fine with our normal support.

Our entire team is out right now (either on leave or at PyCon), so we won't get a chance to take a look for a week or two.

As a workaround for you, you can configure a custom environment for your venv (see our Python Environment docs). Some completions from the standard library will be missing, but debugging/etc. should be fine.

PetrWolf wrote Apr 8 at 6:16 PM

Actually, I sent a link for this thread to Enthought.
Below you can find what Jason McCampbell replied:
We provide a plug-in to PTVS that allows it to automatically recognize our User virtual environment based on some registry keys we set, but we don't currently have a means of detecting alternate virtual environments. We haven't had a request for it before, but I have logged your request so we can look into doing so.
That said, you can manually add the virtual environment through Visual Studio. What you need to do is:
Go to Tools -> Options and select 'Python Tools' and 'Environment Options' under it.
Click "Add Environment"
Fill in the settings in the dialog were "Path" refers to the path to python.exe, "Windows path" refers to the path to "pythonw.exe", and "Library path" should go to the "Lib" directory. The "Path environment variable" should be "PYTHONPATH".
Example:
venv -s c:\temp\test
Path: c:\temp\test\python.exe
Windows path: c:\temp\test\pythonw.exe
Lib path: c:\temp\test\Lib
Architecture: <depends on Canopy version>
Language Version: 2.7
Path Environment Variable: PYTHONPATH
Later, Jason added:
The primary technical issue with supporting multiple venv's in PTVS that we need to figure out is how to handle registration and de-registration. That is, for PTVS to find them we need to add them to the registry or note them somewhere else. But then that also opens up the possibility of creating a lot of "garbage" if someone creates a venv with 'venv' but deletes using Window's Explorer. Not a big technical hurdle, more a matter of getting the right interface.
So this sounds like a topic to be discussed between Enthought and PTVS.

Zooba wrote Apr 8 at 11:20 PM

We've discussed it before, and I believe Enthought already have enough from us to make it work (as in, it doesn't require changes to PTVS). Unsurprisingly (to me), Jason knows exactly what needs to be done here, and can also see the issues. He has my contact details, so if he needs support or changes I'm sure he'll be in touch. But it does sound like it's simply not been requested before - we plan using a similar approach.

That said, it sounds like they may have made significant changes to their venv; it used to work with our normal virtual environment support. When I get a chance I'll try it out and see if we do need to fix something on our side.

pminaev wrote Jun 4 at 10:27 PM

Here's what I get when trying to create a new virtual environment on Canopy 1.4:
C:\Users\pminaev\AppData\Local\Enthought\Canopy\User\python.exe: No module named pip.__main__; 'pip' is a package and cannot be directly executed
Virtual environment is being created at 'c:\users\pminaev\documents\visual studio 2013\Projects\PythonApplication1\PythonApplication1\env'
C:\Users\pminaev\AppData\Local\Enthought\Canopy\User\python.exe: No module named virtualenv
Virtual environment was not created at 'c:\users\pminaev\documents\visual studio 2013\Projects\PythonApplication1\PythonApplication1\env'. Exit code: 1

PetrWolf wrote Jun 5 at 9:09 PM

Yes - this is part of the issue. PTVS treats Canopy as a 2.7 python and tries to install and use virtualenv package. Yet Canopy is ased on venv, backported from python 3 and so it should be treated accordingly.

Zooba wrote Jun 6 at 12:56 AM

The problem may be that we only look at the top layer of library for venv. If Canopy has venv in their base environment then we won't see it and will try to install virtualenv (venv is definitely preferred if it's there).

Failing to launch python.exe -m pip is a separate issue and we may just have to extend our workaround to all versions of Python rather than just the known-problematic versions.

Zooba wrote Jul 15 at 4:33 PM

Hi PetrWolf

I was able to sit down at SciPy recently with Jason from Enthought and we went through and figured out a few issues with both of our products that are causing problems.

A bug on their side is that venv is setting the system Python as the home path in pyvenv.cfg, which PTVS doesn't actually know about. He said they were going to fix that anyway to point to the user Python (which will include any updated packages). Once that happens, PTVS should be able to use the venv as expected (or you can change the path yourself).

I believe the failing to launch pip issue is fixed with a recent update to Canopy - they have a newer version of pip (from 1.0.1 to 1.5.4, IIRC).

We've also made a fix to PTVS (in 2.1 Beta 2) that will correctly detect and use Canopys venv if the completion databases are up to date. Jason said he'd look into making a small change on their side so that we can easily detect that venv is there even if the DB is not up to date. This should keep us from trying to install virtualenv.

Canopy also has a way to register virtual environments (
canopy_cli register-venv %path%`, IIRC) which will then be detected and displayed inside PTVS. This side of things is handled by an extension to PTVS that ships with Canopy, so I have no insight into exactly how it works, but I assume Jason has that covered.

So the shortest answer is: update everything and you'll be fine (that's a very Microsoft answer isn't it... sorry ;) )

PetrWolf wrote Jul 25 at 7:44 PM

Hi Zooba!

I upgraded everything (details below) and it works! Well, almost :)

What works:
  • manually editing pyvenv.cfg to point at a User environment and then adding it to PTVS using "Add Existing Environment", or
  • manually adding the "base" Canopy python as an environment in PTVS and then adding a venv without modifying pyvenv.cfg (not that it is valid to have venv's inheriting from the base, not from User)
  • creating a new venv using Add Existing Virtual Environment
  • PyLint runs (brilliant!)
What doesn't:
  • register-venv seems to have no effect (other than creating a registry entry).
  • Mixed-mode debugging in a venv hits no breakpoints in python code
Please take a look, but anyway, this is great progress! Thanks!

Petr

PTVS 2.1.20620.00
VS 2012 Premium
Canopy 1.4.1.1975 (32-bit)
Windows 7